Road traffic incident video detection method and system

Abstract

The invention relates to the field of traffic event detection, and particularly discloses a road traffic event video detection method and system, and the method comprises the following steps: processing a road monitoring video through a video feature analysis and cross-domain knowledge fusion frame, and obtaining a structured traffic event processing knowledge base; traffic video data are processed through video scene depth analysis and a multi-party collaborative reasoning system, and a multi-dimensional decision analysis report is obtained; processing historical video event data through a video event analysis and decision knowledge adaptive learning system to obtain an optimal practical knowledge model; processing a video detection result and multi-department decision suggestion data through video event evidence fusion and a multi-party conflict coordination system to obtain a comprehensive balance decision scheme; according to the road traffic incident video detection method, high-precision automatic detection and recognition of the road traffic incident are realized by combining a deep video analysis technology and a cross-domain knowledge base, and the processing efficiency is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of traffic incident detection, and more specifically, to a road traffic incident video detection method and system. Background Art

[0002] With the acceleration of urbanization and the rapid growth of motor vehicle ownership, road traffic incidents are frequent, bringing huge challenges to traffic management departments. In particular, multi-vehicle collisions in complex traffic environments, large-scale congestion caused by bad weather, and major traffic accidents often require the coordinated handling of multiple departments such as traffic management, emergency rescue, medical first aid, and insurance claims, which puts higher requirements on the efficiency of incident handling.

[0003] Traditional road traffic incident handling mainly relies on manual video monitoring and on-site investigation, and has the following problems: low utilization rate of video monitoring data: a large amount of road monitoring video data has not been effectively analyzed, and event detection mainly relies on manual inspections, which is prone to delayed discovery or omissions; serious information island phenomenon: the lack of effective information sharing mechanism between departments leads to inconsistent decision-making basis and low processing efficiency; knowledge and experience are difficult to accumulate: the experience and best practices of handling different types of events are difficult to form standardized knowledge, and decision-making support cannot be provided for similar events; frequent multi-party decision-making conflicts: each department makes decisions based on its own responsibilities and goals, often resulting in problems such as difficulty in coordination and decision-making conflicts, which prolongs the time for event processing. Summary of the invention

[0004] The present invention provides a road traffic incident video detection method and system to solve the technical problems in related technologies that are difficult to cope with complex and changeable actual traffic scenes, lack of deep integration of professional field knowledge and support for multi-party collaborative decision-making.

[0005] A first aspect of the present invention provides a road traffic incident video detection method, comprising: Step 100: Process road surveillance videos and multi-source professional domain data through video feature analysis and cross-domain knowledge fusion framework to obtain a structured traffic event processing knowledge base; Step 200: Based on the structured traffic event processing knowledge base, traffic video data is processed through video scene in-depth analysis and a multi-party collaborative reasoning system to obtain a multi-dimensional decision analysis report; Step 300: Based on the structured traffic event processing knowledge base and the multi-dimensional decision analysis report, the historical video event data is processed by the video event analysis and decision knowledge adaptive learning system to obtain a best practice knowledge model; Step 400: Based on the multi-dimensional decision analysis report and the best practice knowledge model, the video detection results and multi-department decision opinion data are processed through the video event evidence fusion and multi-party conflict coordination system to obtain a comprehensive and balanced decision plan.

[0006] Furthermore, the method includes the following steps: Step 101, processing traffic video data and professional domain document collections through video feature extraction and hierarchical ontology construction network to obtain domain ontology knowledge structure; Step 102, processing structured and unstructured data through a multi-level knowledge graph construction system to obtain a multi-domain knowledge graph; Step 103, processing multi-domain knowledge graphs through a three-stage knowledge alignment fusion network to obtain a unified traffic event processing knowledge base.

[0007] Furthermore, in step 101, the process of video feature extraction and hierarchical ontology network construction can be expressed as: ; ; in: Indicates The ontology knowledge structure of a field contains a set of concepts , Relationship Set and constraint rule set ; Indicates A collection of professional documents in various fields; A dictionary of terms representing the field; Represents road traffic incident video data; Represents traffic event features extracted from the video.

[0008] Furthermore, the method includes the following steps: step 201, processing traffic monitoring video data through a deep video analysis and feature extraction network to obtain a structured event feature representation; step 202, processing the structured event feature representation through a dual-path knowledge scene understander to obtain an event semantic scene graph; step 203, processing the event semantic scene graph through an adaptive multi-view report generation model to obtain a multi-dimensional decision analysis report.

[0009] Furthermore, in step 201, the processing process of the deep video analysis and feature extraction network can be expressed as: ; ; in: represents a traffic event detected from a video; Structured feature representation of events, including spatiotemporal features , Object Characteristics and relationship characteristics ; Represents traffic surveillance video data; Represents static image data; Represents sensor data such as radar; Represents geographic information data.

[0010] Furthermore, the method includes the following steps: Step 301, processing traffic video events and multi-party decision data through video event comparison and temporal decision tracking network to obtain a structured decision chain model; Step 302, processing the decision chain model and event result data through a multi-dimensional decision effect evaluation system to obtain a decision effect evaluation report; Step 303, processing the evaluation report and knowledge base data through an incremental knowledge update engine to obtain a best practice knowledge model.

[0011] Furthermore, in step 301, the processing of the video event comparison and timing decision tracking network can be expressed as: ; ; in: Represents historical traffic incident video data; represents the traffic event features extracted from historical videos; Represents a structured decision chain model, including video event features , decision node set , decision basis set and decision result set ; Indicates key decision points in the event handling process; Indicates the institutions or individuals involved in decision-making; represents the observations resulting from the decision; Represents the structured traffic incident processing knowledge base.

[0012] Furthermore, the following steps are included: Step 401, processing multi-department decision opinion data through video evidence evaluation and semantic hierarchical conflict detection engine to obtain a structured conflict analysis model; Step 402, processing the structured conflict analysis model through a constraint balance optimization algorithm to obtain a conflict balance solution; Step 403, processing the conflict balance solution through a collaborative decision solution generation system to obtain a comprehensive balance decision solution.

[0013] Further, in step 403, the processing of the video evidence evaluation and semantic level conflict detection engine can be expressed as: ; ; in: Indicates the video evidence assessment results, including the evidence reliability score and evidence relevance score ; Represents event semantic scene graph; Represents a structured conflict analysis model, including a set of conflict points , conflict type set and conflict severity assessment ; Indicates the decision opinions put forward by different departments based on multi-dimensional decision analysis reports; Represents a collection of multi-dimensional decision analysis reports; Represents a best practice knowledge model.

[0014] The second aspect of the present invention provides a road traffic incident video detection system, including: a video feature analysis and cross-domain knowledge fusion module, which is used to process road monitoring videos and multi-source professional field data to obtain a structured traffic incident processing knowledge base; a video scene depth analysis and multi-party collaborative reasoning module, which is used to process traffic video data and obtain a multi-dimensional decision analysis report; a video event analysis and decision knowledge adaptive learning module, which is used to process historical video event data and obtain a best practice knowledge model; a video event evidence fusion and multi-party conflict coordination module, which is used to process video detection results and multi-department decision opinion data to obtain a comprehensive and balanced decision plan.

[0015] The beneficial effects of the present invention are as follows: the present invention realizes high-precision automatic detection and identification of road traffic incidents by combining deep video analysis technology and a cross-domain knowledge base. The video detection and multi-party collaborative reasoning mechanism enable various departments to make decisions based on unified video evidence and professional knowledge, which significantly improves the efficiency of information sharing and the degree of decision-making coordination, significantly improves processing efficiency, reduces traffic congestion and secondary risks caused by accidents, provides objective event evidence through video detection, combines video event analysis with closed-loop optimization of decision-making knowledge and multi-perspective conflict coordination, continuously accumulates best practice experience and coordinates multi-party decision-making conflicts, and significantly improves the scientificity and objectivity of decision-making. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a flow chart of the steps of the method of the present invention. DETAILED DESCRIPTION

[0017] The subject matter described herein will now be discussed with reference to example embodiments. It should be understood that the discussion of these embodiments is only to enable those skilled in the art to better understand and implement the subject matter described herein, and the functions and arrangements of the elements discussed may be changed without departing from the scope of protection of the contents of this specification. Each example may omit, replace or add various processes or components as needed. In addition, the features described relative to some examples may also be combined in other examples.

[0018] like Figure 1 As shown, at least one embodiment of the present invention provides a road traffic incident video detection method, comprising the following steps: Step 100: Process road surveillance videos and multi-source professional domain data through video feature analysis and cross-domain knowledge fusion framework to obtain a structured traffic event processing knowledge base; This step uses the video feature analysis and cross-domain knowledge fusion framework to process road traffic monitoring videos and knowledge data from multiple professional fields such as traffic management, emergency rescue, medical first aid and insurance claims, and build a structured traffic incident processing knowledge base. The fusion framework consists of four parts: video feature extraction module, ontology construction module, knowledge graph construction module and knowledge alignment fusion module. The four modules are executed in sequence to form a complete video detection knowledge processing pipeline. The video feature extraction module first identifies and extracts event features from traffic monitoring videos; the ontology construction module extracts the conceptual structure of each field and passes the results to the knowledge graph construction module; the knowledge graph construction module further extracts entity relationships to form a domain knowledge graph; finally, the knowledge alignment fusion module integrates video feature data and multiple domain knowledge graphs into a unified traffic incident processing knowledge base.

[0019] Step 101: Process traffic video data and professional domain document collection through video feature extraction and hierarchical ontology construction network to obtain domain ontology knowledge structure Video feature extraction algorithms and hierarchical ontology construction networks are used to process road traffic video data and document collections in various professional fields, including specifications, standards, processing manuals, and case collections, etc. Traffic event features in the video and core concepts, relationships, and rules in various fields are extracted to form a formal representation of domain knowledge. The hierarchical ontology construction network consists of a term extraction layer, a relationship identification layer, and a rule induction layer. The term extraction layer first extracts professional terms from the document and passes the extraction results to the relationship identification layer; the relationship identification layer identifies the semantic relationship between terms, and its output is input into the rule induction layer together with the results of the term extraction layer; the rule induction layer finally integrates the results of the first two layers and combines them with video features to generate a complete domain ontology knowledge structure. The processing process of the hierarchical ontology construction network can be expressed as: ; ; in: Indicates The ontology knowledge structure of a field contains a set of concepts , Relationship Set and constraint rule set ; Indicates A collection of professional documents in various fields. A dictionary of terms representing the field, represents a hierarchical ontology construction network, represents road traffic incident video data, represents the traffic event features extracted from the video, represents the video feature extractor, which uses deep learning-based video analysis technology; The three-level calculation process of the hierarchical ontology construction network combined with video features can be expressed as: ; ; ; ; in, It is the term extraction layer, which uses a term extraction algorithm based on term frequency-inverse document frequency (TF-IDF) and domain adaptability; It is the relation recognition layer, which uses a relation extraction algorithm based on dependency syntactic analysis and semantic pattern matching; It is the rule induction layer, which adopts the constraint rule generation algorithm based on association rule mining and logical reasoning.

[0020] Step 102: Process structured and unstructured data through a multi-level knowledge graph construction system to obtain a multi-domain knowledge graph; A multi-level knowledge graph construction system is used to process structured data (such as historical case databases, traffic monitoring records) and unstructured data (such as accident reports, expert experience descriptions), and entities, relationships, and attributes are extracted based on the domain ontology knowledge structure generated in step 101 to construct a multi-domain knowledge graph. The multi-level knowledge graph construction system consists of an entity recognition module, a relationship extraction module, and an attribute filling module. The entity recognition module first identifies entities from the data and passes the recognition results to the relationship extraction module; the relationship extraction module extracts the relationships between entities based on the entity recognition results, and the results of entity recognition and relationship extraction are jointly input into the attribute filling module; the attribute filling module completes the extraction and filling of entity attributes, and integrates the results of the first two modules to form a complete domain knowledge graph. The knowledge graph construction process can be expressed as: ; in: Indicates The knowledge graph of a field contains entity sets , relationship set and attribute sets , represents the ontology knowledge structure of the domain (output of step 101), Represents a collection of structured data in this field, Represents a collection of unstructured data in this field, Represents a multi-level knowledge graph construction system; The three-module calculation process of the multi-level knowledge graph construction system can be expressed as: ; ; ; ; in, It is an entity recognition module, based on named entity recognition and ontology matching technology; It is a relation extraction module that uses a combination of remote supervised learning and pattern matching; It is an attribute filling module that uses information extraction and missing value inference techniques.

[0021] Step 103: Process the multi-domain knowledge graph through the three-stage knowledge alignment fusion network to obtain a unified traffic incident processing knowledge base This step uses a three-stage knowledge alignment fusion network to process the multi-domain knowledge graph generated in step 102, aligns and fuses the knowledge graphs in each field, eliminates concept conflicts and data redundancy, and builds a unified traffic incident processing knowledge base. The three-stage knowledge alignment fusion network consists of an entity alignment layer, a relationship mapping layer, and a conflict resolution layer. The entity alignment layer first identifies the corresponding relationships of cross-domain entities and passes the alignment results to the relationship mapping layer; the relationship mapping layer performs relationship mapping transformation based on the entity alignment results, and the results of entity alignment and relationship mapping are jointly input into the conflict resolution layer; the conflict resolution layer handles concept overlaps and data conflicts, and integrates the results of the first two layers to generate the final unified knowledge base. The knowledge alignment and fusion process can be expressed as: ; in: represents the final unified traffic incident processing knowledge base; express A collection of knowledge graphs in a field (output of step 102), Represents a three-stage knowledge alignment fusion network The processing process of the three-stage knowledge alignment fusion network can be expressed in detail as follows:

[0022]

[0023]

[0024] in: represents the cross-domain entity alignment matrix, Representation field Entities and fields The corresponding relationship between entities in represents the unified set of relations, , and Represent the parameter matrices of the entity alignment layer, relationship mapping layer, and conflict resolution layer, respectively. It is the entity alignment layer, which uses an entity alignment algorithm based on semantic embedding and structural matching. It is the relational mapping layer, which uses the relational mapping method of pattern matching and statistical learning. It is the conflict resolution layer, which is a conflict resolution strategy based on confidence evaluation and rule reasoning; Structured Traffic Incident Processing Knowledge Base It is a knowledge graph that contains unified entities, relationships, and attributes across domains, and will serve as the knowledge basis for multi-party collaborative decision-making in subsequent steps.

[0025] Step 200: Processing traffic video data through video scene in-depth analysis and multi-party collaborative reasoning system to obtain a multi-dimensional decision analysis report; This step uses the structured traffic incident processing knowledge base constructed in step 100, and uses the video scene deep analysis and multi-party collaborative reasoning system to process the video data from traffic incident monitoring, analyze and identify various types of traffic incidents, and generate multi-dimensional analysis reports that meet the decision-making needs of different departments. The video scene deep analysis and multi-party collaborative reasoning system consists of three main modules: a video event recognition network, a knowledge-enhanced scene understanding engine, and a multi-view report generator, forming a complete video analysis and reasoning chain. The video event recognition network first identifies traffic events from traffic monitoring videos and extracts structured feature representations, and passes the processing results to the knowledge-enhanced scene understanding engine; the scene understanding engine combines the knowledge base information to construct an event semantic scene graph, and its output and the knowledge base are jointly input into the multi-view report generator; the multi-view report generator generates customized analysis reports based on the needs of different departments.

[0026] Step 201: Processing traffic monitoring video data through a deep video analysis and feature extraction network to obtain a structured event feature representation; Steps: Use a deep video analysis and feature extraction network to process video data, image data, radar data, and geographic location data from a road traffic monitoring system, detect and identify traffic events, extract key features of events, and form a structured representation. The deep video analysis and feature extraction network consists of four components: a video event detector, a visual feature extractor, a sensor feature extractor, and a multimodal feature fusion device. The video event detector first detects traffic events from the video stream; the visual feature extractor extracts visual features from event-related video clips and images, and the sensor feature extractor extracts sensor features from radar and geographic location data. The results of the two extractors are input into the multimodal feature fusion device in parallel; the fusion device integrates multiple features and generates a unified structured feature representation. The feature extraction process can be expressed as: ; ; in: represents a traffic event detected from the video, Structured feature representation of events, including spatiotemporal features , Object Characteristics and relationship characteristics , Represents traffic surveillance video data Represents static image data, Represents radar and other sensor data, Represents geographic information data, represents a video event detector, represents a hierarchical multimodal feature extraction network; The calculation process of each component of the deep video analysis and feature extraction network can be expressed as: ; ; ; ; in: represents a traffic event detected from the video, represents visual features, generated by the visual feature extractor, represents the sensor features, generated by the sensor feature extractor, It is a visual feature extractor based on a deep convolutional neural network architecture, which includes two submodules: spatial feature extraction and temporal feature capture. It is a sensor feature extractor that uses a combination of multi-layer perceptron and temporal convolution. It is a multimodal feature fuser that uses attention mechanism and cross-modal transformation for feature integration The expression of the visual feature extractor is: ; in is a convolutional neural network, is the temporal attention mechanism; the expression of the sensor feature extractor is: ; in is a multi-layer perceptron, is a temporal convolutional network; the expression of the multimodal feature fusion is: ; in , and They are the weight matrices of visual features, sensor features and fusion features respectively.

[0027] Step 202: Processing the structured event feature representation through a dual-path knowledge scene understander to obtain an event semantic scene graph; This step uses the dual-path knowledge scene understander to process the structured event feature representation generated in step 201, combined with the structured traffic event processing knowledge base constructed in step 100, to construct a semantic scene graph that describes the entire event. The dual-path knowledge scene understander consists of a data-driven path and a knowledge-driven path. The two paths process inputs in parallel and finally fuse the results. The data-driven path directly identifies objects and relationships from feature representations based on a deep learning model; the knowledge-driven path uses prior information in the knowledge base for reasoning and enhancement; the results of the two paths are integrated into a complete semantic scene graph through a fusion mechanism. The scene understanding process can be expressed as: ; in: A semantic scene graph representing an event, containing a set of object nodes , relationship edge set and attribute sets , Structured feature representation representing the event (output of step 201) represents the structured traffic incident processing knowledge base (output of step 100), represents a dual-path knowledge scene understander; The computation process of the dual-path knowledge scene understander can be expressed as: ; ; ; ; ; in: and Respectively represent the objects and relationships of data-driven path identification, and Respectively represent the objects and relations of knowledge-driven path reasoning, is an object detector that uses a region proposal network and an object classification network. is a relationship predictor based on a graph convolutional network model. It is a knowledge-based entity linker that maps detected objects to entities in the knowledge base. It is a knowledge-based relational reasoner that uses the knowledge base to perform relational reasoning. It is a graph fusion module that integrates the results of two paths through confidence weighting and conflict resolution strategy; The expression of the object detector is: ; The expression of the relationship predictor is: ; in is a graph convolutional network; the expression of the entity linker is: ; The expression of the relational reasoner is: .

[0028] Step 203: Processing the event semantic scene graph through the adaptive multi-view report generation model to obtain a multi-dimensional decision analysis report; This step uses the adaptive multi-perspective report generation model to process the event semantic scene graph constructed in step 202, combined with the structured traffic event processing knowledge base in step 100, to generate a multi-dimensional decision analysis report containing the focus of each department for the decision-making needs of different departments such as traffic management, emergency rescue, medical first aid and insurance claims. The adaptive multi-perspective report generation model consists of three modules: demand analyzer, information filter and report generator. The demand analyzer first identifies the specific needs of different departments and passes the analysis results to the information filter; the information filter extracts relevant information from the semantic scene graph based on department needs, and the screening results and department needs are input into the report generator together; the report generator generates a customized decision analysis report for each department. The report generation process can be expressed as: ; in: Expressed as A collection of multi-dimensional decision analysis reports generated by different departments. a semantic scene graph representing the event (output of step 202), represents the structured traffic incident processing knowledge base (output of step 100), express List of concerns and needs of different departments, represents an adaptive multi-view report generation model; the calculation process of the adaptive multi-view report generation model is: ; ; ; in: Express to the department The results of the structured analysis of requirements, Represented as department Filter related information, It is a demand analyzer that analyzes and quantifies departmental needs based on the knowledge base. is an information filter that uses the attention mechanism to extract relevant information from the semantic scene graph. It is a report generator that uses template filling and natural language generation techniques; Multi-dimensional decision analysis report It contains four parts: event description, key factor analysis, decision-making suggestions and risk assessment. Provides comprehensive decision support information.

[0029] Step 300: Processing historical video event data through a video event analysis and decision knowledge adaptive learning system to obtain a best practice knowledge model; This step processes historical traffic video event data, decision data, and effect feedback data through the video event analysis and decision-making knowledge adaptive learning system. Through the closed-loop process of feature extraction, recording, evaluation, and optimization, it continuously updates the best event detection and processing practice knowledge and builds a knowledge model that supports future similar event decision-making. The system consists of a video event feature library, a decision chain construction module, an effect evaluation module, and a knowledge optimization module: the video event feature library stores typical features of traffic events; the decision chain construction module extracts the complete decision process; the effect evaluation module evaluates the decision effect in combination with the event results; the knowledge optimization module updates the knowledge base based on the evaluation results and generates an optimized best practice knowledge model.

[0030] Step 301: Processing traffic video events and multi-party decision data through video event comparison and temporal decision tracking network to obtain a structured decision chain model; This step uses the video event comparison and temporal decision tracking network to process historical traffic video event data and decision data from departments such as traffic management, emergency rescue, medical first aid, and insurance claims, compare event features, and build a complete decision chain model that includes video event features, decision nodes, decision basis, and decision results. The network consists of a video event feature extractor, a decision identifier, a causal associator, and a temporal organizer: the video event feature extractor extracts event features from historical videos; the decision identifier identifies key decision points and decision subjects; the causal associator analyzes the causal relationship of decisions and provides a decision basis in combination with the structured traffic event processing knowledge base of step 100; the temporal organizer integrates the results to generate a structured decision chain model. The decision chain construction process can be expressed as:

[0031]

[0032] in: Represents historical traffic incident video data, represents the traffic event features extracted from historical videos, Represents a structured decision chain model, including video event features , decision node set , decision basis set and decision result set , Indicates that the event is being processed key decision points, Indicates participation in decision-making An institution or individual, Indicates the decision Observations, represents the structured traffic incident processing knowledge base (output of step 100), represents the video event feature extractor, represents a sequential decision tracing network; The computational process of the video event comparison and temporal decision tracking network can be expressed as: ; ; ; ; in: is a decision identifier that uses an attention-based sequence labeling model to identify decision points. It is a causal associator that combines the knowledge base to analyze the causal relationship between decisions. It is a time-series organizer that constructs an algorithmic organization decision chain structure through a time-series diagram; The expression of the decision recognizer is: ; where is a combined model of bidirectional long short-term memory network and conditional random field; The expression of the causal associator is: ; where, is a graph attention network; The expression of the temporal organizer is: ; where, is a directed acyclic graph construction algorithm.

[0033] Step 302: Process the decision chain model and event result data through a multi-dimensional decision effect evaluation system to obtain a decision effect evaluation report This step uses the multi-dimensional decision effect evaluation system to process the decision chain model constructed in step 301 and the final result data of the corresponding event, comprehensively evaluate the effectiveness and impact of each decision point, and generate a structured decision effect evaluation report. The multi-dimensional decision effect evaluation system consists of an index calculation module, a contribution attribution module, and a comparative analysis module. The index calculation module first calculates the effect scores of each decision based on the multi-dimensional evaluation index system, and the results are passed to the contribution attribution module; the contribution attribution module analyzes the contribution degree of each decision to the overall effect, and the results of the two modules are jointly input into the comparative analysis module; the comparative analysis module compares the current decision with the historical best practice to generate a complete evaluation report. The effect evaluation process can be expressed as:

[0034] where: represents the decision effect evaluation report, including the effect scores of each decision , main contribution analysis and improvement suggestions , represents the structured decision chain model (the output of step 301), represents the final processing result and observed data of the event, represents the effect evaluation index system, including dimensions such as time efficiency, resource utilization, and security guarantee, represents the structured traffic event processing knowledge base (the output of step 100), represents the multi-dimensional decision effect evaluation system; The calculation process of the multi-dimensional decision effect evaluation system can be expressed as: ; ; ; ; Among them: is the index calculation module, which calculates the scores of various evaluation indexes. is the contribution attribution module, which analyzes the contribution degree of each decision. is the comparative analysis module, which compares with the historical best practice and puts forward improvement suggestions. The expression of the index calculation module is: ; Among them is the weight of the th index, is the calculation function of the th index; The expression of the contribution attribution module is: ; Among them represents the contribution degree calculation method based on the Shapley value; The expression of the comparative analysis module is: ; Among them represents the K-nearest neighbor algorithm, represents the best practice record in the knowledge base.

[0035] Step 303: Process the evaluation report and the knowledge base data through the incremental knowledge update engine to obtain the best practice knowledge model. In this step, the incremental knowledge update engine is used to process the decision effect evaluation report generated in step 302, and combined with the structured traffic event processing knowledge base in step 100, new empirical knowledge is refined and the knowledge base is updated to continuously optimize the best practice knowledge model. The incremental knowledge update engine consists of three modules: an experience extractor, a knowledge integrator, and a rule updater. The experience extractor first extracts key experiences from the evaluation report and passes the extraction results to the knowledge integrator; the knowledge integrator fuses the new experiences with the existing knowledge base, and the results of the two modules are jointly input into the rule updater; the rule updater updates the decision rules and best practices in the knowledge base to generate an optimized knowledge model. The knowledge update process can be expressed as: ; Among them: represents the updated best practice knowledge model, represents the original structured traffic event processing knowledge base (the output of step 100), represents the decision effect evaluation report (the output of step 302), represents the structured decision chain model (the output of step 301), represents the incremental knowledge update engine; The calculation process of the incremental knowledge update engine can be expressed as: ; ; ; in: Represents empirical knowledge extracted from evaluation reports and decision chains, represents the temporary knowledge base after integrating new experience, It is an experience extractor that extracts valuable experience from the evaluation report. It is a knowledge integrator that integrates new experience with existing knowledge. It is a rule updater that updates the decision rules and best practices in the knowledge base; The expression of the experience extractor is: ; in represents the pattern mining algorithm; the expression of the knowledge integrator is: ,in represents the knowledge fusion operation; the expression of the rule updater is: ; in Represents a rule updating algorithm based on Bayesian methods.

[0036] Best Practice Knowledge Model It is a knowledge base that includes decision rules, best practice cases, and decision effect evaluations, and will serve as the input of the multi-perspective conflict coordination system in step 400.

[0037] Step 400: Processing the video detection results and multi-department decision opinion data through the video event evidence fusion and multi-party conflict coordination system to obtain a comprehensive and balanced decision-making solution; This step processes the traffic incident evidence generated in step 200 and the decision opinions of departments such as traffic management, emergency rescue, medical first aid and insurance claims through the video event evidence fusion and multi-party conflict coordination system, and coordinates the decision conflicts of various departments and generates a comprehensive decision plan in combination with the best practice knowledge model updated in step 300. The system consists of a video evidence analysis module, a conflict detection module, a multi-objective optimization module and a decision suggestion generation module: the video evidence analysis module evaluates the reliability and relevance of the evidence; the conflict detection module identifies the potential conflict between the decision opinion and the video evidence; the multi-objective optimization module seeks a balanced solution; and the decision suggestion generation module converts the optimization results into an operational comprehensive decision plan.

[0038] Step 401: Processing multi-department decision opinion data through video evidence evaluation and semantic level conflict detection engine to obtain a structured conflict analysis model This step uses the video evidence evaluation and semantic hierarchical conflict detection engine to process the video event evidence generated in step 200 and the multi-dimensional decision analysis reports of various departments. Combined with the best practice knowledge model updated in step 300, the reliability of the video evidence and its consistency with the decision opinions are evaluated, the potential conflict points in the decision opinions of various departments are analyzed, and a structured conflict analysis model is generated. The engine consists of a video evidence evaluator, a semantic aligner, a target analyzer, and a conflict marker: the video evidence evaluator analyzes the reliability and relevance of the evidence; the semantic aligner identifies the semantic inconsistencies in the decision opinions; the target analyzer analyzes departmental goals and decision intentions; the conflict marker marks the conflict points in combination with the best practice knowledge model to generate a structured conflict analysis model. The conflict detection process can be expressed as: ; ; in: Indicates the video evidence assessment results, including the evidence reliability score and evidence relevance score ; represents the event semantic scene graph generated in step 200, Represents a structured conflict analysis model, including a set of conflict points , conflict type set and conflict severity assessment , express Decision opinions put forward by different departments based on multi-dimensional decision analysis reports, represents a set of multi-dimensional decision analysis reports generated in step 200, represents the best practice knowledge model updated in step 300, represents the video evidence evaluator, Represents a semantic level conflict detection engine; The computational process of video evidence evaluation and semantic level conflict detection engine can be expressed as: ; ; ; ; in: represents the semantic similarity matrix between decision opinions, The analysis results of the decision-making objectives of each department are shown. It is a semantic comparer that uses natural language processing technology to analyze the semantic differences between opinions. It is a target analyzer that analyzes the decision-making goals and interests of each department. It is a conflict marker that marks conflict points based on semantic comparison and target analysis results; The expression of the semantic matcher is:

[0039] in is the text embedding function, is the cosine similarity; the expression of the target analyzer is: ,in is the intent classifier; the expression of the conflict marker is: ; in It is a threshold-based grouping algorithm.

[0040] Step 402: Processing the structured conflict analysis model through a constraint balance optimization algorithm to obtain a conflict balance solution; This step uses the constraint balance optimization algorithm to process the structured conflict analysis model generated in step 401, and combines the decision-making opinions of multiple departments to find a conflict solution that can balance the interests and needs of all parties. The constraint balance optimization algorithm consists of three components: an objective function builder, a constraint generator, and a solution space searcher. The objective function builder first constructs a multi-objective optimization function based on the decision-making opinions and goals of each department, and passes the construction result to the constraint generator; the constraint generator generates constraints based on the conflict analysis model and the best practice knowledge model; the results of the two components are jointly input into the solution space searcher to find the optimal balance solution that meets the constraints. The optimization process can be expressed as: ; in, Represents a conflict balance solution, including a set of adjustment suggestions , Priority Sorting and compatibility analysis , represents the structured conflict analysis model (output of step 401), express Decision-making opinions from different departments, represents the best practice knowledge model (output of step 300), Represents the weight vector of different departments in the current event type, represents the constrained balance optimization algorithm, The calculation process of the constrained balance optimization algorithm can be expressed as: ; ; ; in: represents a set of multi-objective optimization functions, represents a set of constraints, It is an objective function builder that builds optimization goals based on the decision-making opinions of various departments. It is a constraint generator that generates constraints based on conflict analysis and knowledge models. It is a solution space searcher that finds the optimal equilibrium solution that satisfies the constraints; The expression of the objective function builder is: ; in For Department The weight of is the utility function; the constraint generator expression is: ; in, is the rule conversion algorithm; the expression of the solution space searcher is: ,in It is the non-dominated sorting genetic algorithm II.

[0041] Step 403: Processing the conflict balance solution through the collaborative decision solution generation system to obtain a comprehensive balance decision solution; This step uses the collaborative decision-making solution generation system to process the conflict balance solution generated in step 402, combines the best practice knowledge model and the original decision opinions, and transforms the mathematical optimization results into specific and operational comprehensive balance decision plans to support various departments to handle traffic incidents in a coordinated manner. The collaborative decision-making solution generation system consists of three modules: action converter, process planner, and effect predictor. The action converter first converts the abstract optimization solution into specific action suggestions and passes the conversion results to the process planner; the process planner designs the inter-departmental collaboration process, and the results of the two modules are jointly input into the effect predictor; the effect predictor analyzes the expected effect of the implementation of the plan and generates a complete comprehensive balance decision plan. The decision plan generation process can be expressed as: ; in: Represents a comprehensive balanced decision-making solution, including a set of action suggestions , Collaboration Process and expected effect analysis , represents the conflict balance solution (output of step 402), represents the best practice knowledge model (output of step 300), express Decision-making opinions from different departments, represents a collaborative decision-making solution generation system; The calculation process of the collaborative decision-making solution generation system can be expressed as: ; ; ; ; in: It is an action converter that converts optimization solutions into specific action suggestions. It is a process planner that designs inter-departmental collaboration processes. It is an effect predictor, analyzing the expected effect of program implementation; The expression of the action converter is: ; in is the template mapping algorithm; the expression of the process planner is: ; in, Generate an algorithm for serializing a directed acyclic graph; the expression of the effect predictor is: ; in, It is a case-based reasoning method.

[0042] Comprehensive and balanced decision-making plan It is a comprehensive decision support document that includes specific action recommendations, departmental collaboration processes and expected effect analysis, providing an operational guide for multi-departmental coordination in handling traffic incidents.

[0043] Take a multi-vehicle chain collision traffic accident that occurred at a complex intersection in a certain city as an example. The intersection is a crossroads of the city's main road and secondary road, with an average daily traffic volume of more than 50,000 vehicles. It is equipped with a high-definition video surveillance system (including 4 fixed cameras and 2 pan-tilt cameras), road radar sensors and traffic signal control systems.

[0044] Description of the incident: At 8:37 am on November 18, 2023, under rainy conditions, a chain collision involving five vehicles occurred in the west-to-east direction of the intersection. The cause of the accident was initially determined to be a brake failure of a heavy truck, which rear-ended a car waiting for the red light in front, resulting in a chain collision. The accident caused the east-west traffic at the intersection to be interrupted, and some vehicle occupants were injured, requiring coordinated handling by multiple departments such as traffic management, emergency rescue, medical first aid, and insurance claims. In this scenario, we deployed a multi-dimensional analysis and collaborative processing system for traffic events based on video detection.

[0045] In this accident handling, the video feature extraction and cross-domain knowledge fusion framework was first activated to process the real-time video data from the intersection and the professional knowledge data of various departments, and to build a structured traffic event processing knowledge base for multi-vehicle chain collision types. The data is shown in Table 1: Table 1

[0046] The output structured traffic event processing knowledge base is optimized for the current multi-vehicle chain collision scenario in rainy days, and especially enhances the feature recognition capability of videos under low-light conditions in rainy days, providing knowledge support for subsequent multi-dimensional analysis and collaborative decision-making.

[0047] Receive the constructed knowledge base and real-time traffic monitoring video, conduct in-depth scene analysis, and generate multi-dimensional decision analysis reports that meet the needs of different departments. The implementation data is shown in Table 2: Table 2

[0048] Based on real-time video analysis and multimodal data fusion, the generation of a multi-dimensional decision analysis report was completed at 8:37:48 (26 seconds after the accident), and each department received a targeted report before 8:38:15, greatly improving the initial response speed. It is particularly worth mentioning that the abnormal brake light status of the heavy truck was successfully captured and combined with trajectory analysis, and the main cause of the accident was correctly determined to be the failure of the brake system, which provided a key basis for subsequent disposal; the video event analysis and decision knowledge adaptive learning system was started to analyze the historical data of multi-vehicle chain collision events similar to rainy days, and optimize the decision knowledge model. The data is shown in Table 3: Table 3 Through video analysis and decision tracking of similar historical events, we identified that for rainy day chain collision accidents, the two key decision points of "prioritizing stabilization of heavy vehicles" and "creating rescue channels in advance" significantly affect the handling efficiency, and optimized the knowledge model accordingly. The update of the best practice knowledge model was completed at 8:40:12, providing experience support for accident handling and giving more precise suggestions on the handling of heavy trucks and the planning of rescue channels.

[0049] As each department arrived at the scene and began operations based on the multi-dimensional decision analysis report, the video event evidence fusion and multi-party conflict coordination system was activated to process the decision opinions put forward by each department, coordinate potential conflicts, and generate a comprehensive and balanced decision plan. The implementation data is shown in Table 4: Table 4

[0050] At 8:43:27, the conflicting points in the decision-making opinions of various departments were identified, and a comprehensive and balanced decision-making plan was generated at 8:45:05. In particular, in response to the conflict between the medical department's desire to transfer the injured as soon as possible and the insurance department's need to preserve on-site evidence, a coordination plan of "full video recording of the transfer process of the injured" was proposed based on video evidence, which not only met the needs of timely treatment of the injured, but also ensured the integrity of the evidence for the determination of accident responsibility.

[0051] Technical effect verification: The results are shown in Table 5-Table 8: Table 5: Comparison of decision quality and collaboration efficiency

[0052] Table 6: Comparison of video detection accuracy and processing efficiency

[0053] Table 7: Knowledge accumulation and system evolution effects Table 8: Socio-economic benefit analysis

[0054] Through tracking and data analysis of the entire process of handling this complex traffic accident, we have verified the significance and superiority of the technical effects of this implementation method.

[0055] Comprehensive analysis shows that this implementation method significantly improved the video detection accuracy and processing efficiency in the process of handling this complex traffic accident, reduced the cost of multi-department collaboration, improved decision-making quality, shortened the accident handling time, reduced the socioeconomic losses caused by traffic congestion, and greatly improved the overall efficiency of traffic incident handling.

[0056] The above describes an embodiment of the present invention, but this embodiment is not limited to the above specific implementation methods. The above specific implementation methods are merely illustrative and not restrictive. Under the guidance of this embodiment, ordinary technicians in this field can also make many forms, all of which are within the protection of this embodiment.

Claims

1. A road traffic incident video detection method, characterized in that: The following steps are involved: Through video feature analysis and cross-domain knowledge fusion framework, road surveillance videos and multi-source professional field data are processed to obtain a structured traffic event processing knowledge base; Based on the structured traffic event processing knowledge base, traffic video data is processed through video scene in-depth analysis and multi-party collaborative reasoning system to obtain a multi-dimensional decision analysis report; Based on the structured traffic event processing knowledge base and the multi-dimensional decision analysis report, the historical video event data is processed by the video event analysis and decision knowledge adaptive learning system to obtain a best practice knowledge model; Based on the multi-dimensional decision analysis report and the best practice knowledge model, the video detection results and multi-department decision opinion data are processed through the video event evidence fusion and multi-party conflict coordination system to obtain a comprehensive and balanced decision-making plan.

2. The road traffic incident video detection method according to claim 1, characterized in that: The following steps are involved: Step 101, processing traffic video data and professional domain document collection by video feature extraction and hierarchical ontology construction network to obtain domain ontology knowledge structure; Step 102: Process structured and unstructured data through a multi-level knowledge graph construction system to obtain a multi-domain knowledge graph; Step 103: Process the multi-domain knowledge graph through the three-stage knowledge alignment fusion network to obtain a unified traffic incident processing knowledge base.

3. The road traffic incident video detection method according to claim 2, characterized in that: The process of video feature extraction and hierarchical ontology construction network can be expressed as: ； ； in: Indicates The ontology knowledge structure of a field contains a set of concepts , relationship set and constraint rule set ; Indicates A collection of professional documents in various fields; A dictionary of terms representing the field; represents a hierarchical ontology construction network, Represents road traffic incident video data; represents the traffic event features extracted from the video; Represents a video feature extractor that uses deep learning-based video analysis techniques.

4. The road traffic incident video detection method according to claim 1, characterized in that: The following steps are involved: Step 201, processing traffic monitoring video data through a deep video analysis and feature extraction network to obtain a structured event feature representation; Step 202: Process the structured event feature representation through a dual-path knowledge scene understander to obtain an event semantic scene graph; Step 203: Process the event semantic scene graph through the adaptive multi-view report generation model to obtain a multi-dimensional decision analysis report.

5. The road traffic incident video detection method according to claim 4, characterized in that: In step 201, the processing process of the deep video analysis and feature extraction network can be expressed as: ； ； in: represents a traffic event detected from a video; Structured feature representation of events, including spatiotemporal features , Object Characteristics and relationship characteristics ; Represents traffic surveillance video data; Represents static image data; Represents radar and other sensor data; Represents geographic information data, represents a video event detector, Represents a hierarchical multimodal feature extraction network.

6. The road traffic incident video detection method according to claim 1, characterized in that: The following steps are involved: Step 301, processing traffic video events and multi-party decision data through video event comparison and temporal decision tracking network to obtain a structured decision chain model; Step 302: Process the decision chain model and event result data through the multi-dimensional decision effect evaluation system to obtain a decision effect evaluation report; Step 303: Process the evaluation report and knowledge base data through the incremental knowledge update engine to obtain a best practice knowledge model.

7. The road traffic incident video detection method according to claim 6, characterized in that: In step 301, the processing process of the video event comparison and timing decision tracking network can be expressed as: ； ； in: Represents historical traffic incident video data; represents the traffic event features extracted from historical videos; Represents a structured decision chain model, including video event features , decision node set , decision basis set and decision result set ; Indicates key decision points in the event handling process; Indicates the institutions or individuals involved in decision-making; represents the observations resulting from the decision; represents the structured traffic incident processing knowledge base, represents the video event feature extractor, Represents a sequential decision tracing network.

8. The road traffic incident video detection method according to claim 1, characterized in that: The following steps are involved: Step 401: Processing multi-department decision opinion data through video evidence evaluation and semantic level conflict detection engine to obtain a structured conflict analysis model; Step 402: Process the structured conflict analysis model through a constraint balance optimization algorithm to obtain a conflict balance solution; Step 403: Process the conflict balance solution through the collaborative decision-making solution generation system to obtain a comprehensive balance decision solution.

9. The road traffic incident video detection method according to claim 8, characterized in that: In step 401, the processing of the video evidence evaluation and semantic level conflict detection engine can be expressed as: ； ； in: Indicates the video evidence assessment results, including the evidence reliability score and evidence relevance score ; Represents event semantic scene graph; Represents a structured conflict analysis model, including a set of conflict points , conflict type set and conflict severity assessment ; Indicates the decision opinions put forward by different departments based on multi-dimensional decision analysis reports; Represents a collection of multi-dimensional decision analysis reports; Represents the best practice knowledge model, represents the video evidence evaluator, Represents the semantic level conflict detection engine.

10. A road traffic incident video detection system, characterized in that: It is used to execute a road traffic incident video detection method as described in any one of claims 1-9, including: a video feature analysis and cross-domain knowledge fusion module, which is used to process road monitoring videos and multi-source professional field data to obtain a structured traffic incident processing knowledge base; a video scene depth analysis and multi-party collaborative reasoning module, which is used to process traffic video data to obtain a multi-dimensional decision analysis report; a video event analysis and decision knowledge adaptive learning module, which is used to process historical video event data to obtain a best practice knowledge model; a video event evidence fusion and multi-party conflict coordination module, which is used to process video detection results and multi-department decision opinion data to obtain a comprehensive and balanced decision plan.

Images