Traffic accident report time series knowledge graph modeling method based on deep learning

Abstract

A method for modeling a time series knowledge graph of traffic accident reports based on deep learning, the method includes: in the accident report after data preprocessing, modeling concepts, relationships and attributes in sequence to obtain an accident domain ontology concept model; using a Bert‑BiLSTM‑CRF network to extract and model the static information related to the accident text as a named entity recognition (NER) task to obtain static information of the accident scene; using a Bert‑CoAtt‑CasRel network to extract and model the dynamic evolution process of the accident and the time series information of key events as a relationship extraction task to reconstruct the spatiotemporal action sequence of the accident vehicle; based on the static information of the accident scene and the spatiotemporal action sequence of the accident vehicle, mapping corresponding concepts, relationships and attributes through the obtained accident domain ontology concept model to generate a dynamic knowledge graph of the accident domain. The present invention performs structured representation on unstructured text and can dynamically evolve the accident process.

Description

Technical Field

[0001] The present invention belongs to the field of traffic safety technology, and specifically relates to a method for modeling a time series knowledge graph of traffic accident reports based on deep learning. Background Art

[0002] As a product with extremely high safety requirements, autonomous driving can only replace human driving and contribute to the analysis and decision-making of traffic safety if its safety far exceeds that of traditional cars. This requires in-depth research on existing accident data to learn from experience and lessons. Accident data can be divided into two categories: structured data and unstructured data. Existing research on the application and analysis of accident data is mostly focused on reconstructing accident scenes. Although many studies have made efforts on data-driven scene simulation, their use of unstructured text information is still insufficient. These unstructured text data are usually written in natural language, which contains important information such as roads, traffic facilities, meteorological conditions, and traffic participants required to reconstruct the scene. Therefore, how to mine the rich information contained in text-type data, structure them, and reveal the application of these data resources in the field of traffic safety is currently a major concern. Summary of the invention

[0003] The purpose of the present invention is to address the problems in the above-mentioned prior art and provide a traffic accident report time series knowledge graph modeling method based on deep learning, to automatically process traffic accident texts based on deep learning, to reduce the omission of dynamic interaction information extraction between vehicles and road networks, and to structure the unstructured text by mapping the extracted information into a dynamic knowledge graph, thereby dynamically evolving the accident process and providing new technical means for accident analysis and prevention.

[0004] In order to achieve the above object, the present invention has the following technical solutions: In the first aspect, a method for modeling a time series knowledge graph of traffic accident reports based on deep learning is provided, comprising: Collect accident reports and perform data preprocessing; In the accident report after data preprocessing, the concept, relationship and attribute modeling are carried out in turn to obtain the accident domain ontology conceptual model; The Bert-BiLSTM-CRF network is used to model the extraction of static information related to the accident text as a named entity recognition (NER) task to obtain static information about the accident scene. The Bert-CoAtt-CasRel network is used to model the accident dynamic evolution process and the extraction of key event time sequence information as a relationship extraction task, and the spatiotemporal action sequence of the accident vehicle is reconstructed; Based on the static information of the accident scene and the spatiotemporal action sequence of the accident vehicle, the corresponding concepts, relationships and attributes are mapped through the obtained accident domain ontology concept model to generate a dynamic knowledge graph of the accident domain.

[0005] As a preferred solution, in the step of collecting accident reports and performing data preprocessing, the data preprocessing includes: Convert all capital letters in the accident report text to lowercase so that the information extracted later is unified in form; Define a domain-specific dictionary, and then use regular matching to convert specific words in the accident report that meet the matching conditions into special symbols; Perform reference resolution to obtain the named entity that the pronoun actually refers to, and restore the pronouns in the accident report text to the objects they originally referred to; The sentences in the accident report text are processed by boundary division and the text is converted into several single sentences.

[0006] As a preferred solution, the steps of sequentially performing concept, relationship and attribute modeling in the accident report after data preprocessing to obtain the accident domain ontology concept model include: Based on component syntactic analysis, concept modeling is performed. Noun phrases are extracted using component syntactic analysis. The PAC algorithm is used to reduce the dimension of the vector representation of the phrases. The K-Means algorithm is used to cluster the noun phrases after the dimension reduction vector representation to obtain the accident domain concept ontology. Static road network relationship extraction is performed based on dependency syntactic analysis and external knowledge fusion. The relationship between concepts is obtained by using the dependency tree, connecting verbs or verb phrases in the dependency tree, and using the external knowledge network. Vehicle dynamic relations are extracted based on dependency syntactic analysis and external knowledge fusion. Verbs with vehicles as noun subjects are searched in sentence dependencies, and verb phrases containing corresponding verbs are searched in the syntactic structure tree. Combined with statistical methods, vehicle action information contained in the accident domain knowledge graph is screened out. Based on the attribute supplement of dependency syntactic analysis, a dependency tree containing concept sentences is constructed. By searching for subtrees containing concepts within a set distance, the attributes that modify the concepts are obtained. After statistical analysis, the attributes and corresponding data types are obtained.

[0007] As a preferred solution, the use of the Bert-BiLSTM-CRF network to extract and model the static information related to the accident text as a named entity recognition (NER) task to obtain the static information of the accident scene includes the following steps: Labeling data: using labeling tools to label various entities in the static road network of multiple accident report texts to obtain entity labels of the static road network; The labeled data is processed and input into the Bert-BiLSTM-CRF network for training and prediction, and multiple defined entities are identified; the Bert-BiLSTM-CRF network is divided into three modules: Bert layer, BiLSTM layer and CRF layer. The Bert layer is used to convert the input text sequence into a higher-dimensional semantic vector representation and provide a pre-trained representation; the BiLSTM layer uses the vector generated by the Bert layer as input to extract and encode contextual features and enhance the extraction of sequence features; the CRF layer decodes the features output by the BiLSTM layer, predicts the most likely label sequence, and ensures the consistency and rationality of the label sequence; By traversing the predicted output text and the corresponding labels, we first determine the road type and generate the corresponding number of branches. Then we extract the entity information in the sentence in turn and assign the entity information to the corresponding branches. We fill in the corresponding attributes of different branches, complete the static information instance extraction, and finally output the structured road network information.

[0008] As a preferred solution, the Bert-CoAtt-CasRel network is used to model the accident dynamic evolution process and the extraction of key event time sequence information as a relationship extraction task, and the reconstruction of the accident vehicle spatiotemporal action sequence includes the following steps: Use the annotation tool doccano to annotate the entities, relations, and triple labels of dynamic traffic participants in multiple accident report texts; The labeled data is processed and input into the Bert-CoAtt-CasRel network for training to extract relation triplets. The Bert-CoAtt-CasRel network is divided into Bert encoding module, CoAttention improvement module, subject recognition module, and object recognition module under relation conditions. The Bert-CoAtt-CasRel network adopts a cascade binary decoding mechanism to first identify all possible subjects in the sentence, and then predict the related relation relation and object objec for each identified subject. The extracted triples are reconstructed into a spatiotemporal sequence, starting from the execution of the perform relationship, searching for relationships, and generating an action list actionlist in the format of (vehicle, action, direction, road, lane, hit vehicle, timestamp). The subject of the triple (Vehicle, Perform, Action) corresponding to the perform relationship is assigned to the vehicle of the corresponding action list actionlist, and the object is assigned to the action. Taking the corresponding action as the subject, traverse the triples to search for other defined relationships. This process is executed for each perform relationship to generate a behavior sequence for each vehicle.

[0009] As a preferred solution, the static information based on the accident scene and the spatiotemporal action sequence of the accident vehicle is mapped to corresponding concepts, relationships and attributes through the obtained accident domain ontology concept model to generate a dynamic knowledge graph in the accident domain, including the following steps: Create scenarios and road network concept node instances in the graph database Neo4j, generate road network relationship edge instances, and connect the nodes; Create a vehicle node in the graph database Neo4j, traverse the action list from the initial moment, create vehicle action relationship edges, and obtain a dynamic time series knowledge graph in the accident field.

[0010] As a preferred solution, after generating the dynamic knowledge graph of the accident field, the process further includes instantiating the application of the dynamic knowledge graph of the accident scene, and the instantiating the application of the dynamic knowledge graph of the accident scene includes the following steps: The dynamic knowledge graphs generated by all scenarios are stored in the graph database Neo4j. Users can search for the dynamic knowledge graphs of accident scenarios that meet their requirements through different query condition combinations, and search by restricting accident scenarios with multiple conditions. Based on the generated dynamic knowledge graph, a trajectory reproduction algorithm is used, supplemented by mathematical constraints to complete the vehicle's movements and directions, reconstruct the accident scenario, replace any vehicle in the scenario with an autonomous driving vehicle, and test whether the path planning of the autonomous driving system ADS can avoid accidents.

[0011] In the second aspect, a traffic accident report time series knowledge graph modeling system based on deep learning is provided, including: Data preprocessing module, used to collect accident reports and perform data preprocessing; The accident domain ontology conceptual model construction module is used to perform concept, relationship and attribute modeling in the accident report after data preprocessing, and then obtain the accident domain ontology conceptual model; The accident scene static information acquisition module is used to use the Bert-BiLSTM-CRF network to extract the static information related to the accident text and model it as a named entity recognition (NER) task to obtain the static information of the accident scene; The accident vehicle spatiotemporal action sequence reconstruction module is used to use the Bert-CoAtt-CasRel network to model the accident dynamic evolution process and key event time sequence information extraction as a relationship extraction task to reconstruct the accident vehicle spatiotemporal action sequence; The accident domain dynamic knowledge graph generation module is used to generate a dynamic knowledge graph in the accident domain by mapping corresponding concepts, relationships and attributes based on the static information of the accident scene and the spatiotemporal action sequence of the accident vehicle through the obtained accident domain ontology concept model.

[0012] In a third aspect, an electronic device is provided, including: A memory storing at least one instruction; and a processor executing the instructions stored in the memory to implement the deep learning-based traffic accident report time series knowledge graph modeling method.

[0013] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for modeling a time series knowledge graph of traffic accident reports based on deep learning is implemented.

[0014] Compared with the prior art, the present invention has at least the following beneficial effects: After the present invention performs data preprocessing on the accident report, it follows the basic steps of knowledge graph construction, and sequentially models concepts, relationships and attributes, and sorts out the accident domain ontology concept model. Then, the Bert-BiLSTM-CRF network is used to extract and model the static information related to the accident text as a named entity recognition (NER) task to obtain the static information of the accident scene. In addition, the Bert-CoAtt-CasRel network is used to model the dynamic evolution process of the accident and the temporal information extraction of key events as a relationship extraction task, and the spatiotemporal action sequence of the accident vehicle is reconstructed to obtain dynamic evolution and temporal information. Furthermore, based on the static information of the accident scene and the spatiotemporal action sequence of the accident vehicle, the corresponding concepts, relationships and attributes are mapped through the obtained accident domain ontology concept model to generate a dynamic knowledge graph for the accident domain. The effectiveness of the knowledge graph is verified by analyzing the spatiotemporal factors of complex semantic traffic accidents and reconstructing the accident scene in a simulation environment. The present invention can automatically extract and model accident scene information from unstructured accident reports and dynamically evolve the accident process, overcoming the limitation of existing methods that mainly focus on static data analysis. It can effectively model and study the occurrence sequence and time dependency of accident events, and also solves the problem of missing extraction of dynamic interaction information between vehicles and road networks, providing a new technical means for accident analysis and prevention. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0016] Figure 1 It is a schematic diagram of the principle of a method for modeling a time series knowledge graph of traffic accident reports based on deep learning in an embodiment of the present invention; Figure 2It is a diagram of the Bert-BiLSTM-CRF neural network model architecture in an embodiment of the present invention; Figure 3 is a flow chart of static road network information extraction in an embodiment of the present invention; Figure 4 It is a diagram of the Bert-CoAtt-CasRel (BCCR) network model architecture in an embodiment of the present invention; Figure 5 is a flow chart of extracting spatiotemporal action sequences of dynamic traffic participants in an embodiment of the present invention; Figure 6 It is a diagram of the scene dynamic knowledge graph generation process in an embodiment of the present invention; Figure 7 It is the ontological conceptual model of the traffic accident domain; Figure 8 This is a sample diagram of an accident report; Fig. 9 It is a static information text example diagram; Fig.10 This is an example diagram of static information text extraction results; Fig.11 This is a sample picture of dynamic information text; Fig.12 This is an example diagram of the dynamic information text triple extraction result; Fig.13 This is an example diagram of the reconstruction result of the vehicle's spatiotemporal action sequence; Fig.14 This is an example diagram of a dynamic temporal knowledge graph; Fig.15 This is an example diagram of implementing complex conditional searches; Fig.16 This is an example diagram of a complex conditional view graph; FIG17 (a) is an example diagram of the initial state of the vehicle in the scene reconstruction result; FIG17( b ) is an example of a scene reconstruction result in which vehicle V1 is performing a left turn and V2 is still driving straight; FIG17 ( c ) is an example diagram of a state in which vehicles V1 and V2 collide as a result of scene reconstruction; FIG18 (a) is a schematic diagram of ADS path planning when vehicle V2 is replaced by an autonomous driving vehicle; Figure 18 (b) is a schematic diagram showing that the ADS path planning fails to avoid a collision when vehicle V1 turns at an intersection. DETAILED DESCRIPTION

[0017] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, ordinary technicians in this field can also obtain other embodiments without making creative work.

[0018] See also Figure 1 , the traffic accident report time series knowledge graph modeling method based on deep learning in an embodiment of the present invention includes: Step 1: Collect accident reports and perform data preprocessing, including lowercase conversion, special symbol and vocabulary processing, reference resolution, and sentence segmentation; Step 2: In the accident report after data preprocessing, the concepts, relationships and attributes are modeled in turn to obtain the accident domain ontology concept model; the bottom-up method is adopted, following the basic steps of knowledge graph construction, and the concepts, relationships and attributes in the model are modeled in turn with the help of manual work. Finally, natural language processing technologies such as K-Means clustering algorithm, PCA dimensionality reduction algorithm, component syntax analysis, etc. are used, combined with BabeNet external semantic network, to construct the accident domain ontology concept model; Step 3: Extract static information from the accident report and use the Bert-BiLSTM-CRF network to model the extraction of static information related to the accident text as a named entity recognition (NER) task to obtain static information of the accident scene. Step 4: Use the Bert-CoAtt-CasRel (BCCR) network to model the accident dynamic evolution process and the extraction of key event time sequence information as a relation extraction (RE) task. After obtaining the dynamic information of the accident scene, reconstruct the spatiotemporal action sequence of the accident vehicle. Step 5: Based on the static information of the accident scene and the spatiotemporal action sequence of the accident vehicle, the corresponding concepts, relationships and attributes are mapped through the obtained accident domain ontology concept model to generate a dynamic knowledge graph of the accident domain; Step six: The application of the dynamic knowledge graph of accident scenarios is illustrated by examples. On the one hand, the storage and query of the knowledge graph are completed through the graph database Neo4j; on the other hand, the reconstruction of the traffic accident scenario is completed through the MatLab2021b simulation software, and examples are used to illustrate that the traffic accident scenario of the present invention can further verify the safety of the autonomous driving algorithm.

[0019] In a possible implementation manner, the data preprocessing in step 1 includes the following steps: Step 101: Convert all capital letters in the accident report text to lowercase, so that the information extracted later is unified in form; Step 102: Define a domain-specific dictionary, and then use regular matching to convert specific words in the accident report that meet the matching conditions into special symbols, such as converting the word "percentage" into the form of "%" to facilitate subsequent processing; Step 103: Perform reference resolution to obtain the named entity actually referred to by the pronoun, and restore the pronoun in the accident report text to the object it originally referred to; Step 104: Perform boundary division processing on the sentences in the accident report text, that is, perform sentence boundary detection and convert the text into several single sentences to facilitate subsequent processing.

[0020] In a possible implementation manner, step 2 specifically includes the following steps: Step 201: Perform concept modeling based on component syntactic analysis, extract noun phrases using component syntactic analysis, use the PAC algorithm to reduce the dimensionality of the vector representation of the phrases, and then use the K-Means algorithm to cluster the noun phrases after the dimensionality reduction vector representation to obtain the accident domain concept ontology, including 11 concepts: Accident, Weather, Speed ​​limit, Roadway, Vehicle, Lane, Time, Traffic Signal, Driver, Passenger, and Intersection.

[0021] The concepts and descriptions are shown in Table 1: Table 1

[0022] Step 202: Extract static road network relationships based on dependency syntactic analysis and external knowledge fusion, use the dependency tree, connect verbs or verb phrases in the dependency tree, and use the external knowledge network to obtain the relationship between concepts.

[0023] The concept relationship of static road network is shown in Table 2: Table 2

[0024] Step 203: Vehicle dynamic relations are extracted based on dependency syntactic analysis and external knowledge fusion. Verbs with vehicles as noun subjects are searched in sentence dependencies. The simplest verb phrases containing the corresponding verbs are searched in the syntactic structure tree. Statistical methods are used to filter out vehicle action information contained in the accident domain knowledge graph.

[0025] The extracted vehicle dynamic relations are shown in Table 3: Table 3

[0026] Step 204: Based on the attribute supplement of dependency syntactic analysis, a dependency tree containing the concept sentence is constructed. By searching for the subtree containing the concept within the set distance, the attributes that modify the concept are obtained. After statistical analysis, the attributes and corresponding data types are obtained. The obtained attributes and their data types are shown in Table 4.

[0027] Table 4

[0028] Step 205: Organize and obtain the accident domain ontology conceptual model.

[0029] In a possible implementation, step three uses a Bert-BiLSTM-CRF network to extract and model the static information related to the accident text as a named entity recognition (NER) task, and obtaining the static information of the accident scene includes the following steps: Step 301: Label the data. Use the labeling tool YEDDA to label the six entities in the static road network of 100 accident report texts. The entity labels of the static road network are shown in Table 5: Table 5

[0030] Step 302: Process the labeled data and input it into Figure 2 The Bert-BiLSTM-CRF network shown in the figure is used for training and prediction, and the six defined entities are identified; the Bert-BiLSTM-CRF network is divided into three modules: Bert layer, BiLSTM layer and CRF layer. The Bert layer is used to convert the input text sequence into a higher-dimensional semantic vector representation and provide a pre-trained representation; the BiLSTM layer uses the vector generated by the Bert layer as input to extract and encode context features and enhance the extraction of sequence features; the CRF layer decodes the features output by the BiLSTM layer, predicts the most likely label sequence, and ensures the consistency and rationality of the label sequence; Step 303: The static road network information extraction process is as follows Figure 3 By traversing the predicted output text and the corresponding labels, the road type is first determined, the corresponding number of branches is generated, and then the entity information in the sentence is extracted in sequence, and the entity information is assigned to the corresponding branches, and the corresponding attributes of different branches are filled in to complete the static information instance extraction, and finally the structured road network information is output.

[0031] In a possible implementation, step 4 uses the Bert-CoAtt-CasRel network to model the accident dynamic evolution process and the extraction of key event time sequence information as a relationship extraction task, and the reconstruction of the accident vehicle spatiotemporal action sequence includes the following steps: Step 401: Use the annotation tool doccano to annotate the entities, relations, and triples of dynamic traffic participants in 1,000 accident report texts. The annotation labels are shown in Table 6: Table 6

[0032] Step 402: Process the labeled data and input it into Figure 4 The Bert-CoAtt-CasRel network shown in the figure is trained to extract relation triplets; the Bert-CoAtt-CasRel network is divided into a Bert encoding module, a CoAttention improvement module, a subject recognition module, and an object recognition module under relation conditions; the Bert-CoAtt-CasRel network adopts a cascade binary decoding mechanism, first identifying all possible subjects in the sentence, and then predicting the related relation relation and object objec for each identified subject, so that the model can better handle the situation of entity overlap.

[0033] Step 403: Reconstruct the spatiotemporal sequence of the extracted triples. The spatiotemporal sequence reconstruction algorithm of triples is as follows: Figure 5 As shown. Starting from the execution of the perform relationship, search for relationships and generate an action list actionlist in the format of (vehicle, action, direction, road, lane, hit vehicle, timestamp). Assign the subject of the triple (Vehicle, Perform, Action) corresponding to the perform relationship to the vehicle of the corresponding action list actionlist, and assign the object to the action; take the corresponding action as the subject, traverse the triple to search for other defined relationships, execute this process for each perform relationship, and generate the behavior sequence of each vehicle.

[0034] In one possible implementation, see Figure 6 , Step 5: Based on the static information of the accident scene and the spatiotemporal action sequence of the accident vehicle, the corresponding concepts, relationships and attributes are mapped through the accident domain ontology concept model to generate the accident domain dynamic knowledge graph, which includes the following steps: Step 501: Create scenarios and road network concept node instances in the graph database Neo4j, generate road network relationship edge instances, and connect the nodes; Step 502: Create a vehicle node in the graph database Neo4j, traverse the action list from the initial moment, create vehicle action relationship edges, and obtain a dynamic time series knowledge graph in the accident field.

[0035] In a possible implementation, step 6 instantiates the application of the accident scenario dynamic knowledge graph, including: Step 601: The dynamic knowledge graphs generated by all scenarios are stored in the graph database Neo4j. Users can search for the dynamic knowledge graphs of accident scenarios that meet the requirements by combining different query conditions, and search by restricting the accident scenarios with multiple conditions; Step 602: Based on the generated dynamic knowledge graph, a trajectory reproduction algorithm is used, supplemented by mathematical constraints to complete the vehicle's motion and direction, and the accident scene is reconstructed. Any vehicle in the scene is replaced with an autonomous driving vehicle to test whether the path planning of the autonomous driving system (ADS) can avoid the occurrence of accidents.

[0036] The traffic accident report time series knowledge graph modeling method of the embodiment of the present invention adopts a deep learning method to model the dynamic time series knowledge graph, uses the Bert-BiLSTM-CRF network to model the static information extraction related to the accident as a named entity recognition (NER) task, and uses the Bert-CoAtt-CasRel (BCCR) network to model the dynamic evolution of the accident and the time series information extraction of key events as a relation extraction (RE) task, thereby realizing the process of automatically extracting accident scene information instances from unstructured text accident reports, mapping them to the domain ontology concept model, and generating a dynamic knowledge graph of the accident scene. The effectiveness of the knowledge graph is verified by analyzing the spatiotemporal factors of complex semantic traffic accidents and reconstructing the accident scene in a simulation environment.

[0037] An example of modeling the time series knowledge graph of traffic accident reports based on deep learning in the present invention is described in detail below.

[0038] The accident domain ontology conceptual model constructed by the present invention is as follows: Figure 7 As shown in the figure, by analyzing the accident reports, we can find the following rules: the first paragraph, as the source of extracting static road network and environmental information, records the weather conditions, lighting conditions, road network environment, etc. when the accident occurred; the second paragraph, as the source of extracting dynamic participant behavior information, records the vehicle manufacturer, year, driver age, gender, and vehicle movements, direction, collision conditions, etc.; the subsequent paragraphs mainly record the verbal statements of the parties involved and the casualties of the accident. Figure 8 This is part of the original accident report.

[0039] Use the Bert-BiLSTM-CRF network to extract static instance information. Examples of static information text are as follows: Fig. 9 The extraction results are shown in Fig.10 As shown. The BCCR network is used to reconstruct the spatiotemporal action sequence of the accident vehicle. The dynamic information text example is as follows Fig.11 As shown in the figure, the dynamic information text triple extraction results are as follows Fig.12As shown in the figure, the reconstruction results of the vehicle spatiotemporal action sequence are as follows: Fig.13 Based on the structured information extracted from the accident text, an example of a dynamic time series knowledge graph in the accident field is constructed as follows: Fig.14 shown.

[0040] After storing the dynamic knowledge graph generated by all scenarios in the Neo4j database, the example query condition is "find the accident scene that occurred at the intersection, and in this accident, the vehicle went straight and then turned left", and the search function for the above multi-condition query is implemented as follows Fig.15 As shown, a total of 7 query results are returned. Fig.16 It means to check a dynamic knowledge graph in the query results. The accident scene is a multi-vehicle scene that meets the search conditions and the scene semantics are relatively complex.

[0041] The accident scene in the dynamic information extraction example is reconstructed to obtain the results shown in Figures 17 (a), 17 (b), and 17 (c), where V1 is represented by a blue square, V2 is represented by a red square, and V3 is represented by a yellow square. Figure 17 (a) is the initial state of the vehicle, that is, V1 is going straight to the south, V2 is going straight to the west, and V3 is parked. Figure 17 (b) shows that V1 is turning left and V2 is still going straight. Figure 17 (c) shows that V1 and V2 collide. These three figures show the restoration of each action performed by the vehicle in turn.

[0042] After replacing V2 in the scene with an autonomous driving vehicle (CAV), the running scenario is shown in Figure 18 (a) and Figure 18 (b). The green dashed line is the path planning result of the CAV algorithm during operation. The results show that when the vehicle detects that V3 is parked in front, the trajectory planning algorithm successfully avoids the collision, but when it detects that V1 is turning at the intersection, the ADS algorithm fails to avoid the collision and the system stops running. This scenario exposes the potential loopholes of the ADS algorithm.

[0043] Another embodiment of the present invention further proposes a traffic accident report time series knowledge graph modeling system based on deep learning, comprising: Data preprocessing module, used to collect accident reports and perform data preprocessing; The accident domain ontology conceptual model construction module is used to perform concept, relationship and attribute modeling in the accident report after data preprocessing, and then obtain the accident domain ontology conceptual model; The accident scene static information acquisition module is used to use the Bert-BiLSTM-CRF network to extract the static information related to the accident text and model it as a named entity recognition (NER) task to obtain the static information of the accident scene; The accident vehicle spatiotemporal action sequence reconstruction module is used to use the Bert-CoAtt-CasRel network to model the accident dynamic evolution process and key event time sequence information extraction as a relationship extraction task to reconstruct the accident vehicle spatiotemporal action sequence; The accident domain dynamic knowledge graph generation module is used to generate a dynamic knowledge graph in the accident domain by mapping corresponding concepts, relationships and attributes based on the static information of the accident scene and the spatiotemporal action sequence of the accident vehicle through the obtained accident domain ontology concept model.

[0044] Another embodiment of the present invention further provides an electronic device, characterized in that it includes: a memory storing at least one instruction; and A processor executes instructions stored in the memory to implement the traffic accident report time series knowledge graph modeling method based on deep learning.

[0045] Another embodiment of the present invention further proposes a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the deep learning-based traffic accident report time series knowledge graph modeling method.

[0046] Exemplarily, the instructions stored in the memory may be divided into one or more modules / units, which are stored in a computer-readable storage medium and executed by the processor to complete the traffic accident report time series knowledge graph modeling method based on deep learning of the present invention. The one or more modules / units may be a series of computer-readable instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program in the server.

[0047] The electronic device may be a computing device such as a smart phone, a notebook, a PDA, and a cloud server. The electronic device may include, but is not limited to, a processor and a memory. Those skilled in the art will appreciate that the electronic device may also include more or fewer components, or a combination of certain components, or different components, for example, the electronic device may also include an input / output device, a network access device, a bus, etc.

[0048] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0049] The memory may be an internal storage unit of the server, such as a hard disk or memory of the server. The memory may also be an external storage device of the server, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the server. Furthermore, the memory may include both an internal storage unit of the server and an external storage device. The memory is used to store the computer-readable instructions and other programs and data required by the server. The memory may also be used to temporarily store data that has been output or is to be output.

[0050] It should be noted that the information interaction, execution process and other contents between the above-mentioned module units are based on the same concept as the method embodiment. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0051] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0052] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the camera device / terminal device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a disk or an optical disk.

[0053] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0054] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A traffic accident report time series knowledge graph modeling method based on deep learning, characterized in that: include: Collect accident reports and perform data preprocessing; In the accident report after data preprocessing, the concept, relationship and attribute modeling are carried out in turn to obtain the accident domain ontology conceptual model; The Bert-BiLSTM-CRF network is used to model the extraction of static information related to the accident text as a named entity recognition (NER) task to obtain static information about the accident scene. The Bert-CoAtt-CasRel network is used to model the accident dynamic evolution process and the extraction of key event time sequence information as a relationship extraction task, and the spatiotemporal action sequence of the accident vehicle is reconstructed; Based on the static information of the accident scene and the spatiotemporal action sequence of the accident vehicle, the corresponding concepts, relationships and attributes are mapped through the obtained accident domain ontology concept model to generate a dynamic knowledge graph of the accident domain.

2. The method for modeling traffic accident report time series knowledge graph based on deep learning according to claim 1 is characterized in that: In the step of collecting accident reports and performing data preprocessing, the data preprocessing includes: Convert all capital letters in the accident report text to lowercase so that the information extracted later is unified in form; Define a domain-specific dictionary, and then use regular matching to convert specific words in the accident report that meet the matching conditions into special symbols; Perform reference resolution to obtain the named entity that the pronoun actually refers to, and restore the pronouns in the accident report text to the objects they originally referred to; The sentences in the accident report text are processed by boundary division and the text is converted into several single sentences.

3. The method for modeling traffic accident report time series knowledge graph based on deep learning according to claim 1 is characterized in that: The steps of sequentially modeling concepts, relationships, and attributes in the accident report after data preprocessing to obtain the accident domain ontology concept model include: Based on component syntactic analysis, concept modeling is performed. Noun phrases are extracted using component syntactic analysis. The PAC algorithm is used to reduce the dimension of the vector representation of the phrases. The K-Means algorithm is used to cluster the noun phrases after the dimension reduction vector representation to obtain the accident domain concept ontology. Static road network relationship extraction is performed based on dependency syntactic analysis and external knowledge fusion. The relationship between concepts is obtained by using the dependency tree, connecting verbs or verb phrases in the dependency tree, and using the external knowledge network. Vehicle dynamic relations are extracted based on dependency syntactic analysis and external knowledge fusion. Verbs with vehicles as noun subjects are searched in sentence dependencies, and verb phrases containing corresponding verbs are searched in the syntactic structure tree. Combined with statistical methods, vehicle action information contained in the accident domain knowledge graph is screened out. Based on the attribute supplement of dependency syntactic analysis, a dependency tree containing concept sentences is constructed. By searching for subtrees containing concepts within a set distance, the attributes that modify the concepts are obtained. After statistical analysis, the attributes and corresponding data types are obtained.

4. The method for modeling traffic accident report time series knowledge graph based on deep learning according to claim 1 is characterized in that: The method of using the Bert-BiLSTM-CRF network to extract static information related to the accident text and modeling it as a named entity recognition (NER) task to obtain static information of the accident scene includes the following steps: Labeling data: using labeling tools to label various entities in the static road network of multiple accident report texts to obtain entity labels of the static road network; The labeled data is processed and input into the Bert-BiLSTM-CRF network for training and prediction, and multiple defined entities are identified; the Bert-BiLSTM-CRF network is divided into three modules: Bert layer, BiLSTM layer and CRF layer. The Bert layer is used to convert the input text sequence into a higher-dimensional semantic vector representation and provide a pre-trained representation; the BiLSTM layer uses the vector generated by the Bert layer as input to extract and encode contextual features and enhance the extraction of sequence features; the CRF layer decodes the features output by the BiLSTM layer, predicts the most likely label sequence, and ensures the consistency and rationality of the label sequence; By traversing the predicted output text and the corresponding labels, we first determine the road type and generate the corresponding number of branches. Then we extract the entity information in the sentence in turn and assign the entity information to the corresponding branches. We fill in the corresponding attributes of different branches, complete the static information instance extraction, and finally output the structured road network information.

5. The method for modeling traffic accident report time series knowledge graph based on deep learning according to claim 1 is characterized in that: The Bert-CoAtt-CasRel network is used to model the accident dynamic evolution process and the extraction of key event time sequence information as a relationship extraction task, and the reconstruction of the accident vehicle spatiotemporal action sequence includes the following steps: Use the annotation tool doccano to annotate the entities, relations, and triple labels of dynamic traffic participants in multiple accident report texts; The labeled data is processed and input into the Bert-CoAtt-CasRel network for training to extract relation triplets. The Bert-CoAtt-CasRel network is divided into Bert encoding module, CoAttention improvement module, subject recognition module, and object recognition module under relation conditions. The Bert-CoAtt-CasRel network adopts a cascade binary decoding mechanism to first identify all possible subjects in the sentence, and then predict the related relation relation and object objec for each identified subject. The extracted triples are reconstructed into a spatiotemporal sequence, starting from the execution of the perform relationship, searching for relationships, and generating an action list actionlist in the format of (vehicle, action, direction, road, lane, hit vehicle, timestamp). The subject of the triple (Vehicle, Perform, Action) corresponding to the perform relationship is assigned to the vehicle of the corresponding action list actionlist, and the object is assigned to the action. Taking the corresponding action as the subject, traverse the triples to search for other defined relationships. This process is executed for each perform relationship to generate a behavior sequence for each vehicle.

6. The method for modeling traffic accident report time series knowledge graph based on deep learning according to claim 1 is characterized in that: The static information based on the accident scene and the spatiotemporal action sequence of the accident vehicle is mapped to corresponding concepts, relationships and attributes through the obtained accident domain ontology concept model to generate the accident domain dynamic knowledge graph, which includes the following steps: Create scenarios and road network concept node instances in the graph database Neo4j, generate road network relationship edge instances, and connect the nodes; Create a vehicle node in the graph database Neo4j, traverse the action list from the initial moment, create vehicle action relationship edges, and obtain a dynamic time series knowledge graph in the accident field.

7. The method for modeling traffic accident report time series knowledge graph based on deep learning according to claim 1 is characterized in that: After generating the dynamic knowledge graph of the accident field, the application of the dynamic knowledge graph of the accident scene is instantiated and described, and the application of the dynamic knowledge graph of the accident scene is instantiated and described, including the following steps: The dynamic knowledge graphs generated by all scenarios are stored in the graph database Neo4j. Users can search for the dynamic knowledge graphs of accident scenarios that meet their requirements through different query condition combinations, and search by restricting accident scenarios with multiple conditions. Based on the generated dynamic knowledge graph, a trajectory reproduction algorithm is used, supplemented by mathematical constraints to complete the vehicle's movements and directions, reconstruct the accident scenario, replace any vehicle in the scenario with an autonomous driving vehicle, and test whether the path planning of the autonomous driving system ADS can avoid accidents.

8. A traffic accident report time series knowledge graph modeling system based on deep learning, characterized in that: include: Data preprocessing module, used to collect accident reports and perform data preprocessing; The accident domain ontology conceptual model construction module is used to perform concept, relationship and attribute modeling in the accident report after data preprocessing, and then obtain the accident domain ontology conceptual model; The accident scene static information acquisition module is used to use the Bert-BiLSTM-CRF network to extract the static information related to the accident text and model it as a named entity recognition (NER) task to obtain the static information of the accident scene; The accident vehicle spatiotemporal action sequence reconstruction module is used to use the Bert-CoAtt-CasRel network to model the accident dynamic evolution process and key event time sequence information extraction as a relationship extraction task to reconstruct the accident vehicle spatiotemporal action sequence; The accident domain dynamic knowledge graph generation module is used to generate a dynamic knowledge graph in the accident domain by mapping corresponding concepts, relationships and attributes based on the static information of the accident scene and the spatiotemporal action sequence of the accident vehicle through the obtained accident domain ontology concept model.

9. An electronic device, characterized in that: include: A memory storing at least one instruction; and A processor executes instructions stored in the memory to implement the deep learning-based time series knowledge graph modeling method for traffic accident reports as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, it implements the deep learning-based time series knowledge graph modeling method for traffic accident reports as described in any one of claims 1 to 7.

Images