Accident prediction model training method and device, equipment and storage medium

Abstract

The invention relates to the field of digital image processing, and relates to an accident prediction model training method comprising the following steps: obtaining a training data set which comprises a plurality of image frames collected by an automobile data recorder; constructing a network structure of the accident prediction model; the training data set is used to train a network of the accident prediction model, in each training, a loss function of the accident prediction model to be trained is determined according to the features of the image frames and an accident prediction result output by the accident prediction model to be trained, and the loss function comprises consistency comparison loss and prediction loss; and optimizing network parameters of the to-be-trained accident prediction model according to the loss function to obtain a final accident prediction model. The invention further provides an accident prediction model training device, equipment and a storage medium. The method can improve the understanding capability of the traffic accident model for the image frame, and improves the accuracy of the accident prediction result.

Description

Technical Field

[0001] The present invention relates to the field of digital image processing, and in particular to an accident prediction model training method, device, equipment and storage medium. Background Art

[0002] Automatic assisted driving, also known as "Advanced Driver Assistant System" (ADAS). ADAS uses various sensors installed on the car to perceive the surrounding environment in real time during driving, collect data, identify, detect and track static and dynamic objects, and perform system calculations and analysis in combination with navigation map data, so as to make the driver aware of possible dangers in advance and effectively improve the comfort and safety of car driving.

[0003] At present, early accident prediction, usually using dashcam videos, plays a vital role in improving the safety of modern intelligent transportation systems. Accurately predicting accidents in advance can significantly reduce traffic accidents and improve overall road safety. However, despite extensive research efforts to capture more visual information by adopting different feature extraction methods within the same frame, the consistency between features within the same frame and the differences between features between different frames have not been fully valued. Summary of the invention

[0004] In order to improve the ability of a traffic accident model to understand image frames and improve the accuracy of accident prediction results, an accident prediction model training method, device, equipment and storage medium are provided.

[0005] In order to achieve the above-mentioned object of the present invention, the present invention provides an accident prediction model training method, comprising:

[0006] Acquire a training data set, wherein the data training set includes a plurality of image frames captured by a driving recorder;

[0007] Construct the network structure of the accident prediction model;

[0008] The network of the accident prediction model is trained using the training data set. In each training, a loss function of the accident prediction model to be trained is determined according to the features of the image frame and the accident prediction results output by the accident prediction model to be trained, wherein the loss function includes a consistency comparison loss and a prediction loss;

[0009] The network parameters of the accident prediction model to be trained are optimized according to the loss function to obtain the final accident prediction model.

[0010] Optionally, the features of the image frame include frame-level features of the image frame and target-level features of the image frame, wherein the frame-level features are global features of the image frame, and the target-level features are local features corresponding to preset traffic participants in the image frame;

[0011] The consistency contrast loss of the accident prediction model to be trained is determined according to the frame-level features and the object-level features of the image frame.

[0012] Optionally, determining the consistency contrast loss of the accident prediction model to be trained according to the frame-level features and the object-level features of the image frame includes:

[0013] The frame-level features and object-level features from the same image frame are regarded as positive sample pairs;

[0014] The frame-level features and object-level features from different image frames are regarded as negative sample pairs;

[0015] The cosine similarity is calculated for the positive sample pairs and the negative sample pairs respectively, and the consistency comparison loss is determined according to the cosine similarity of the positive sample pairs and the cosine similarity of the negative sample pairs.

[0016] Optionally, the consistency contrast loss L consistency The calculation formula is:

[0017]

[0018] L frame-object and L object-frame is the symmetrical contrast loss, where L frame-object is the loss of the frame-level features of the preset image frame and the target-level features of all image frames in the training dataset, L object-frame is the loss of the target-level features of the preset image frame and the frame-level features of all image frames in the training dataset;

[0019] L frame-object The expression is:

[0020]

[0021] Where N represents the number of image frames in the training data set, s(.) represents the cosine similarity calculation, and τ represents the temperature coefficient used to adjust the learning effect; i represents the object-level feature of the i-th image frame, f i represents the frame-level features of the i-th image frame; i represents the index of the preset image frame, j represents the index of the image frame in the training data set, O j represents the object-level features of the j-th image frame;

[0022] L object-frame The expression is:

[0023]

[0024] f j represents the frame-level features of the j-th image frame.

[0025] Optionally, the consistency contrast loss L consistency The calculation formula is

[0026]

[0027] L frame-object and L object-frame is the symmetrical contrast loss, where L frame-object is the loss of the frame-level features of the preset image frame and the target-level features of all image frames in the training dataset, L object-frame is the loss of the target-level features of the preset image frame and the frame-level features of all image frames in the training dataset;

[0028] L frame-object The expression is:

[0029]

[0030] Where N represents the number of image frames in the training data set, s(.) represents the cosine similarity calculation, and τ represents the temperature coefficient used to adjust the learning effect; i represents the object-level feature of the i-th image frame, f i represents the frame-level features of the i-th image frame; i represents the index of the preset image frame, j represents the index of the image frame in the training data set, O j represents the object-level features of the j-th image frame;

[0031] L object-frame The expression is:

[0032]

[0033] f j represents the frame-level features of the j-th image frame.

[0034] Optionally, generating a directed graph based on fusion information of a plurality of image frames, and generating a risk score of a traffic accident based on the directed graph network includes:

[0035] Each image frame observed by the driving recorder is recorded as a node;

[0036] The nodes corresponding to adjacent image frames are connected by arrows to generate a directed graph between image frames; the arrows point from the previous image frame to the current image frame;

[0037] The graph attention layer is used to process the directed graph to output the predicted probability of traffic accidents and obtain the risk score of traffic accidents.

[0038] Optionally, the prediction loss is expressed as

[0039] Among them, a t Represents the predicted probability of a traffic accident, N represents the number of image frames in the training data set, i represents the index of the image frame in the training data set, and y is the true label of the image frame.

[0040] In order to solve the above problems, the present invention also provides an accident prediction model training device, the training device comprising:

[0041] An acquisition module is used to acquire a training data set, where the data training set includes a number of image frames collected by a driving recorder;

[0042] A construction module is used to construct the network structure of the accident prediction model;

[0043] A training module, used to train the network of the accident prediction model using the training data set. In each training, a loss function of the accident prediction model to be trained is determined according to the features of the image frame and the accident prediction results output by the accident prediction model to be trained. The loss function includes a consistency comparison loss and a prediction loss.

[0044] The optimization module is used to optimize the network parameters of the accident prediction model to be trained according to the loss function to obtain the final accident prediction model.

[0045] In order to solve the above problem, the present invention further provides an electronic device, the electronic device comprising:

[0046] at least one processor; and,

[0047] a memory communicatively connected to the at least one processor; wherein,

[0048] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the accident prediction model training method described above.

[0049] In order to solve the above problems, the present invention also provides a computer-readable storage medium, in which at least one computer program is stored. The at least one computer program is executed by a processor in an electronic device to implement the accident prediction model training method described above.

[0050] In summary, this application includes the following beneficial technical effects:

[0051] Using feature consistency representation to predict traffic accidents. The purpose of consistency comparison is to help the model better capture the consistency between different features of the same frame. Introducing contrastive learning into accident prediction, amplifying the similarities between different features of the same frame, and maximizing the differences between features of different frames, in order to distinguish the differences between features of different frames, thus achieving better feature extraction and fusion, improving the ability of the traffic accident model to understand image frames, and improving the accuracy of accident prediction results;

[0052] Directed graphs are used to learn the temporal relationship between image frames. The ability to combine multiple consecutive image frames using directed graphs provides more critical information and helps understand the correlation between image frames, thereby facilitating better accident prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 A schematic diagram of a flow chart of an accident prediction model training method provided by an embodiment of the present invention;

[0054] Figure 2 A system structure diagram of an accident prediction model provided by an embodiment of the present invention;

[0055] Figure 3 A directed graph of image frames in a driving recorder video provided by an embodiment of the present invention;

[0056] Figure 4 A schematic diagram of a consistency comparison loss calculation process provided by an embodiment of the present invention;

[0057] Figure 5 A diagram showing the prediction results of a driving recorder video by an accident prediction model provided by an embodiment of the present invention;

[0058] Figure 6 A functional module diagram of an accident prediction model training device provided by one embodiment of the present invention;

[0059] Figure 7 A schematic diagram of the structure of an electronic device for implementing the accident prediction model training method provided in one embodiment of the present invention.

[0060] Figure numerals: 10, processor; 11, memory; 12, communication bus; 13, communication interface; 61, acquisition module; 62, construction module; 63, training module; 64, optimization module.

[0061] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0062] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0063] In the description of the present invention, it is necessary to understand that the terms "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0064] In the description of the present invention, unless otherwise specified and limited, it should be noted that the terms "installed", "connected" and "connected" should be understood in a broad sense. For example, it can be a mechanical connection or an electrical connection, or it can be the internal connection between two components. It can be a direct connection or an indirect connection through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to the specific circumstances.

[0065] Reference Figure 1 FIG. 1 is a flow chart of an accident prediction model training method provided by an embodiment of the present invention. In this embodiment, the accident prediction model training method includes:

[0066] S1. Obtain a training data set, where the data training set includes a number of image frames captured by a driving recorder.

[0067] The several image frames of the dashcam in the training data set can be screened by the staff from the dashcam video clips, or the currently commonly used data sets recording dashcam video clips can be used, such as the dashcam accident data set (DAD) and the car collision data set (CCD).

[0068] DAD stands for Driver Attention Detection, which is a dataset widely used for dashcam accident prediction. The DAD dataset includes 678 accident videos.

[0069] CCD stands for Car Crash Datset, which is a video dataset for traffic accident analysis. It contains 1,500 cropped videos from YouTube (50 frames per video, 10 seconds per frame) and 3,000 regular videos from the BDD100K dataset, which are shot by driving recorders.

[0070] S2. Construct the network structure of the accident prediction model.

[0071] Reference Figure 2 ,In this embodiment, the network of the accident prediction model includes a feature extraction module, a feature fusion module, a generation module and a scoring module;

[0072] The feature extraction module is used to extract frame-level features and target-level features of image frames; frame-level features are global features of image frames, target-level features are local features corresponding to traffic participants preset in image frames, and targets are traffic participants related to traffic accidents in image frames; traffic participants refer to moving objects such as people, vehicles and animals in motion in vehicles, as well as objects that are about to move, such as people, vehicles or animals that are about to enter motion; traffic participants also include stationary objects on or near the road (such as green plants, guardrails, street lights, buildings, temporarily parked vehicles and some temporarily placed objects, etc.); traffic participants can also be other objects that affect the operation of the vehicle, which will not be listed here one by one for the sake of brevity of the specification.

[0073] In this embodiment, a target extractor is used to extract targets in image frames. The target extractor can be Faster R-CNN or Cascade R-CNN. Faster R-CNN is a two-stage target detection algorithm that can identify targets in dashcam image frames. Cascade ...

[0074] In this embodiment, a two-stream inflated 3D convolutional network (Two-Stream Inflated 3D ConvNets, referred to as I3D) is used to extract frame-level features. The frame-level features extracted by I3D contain time information, so that the accident prediction model can better analyze the time relationship in the data set and improve the accuracy of traffic accident recognition results.

[0075] The feature fusion module is used to fuse the frame-level features and the object-level features of the image frame to obtain the fusion features of the image frame;

[0076] In a preferred implementation of this embodiment, a global-local feature fusion module based on an attention mechanism is used to fuse frame-level features and target-level features, wherein the local features are target-level features and the global features are frame-level features.

[0077] The global-local feature fusion module includes an upper branch and a lower branch. Taking the upper branch as an example, the target-level feature first passes through a linear layer to obtain the query component Q; then, the frame-level feature passes through two independent fully connected layers to obtain the key component K and the value component V; then the query component Q of the target-level feature, the value component V of the frame-level feature, and the key component K of the frame-level feature are input into the attention calculation module to obtain the upper branch fusion feature.

[0078] When the lower branch of the global-local feature fusion module processes the frame-level features and the target-level features, the frame-level features first pass through a linear layer to obtain the query component Q; then, the target-level features pass through two independent fully connected layers to obtain the key component K and the value component V; then, the query component Q of the frame-level features, the value component V of the target-level features, and the key component K of the target-level features are input into the attention calculation module to obtain the lower-branch fusion features.

[0079] The upper branch fusion features, lower branch fusion features, frame-level features and object-level features are concatenated to obtain the final fusion features.

[0080] The scoring module is used to generate a directed graph based on the fusion information of several image frames, generate a risk score of a traffic accident based on the directed graph network, and obtain an accident prediction result.

[0081] Reference Figure 3 In this embodiment, a directed graph is generated based on the fusion information of several image frames, and a processing method for generating a risk score of a traffic accident based on the directed graph network is as follows:

[0082] S21, recording each image frame observed by the driving recorder as a node;

[0083] S22, connecting nodes corresponding to adjacent image frames using arrows to generate a directed graph between image frames; the arrows point from the previous image frame to the current image frame;

[0084] Dashcam video data is a collection of time series, and each image frame contains specific visual information. Single-frame information is not enough for accurate prediction. Using directed graphs to combine multiple consecutive image frames provides more critical information. Understanding the correlation between image frames helps to better predict accidents. Use directed graphs to learn the temporal relationship between frames. In the directed graph, each node represents an image frame, and the node embedding is the single-frame feature previously obtained. The edges in the directed graph are directed, pointing only from the previous frame to the current frame, to better simulate the actual accident prediction scenario - only using past information for prediction, not future events.

[0085] S23. Use the graph attention layer to process the directed graph to output the predicted probability of traffic accidents and obtain the risk score of traffic accidents.

[0086] The adjacency matrix of a directed graph in the time domain is defined as follows:

[0087]

[0088] Where i represents the current image frame, j represents the index of the image frame before the current image frame i in the training data set, and k is the number of image frames observed in the dashcam, that is, how many previous frames will affect the prediction result of the current frame at most. For example, if i is 50, j can be 0-49.

[0089] In actual processing, after obtaining the directed graph, the graph attention network (GAT) is used to process the directed graph, and then the predicted probability of future accidents is obtained through the fully connected layer. The risk score of the image frame for future accidents and the label of the image frame are determined according to the predicted probability of future accidents. In this embodiment, the traffic conditions of the image frame are labeled with soft labels, and the labels include normal labels and abnormal labels. The normal label indicates that the probability of a traffic accident in the image frame in the future is low, and the abnormal label indicates that the probability of a traffic accident in the future is high, and a traffic accident may occur.

[0090] In this embodiment, the predicted probability is recorded as a t , predicted probability a t The expression is

[0091] a t =FC(GAT(F t ,A T ))

[0092] Among them, F t is the fusion feature of the image frame, A T is the adjacency matrix of the directed graph, GAT is the graph attention layer processing, and FC is the fully connected layer processing.

[0093] S3. Train the network of the accident prediction model using the training data set. In each training, determine the loss function of the accident prediction model to be trained based on the features of the image frame and the accident prediction results output by the accident prediction model to be trained.

[0094] Specifically, the loss function includes consistency contrast loss and prediction loss.

[0095] The features of the image frame include frame-level features of the image frame and target-level features of the image frame. The consistency comparison loss of the accident prediction model to be trained is determined according to the frame-level features and target-level features of the image frame. The prediction loss is calculated based on the risk score and label output by the accident prediction model.

[0096] The expression of the loss function L of the accident prediction model to be trained is:

[0097] L=L accident +ωL consistency ,

[0098] Among them, L accident represents the prediction loss, L consistency represents the consistency contrast loss, ω is a hyperparameter, and ω is used to adjust the influence of the consistency loss on the overall loss to achieve the effect of assisting model learning. After a large number of experiments, ω=0.02 is selected in the preferred implementation of this embodiment.

[0099] Reference Figure 4 , when calculating the consistency contrast loss:

[0100] The frame-level features and object-level features from the same image frame are regarded as positive sample pairs;

[0101] The frame-level features and object-level features from different image frames are regarded as negative sample pairs;

[0102] The cosine similarity is calculated for the positive sample pairs and the negative sample pairs respectively, and the consistency comparison loss is determined according to the cosine similarity of the positive sample pairs and the cosine similarity of the negative sample pairs.

[0103] Consistency contrast loss L consistency The calculation formula is:

[0104]

[0105] Among them, L frame-object and L object-frame is the symmetrical contrast loss, where L frame-object is the loss of the frame-level features of the preset image frame and the target-level features of all image frames in the training dataset, L object-frameIt is the loss of the target-level features of a preset image frame and the frame-level features of all image frames in the training data set; in other words, it can be understood as when the target-level representation of a certain image frame is fixed, the loss of all frame-level representations and the target-level representation of this image frame is calculated; and when the frame-level representation of a certain frame is fixed, the loss of all target-level representations and the frame-level representation of this image frame is calculated.

[0106] L frame-object The expression is:

[0107]

[0108] Where N represents the number of image frames in the training data set, s(.) represents the cosine similarity calculation, and τ represents the temperature coefficient, which is used to adjust the learning effect of the model. i represents the object-level feature of the i-th image frame, f i represents the frame-level features of the i-th image frame; i represents the index of the preset image frame, j represents the index of the image frame in the training data set, i and j can be different image frames or the same image frame, j represents the object-level features of the j-th image frame;

[0109] L object-frame The expression is:

[0110]

[0111] f j represents the frame-level features of the j-th image frame.

[0112] The expression of prediction loss is

[0113] Among them, a t Represents the predicted probability of a traffic accident, N represents the number of image frames in the training data set, i represents the index of the image frame in the training data set, and y is the true label of the image frame.

[0114] In the preferred implementation of this embodiment, soft labels are used to guide the learning process of the model, transforming the original classification task into a regression task. This method enables the model to learn predictions that more accurately reflect real-world scenarios.

[0115] Existing studies often use cross entropy loss as the loss function of the prediction task and model the accident prediction problem as a classification problem. However, in the real world, clues that can predict accidents will gradually appear over time, and the risk of accidents will gradually increase. Correspondingly, the labels used to guide the model should not be constant at 0 or 1. Therefore, we model accident prediction as a regression problem and change the labels used to guide model learning to a form that is more in line with real scenarios. For accident videos, we let the labels gradually increase over time. The specific implementation method is:

[0116]

[0117] Wherein, t represents the current frame, and y represents the label value modified based on the real label of the image frame. In this embodiment, the loss function of the prediction result of the accident prediction model to be trained is the mean square error loss (Mean Squared Error Loss).

[0118] S4. Optimize the network parameters of the accident prediction model to be trained according to the loss function to obtain the final accident prediction model.

[0119] To demonstrate the effectiveness of the proposed consistency contrast loss, we conduct comprehensive ablation experiments. The results of this experiment are listed in Table 1.

[0120] Table 1 Ablation study of consistency contrast loss on DAD dataset

[0121]

[0122] The analysis results show that eliminating the consistency contrast loss will lead to a significant drop in performance indicators. Specifically, mAP dropped by 2.5% and mTTA was reduced by 0.14 seconds. These drops highlight the key role that consistency contrast loss plays in improving model performance. “No consistency contrast loss” means that the final loss function does not include consistency contrast loss.

[0123] mAP stands for Mean Average Precision, which means the mean precision. mAP is a key indicator for evaluating target detection models. mAP is used to measure the detection performance of a model for multiple categories. MTTA stands for Mean Time To Acknowledge, which means the average confirmation time.

[0124] The observed decrease in mAP indicates that without the auxiliary term, the model's ability to accurately predict accidents is reduced, while the decrease in mTTA indicates a decrease in early accident prediction ability. Therefore, these findings confirm the effectiveness of incorporating the consistency contrast loss into the loss function, demonstrating its importance in improving the accuracy and timeliness of accident prediction.

[0125] We are Figure 5 Qualitative results are shown in , illustrating the prediction results of the DAD dataset for accident and normal videos. Figure 5 (a) and Figure 5 (b) shows the prediction results of the accident video, while Figure 5 (c) and Figure 5 (d) shows the prediction results of normal videos; Figure 5 (a) Figure 5 (b) and Figure 5 (c) is an example of a correct prediction, Figure 5 (d) is an example of an incorrect prediction.

[0126] Our accident prediction model Figure 5 Early accident prediction is successfully achieved in (a), where the candidate targets appear early in the video and consistently exhibit high-risk motions, allowing the model to assign higher risk scores at an early stage.

[0127] on the contrary, Figure 5 (b) shows a relatively delayed accident prediction. In this case, the candidate object appears late in the video, which causes the model to lack sufficient information to make early accident inferences.

[0128] Reference Figure 6 Based on the same inventive concept, an embodiment of the present invention provides an accident prediction model training device.

[0129] The accident prediction model training device described in the present invention can be loaded in an electronic device. According to the functions to be implemented, the accident prediction model training device includes an acquisition module 61, a construction module 62, a training module 63 and an optimization module 64. The acquisition module 61 can acquire a training data set, and the data training set includes a number of image frames collected by a driving recorder; the construction module 62 can build a network structure of an accident prediction model; the training module 63 can use the training data set to train the network of the accident prediction model. In each training, the loss function of the accident prediction model to be trained is determined according to the characteristics of the image frame and the accident prediction results output by the accident prediction model to be trained. The loss function includes consistency comparison loss and prediction loss; the optimization module 64 can optimize the network parameters of the accident prediction model to be trained according to the loss function to obtain the final accident prediction model. The module described in the present invention can also be called a unit, which refers to a series of computer program segments that can be executed by an electronic device processor and can complete fixed functions, which are stored in the memory of the electronic device.

[0130] The various variations and specific examples of the accident prediction model training method provided in the above embodiment are also applicable to the accident prediction model training device of the present embodiment. Through the above detailed description of the accident prediction model training method, those skilled in the art can clearly know the implementation method of the accident prediction model training device in the present embodiment. For the sake of brevity of the specification, it will not be described in detail here.

[0131] The present application also discloses an electronic device, such as Figure 7 , which is a schematic diagram of the structure of an electronic device of an accident prediction model training method provided by an embodiment of the present invention. The electronic device may include at least one processor 10, a memory 11 connected to the at least one processor for communication, a communication bus 12, and a communication interface 13, and may also include a computer program stored in the memory 11 and executable on the processor 10, such as an accident prediction model training method program.

[0132] In some embodiments, the processor 10 may be composed of an integrated circuit, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and combinations of various control chips. The processor 10 is the control core (Control Unit) of the electronic device, and uses various interfaces and lines to connect the various components of the entire electronic device, and executes or executes the programs or modules stored in the memory 11 (for example, executing the accident prediction model training method, etc.), and calls the data stored in the memory 11 to execute various functions of the electronic device and process data.

[0133] The memory 11 includes at least one type of readable storage medium, and the readable storage medium includes a flash memory, a mobile hard disk, a multimedia card, a card-type memory (for example, SD or DX memory, etc.), a magnetic memory, a disk, an optical disk, etc. In some embodiments, the memory 11 may be an internal storage unit of an electronic device, such as a mobile hard disk of the electronic device. In other embodiments, the memory 11 may also be an external storage device of an electronic device, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device. Furthermore, the memory 11 may also include both an internal storage unit of the electronic device and an external storage device. The memory 11 can not only be used to store application software and various types of data installed in the electronic device, such as the code of the accident prediction model training method program, but also can be used to temporarily store data that has been output or is to be output.

[0134] The communication bus 12 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The bus is configured to realize connection and communication between the memory 11 and at least one processor 10, etc.

[0135] The communication interface 13 is used for communication between the above-mentioned electronic device and other devices, including a network interface and a user interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the electronic device and other electronic devices. The user interface may be a display (Display), an input unit (such as a keyboard (Keyboard)), and optionally, the user interface may also be a standard wired interface, a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode, organic light-emitting diode) touch device, etc. Among them, the display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device and to display a visual user interface.

[0136] Figure 7 Only an electronic device with components is shown, and those skilled in the art will understand that Figure 7The structure shown does not constitute a limitation on the electronic device, and may include fewer or more components than shown, or combine certain components, or arrange the components differently. For example, although not shown, the electronic device may also include a power supply (such as a battery) for supplying power to each component. Preferably, the power supply may be logically connected to at least one processor 10 through a power management device, so that functions such as charging management, discharging management, and power consumption management are implemented through the power management device. The power supply may also include any components such as one or more DC or AC power supplies, recharging devices, power failure detection circuits, power converters or inverters, power status indicators, etc. The electronic device may also include a variety of sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be repeated here.

[0137] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited by this structure.

[0138] Furthermore, if the module / unit integrated in the electronic device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile.

[0139] The present application embodiment provides a computer-readable storage medium, for example, including: any entity or device, recording medium, USB flash drive, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM) that can carry the computer program code. The computer-readable storage medium stores a computer program that can be loaded by a processor and execute the accident prediction model training method of the above embodiment.

[0140] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", "an implementation", "a preferred implementation" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0141] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

Claims

1. A method for training an accident prediction model, characterized in that: Acquire a training data set, wherein the data training set includes a plurality of image frames captured by a driving recorder; Construct the network structure of the accident prediction model; The network of the accident prediction model is trained using the training data set. In each training, a loss function of the accident prediction model to be trained is determined according to the features of the image frame and the accident prediction results output by the accident prediction model to be trained, wherein the loss function includes a consistency comparison loss and a prediction loss; The network parameters of the accident prediction model to be trained are optimized according to the loss function to obtain the final accident prediction model.

2. The accident prediction model training method according to claim 1, characterized in that: The features of the image frame include frame-level features of the image frame and target-level features of the image frame, wherein the frame-level features are global features of the image frame, and the target-level features are local features corresponding to preset traffic participants in the image frame; The consistency contrast loss of the accident prediction model to be trained is determined according to the frame-level features and the object-level features of the image frame.

3. The accident prediction model training method according to claim 2, characterized in that: The step of determining the consistency comparison loss of the accident prediction model to be trained according to the frame-level features and the object-level features of the image frame includes: The frame-level features and object-level features from the same image frame are regarded as positive sample pairs; The frame-level features and object-level features from different image frames are regarded as negative sample pairs; The cosine similarity is calculated for the positive sample pairs and the negative sample pairs respectively, and the consistency comparison loss is determined according to the cosine similarity of the positive sample pairs and the cosine similarity of the negative sample pairs.

4. The accident prediction model training method according to claim 3, characterized in that: Consistency contrast loss L consistency The calculation formula is: L frame-object and L object-frame is the symmetrical contrast loss, where L frame-object is the loss of the frame-level features of the preset image frame and the target-level features of all image frames in the training dataset, L object-frame is the loss of the target-level features of the preset image frame and the frame-level features of all image frames in the training dataset; L frame-object The expression is: Where N represents the number of image frames in the training data set, s(.) represents the cosine similarity calculation, and τ represents the temperature coefficient; i represents the object-level feature of the i-th image frame, f i represents the frame-level features of the i-th image frame; i represents the index of the preset image frame, j represents the index of the image frame in the training data set, O j represents the object-level features of the j-th image frame; L object-frame The expression is: f j represents the frame-level features of the j-th image frame.

5. The accident prediction model training method according to any one of claims 1 to 4, characterized in that: The network of the accident prediction model includes a feature extraction module, a feature fusion module, a generation module and a scoring module; The feature extraction module is used to extract frame-level features and object-level features of image frames; The feature fusion module is used to fuse the frame-level features and the object-level features of the image frame to obtain the fusion features of the image frame; The scoring module is used to generate a directed graph based on the fusion information of several image frames, generate a risk score of a traffic accident based on the directed graph network, and obtain an accident prediction result.

6. The accident prediction model training method according to claim 5, characterized in that: The method of generating a directed graph based on the fusion information of a plurality of image frames and generating a risk score of a traffic accident based on the directed graph network includes: Each image frame observed by the driving recorder is recorded as a node; The nodes corresponding to adjacent image frames are connected by arrows to generate a directed graph between image frames; the arrows point from the previous image frame to the current image frame; The graph attention layer is used to process the directed graph to output the predicted probability of traffic accidents and obtain the risk score of traffic accidents.

7. The accident prediction model training method according to claim 6, characterized in that: The expression of prediction loss is Among them, a t Represents the predicted probability of a traffic accident, N represents the number of image frames in the training data set, i represents the index of the image frame in the training data set, and y is the true label of the image frame.

8. An accident prediction model training device, used to implement the accident prediction model training method according to any one of claims 1 to 7, characterized in that: include: An acquisition module (61) is used to acquire a training data set, wherein the data training set includes a plurality of image frames collected by a driving recorder; A construction module (62) for constructing a network structure of an accident prediction model; A training module (63) is used to train the network of the accident prediction model using the training data set. In each training, a loss function of the accident prediction model to be trained is determined according to the features of the image frame and the accident prediction result output by the accident prediction model to be trained, wherein the loss function includes a consistency comparison loss and a prediction loss. The optimization module (64) is used to optimize the network parameters of the accident prediction model to be trained according to the loss function to obtain the final accident prediction model.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor (10); and, a memory (11) communicatively connected to the at least one processor (10); The memory (11) stores a computer program executable by the at least one processor (10), and the computer program is executed by the at least one processor (10) so that the at least one processor (10) can execute the accident prediction model training method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer storage medium stores a computer program; when the computer program is executed by a processor, the accident prediction model training method according to any one of claims 1 to 7 is implemented.

Images