Road traffic accident detection method and system based on holographic perception

Abstract

The present application provides a road traffic accident detection method and system based on holographic perception. First, holographic perception technology is used to comprehensively capture multi-dimensional information in road traffic scenes, effectively improving the sensitivity and accuracy of accident detection. By repeatedly loading the target template holographic road traffic perception flow to the traffic accident knowledge point description network, and generating high-likelihood traffic accident knowledge points based on comparative analysis, it not only significantly enhances the recognition ability of complex traffic scenes, but also ensures the reliability of the prediction results. In addition, by calculating and training the network in a targeted manner based on the first network training cost, the network performance is effectively optimized, so that the converged traffic accident knowledge point description network can more accurately predict potential accident risks when facing actual traffic conditions, thereby significantly improving the real-time and accuracy of road traffic accident detection, helping to reduce the occurrence of traffic accidents and ensure road traffic safety.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and more specifically, to a road traffic accident detection method and system based on holographic perception. Background Art

[0002] With the acceleration of urbanization and the continuous increase in the number of cars, road traffic safety issues are becoming increasingly serious. Traditional road traffic accident detection methods often rely on manual monitoring and post-analysis, which have the disadvantages of delayed response and low efficiency, making it difficult to effectively prevent traffic accidents. Therefore, it is particularly important to develop a technology that can detect road traffic accidents in real time and accurately.

[0003] In recent years, holographic perception technology has shown great application potential in the field of intelligent transportation with its all-round and multi-angle data collection capabilities. By integrating multiple sensors (such as lidar, millimeter-wave radar, cameras, etc.), holographic perception technology can obtain accurate three-dimensional models of road traffic environments in real time, providing a rich data source for road traffic accident detection. However, how to use these massive data for efficient and accurate traffic accident detection is still a technical problem that needs to be solved urgently.

[0004] Most of the related technologies are based on single sensor data or simple multi-sensor fusion methods. These methods often have problems such as high false alarm rate and large missed alarm rate when dealing with complex and changeable road traffic scenes. In addition, due to the diversity and dynamics of the road traffic environment, traditional rule-based detection methods are difficult to adapt to various emergencies, limiting their effectiveness in practical applications. Summary of the invention

[0005] In view of the above-mentioned problems, in combination with the first aspect of the present application, an embodiment of the present application provides a road traffic accident detection method based on holographic perception, the method comprising:

[0006] Loading the template holographic road traffic perception stream in the basic template holographic road traffic perception stream sequence into multiple traffic accident knowledge point description networks, generating high-likelihood traffic accident knowledge points corresponding to each of the traffic accident knowledge point description networks;

[0007] According to the first a priori traffic accident knowledge point of the template holographic road traffic perception flow and the comparative analysis of the high likelihood traffic accident knowledge point corresponding to each traffic accident knowledge point description network, the target template holographic road traffic perception flow is extracted from the basic template holographic road traffic perception flow sequence, and the extracted target template holographic road traffic perception flow constitutes the target template holographic road traffic perception flow sequence;

[0008] Loading the target template holographic road traffic perception stream into each of the traffic accident knowledge point description networks to generate a first positive prediction confidence corresponding to each traffic accident knowledge point;

[0009] Calculating a first network training cost corresponding to each of the traffic accident knowledge point description networks according to the first a priori traffic accident knowledge point of each of the traffic accident knowledge points and the first positive prediction confidence;

[0010] For each of the traffic accident knowledge point description networks, training the traffic accident knowledge point description network according to the first network training cost corresponding to the traffic accident knowledge point description network;

[0011] The target template holographic road traffic perception flow sequence is overwritten with the basic template holographic road traffic perception flow sequence, and the step of loading the template holographic road traffic perception flow in the basic template holographic road traffic perception flow sequence into the multiple traffic accident knowledge point description networks is returned until the training cost of the first network converges, and the converged traffic accident knowledge point description network is determined from the multiple traffic accident knowledge point description networks.

[0012] On the other hand, an embodiment of the present application also provides a road traffic accident detection system based on holographic perception, including a processor and a machine-readable storage medium, wherein the machine-readable storage medium is connected to the processor, the machine-readable storage medium is used to store programs, instructions or codes, and the processor is used to execute the programs, instructions or codes in the machine-readable storage medium to implement the above method.

[0013] Based on the above aspects, the embodiment of the present application realizes efficient and accurate detection of road traffic accidents by constructing and training multiple traffic accident knowledge point description networks. First, holographic perception technology is used to comprehensively capture multi-dimensional information in road traffic scenes, effectively improving the sensitivity and accuracy of accident detection. By repeatedly loading the target template holographic road traffic perception flow to the traffic accident knowledge point description network, and generating high-likelihood traffic accident knowledge points based on comparative analysis, it not only significantly enhances the network's recognition ability for complex traffic scenes, but also ensures the reliability of the prediction results. In addition, by calculating and training the network in a targeted manner based on the first network training cost, the network performance is effectively optimized, so that the converged traffic accident knowledge point description network can more accurately predict potential accident risks when facing actual traffic conditions, and provide decision-making support for traffic management departments, thereby significantly improving the real-time and accuracy of road traffic accident detection, helping to reduce the occurrence of traffic accidents and ensure road traffic safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1It is a schematic diagram of the execution flow of the road traffic accident detection method based on holographic perception provided in an embodiment of the present application.

[0015] Figure 2 It is a schematic diagram of the hardware architecture of a road traffic accident detection system based on holographic perception provided in an embodiment of the present application. DETAILED DESCRIPTION

[0016] The present application will be described in detail below with reference to the accompanying drawings. Figure 1 It is a flow chart of a road traffic accident detection method based on holographic perception provided by an embodiment of the present application. The road traffic accident detection method based on holographic perception is introduced in detail below.

[0017] Step S110, loading the template holographic road traffic perception flow in the basic template holographic road traffic perception flow sequence into multiple traffic accident knowledge point description networks, and generating high-likelihood traffic accident knowledge points corresponding to each of the traffic accident knowledge point description networks.

[0018] In detail, the template holographic road traffic perception flow is a highly detailed and comprehensive representation of road traffic data, which contains real-time and dynamic information on roads, vehicles, drivers, environmental conditions, traffic facilities, etc., which can be collected through various sensors (such as cameras, radars, lidars, etc.), and after processing and analysis, a data flow that can comprehensively reflect the road traffic conditions is formed.

[0019] The traffic accident knowledge point description network is a neural network model based on deep learning or other machine learning technologies, which is used to process and analyze data in the template holographic road traffic perception stream to identify and predict potential traffic accident knowledge points. Traffic accident knowledge points refer to various factors related to the occurrence of traffic accidents, such as driver behavior, vehicle status, road conditions, etc.

[0020] The high-likelihood traffic accident knowledge points are the knowledge points that are most likely to cause traffic accidents when the traffic accident knowledge point description network analyzes the data in the perception stream under a given template holographic road traffic perception stream, and are the high-likelihood traffic accident knowledge points.

[0021] That is, in this embodiment, the basic template holographic road traffic perception stream sequence contains extremely rich and detailed road traffic information. These basic template holographic road traffic perception streams not only cover common vehicle driving data (such as speed, location, driving direction), traffic light status and road congestion, but also include vehicle model, driver behavior characteristics (such as whether fatigue driving, whether using mobile phone), road environmental conditions (such as weather, light intensity, road surface conditions), distribution of surrounding buildings and facilities and many other complex information.

[0022] At the same time, there are multiple traffic accident knowledge point description networks with different structures and functions in the server, such as network A, network B and network C. These networks have been designed and trained to process such complex perception stream data and analyze and predict various potential traffic accident knowledge points.

[0023] When executing step S110, the server first selects a template holographic road traffic perception stream from the basic template holographic road traffic perception stream sequence, such as perception stream X. Perception stream X may include a large truck driving at night on a rainy day, its speed is slightly higher than the speed limit, the driver has signs of fatigue due to frequent blinking, the road is slippery and partially flooded, and the surrounding buildings are densely packed, resulting in obstructed vision.

[0024] Then, the perception stream X is input into network A, network B and network C respectively.

[0025] Taking network A as an example, it has a multi-layer deep neural network structure that can extract and analyze various complex features in the perception stream. For vehicle information in perception stream X, the network will analyze the impact of vehicle model and load on braking distance; for driver behavior, it will use a special module to evaluate the effect of fatigue on reaction time; for road environment, it will consider the changes in friction caused by weather and road conditions.

[0026] Network A will calculate a positive prediction confidence for each traffic accident knowledge point. Assume that there are a series of complex traffic accident knowledge points, such as "vehicle loss of control due to driver fatigue", "brake failure caused by slippery road surface", "collision risk caused by obstructed vision", "degradation of braking performance caused by overloading of large trucks", etc. Network A may calculate that the positive prediction confidence for the knowledge point "vehicle loss of control due to driver fatigue" is 0.75, the positive prediction confidence for "brake failure caused by slippery road surface" is 0.8, the positive prediction confidence for "collision risk caused by obstructed vision" is 0.6, and the positive prediction confidence for "degradation of braking performance caused by overloading of large trucks" is 0.4.

[0027] After calculating the positive prediction confidence of all traffic accident knowledge points, network A will determine the traffic accident knowledge point with the largest positive prediction confidence, such as "brake failure caused by slippery road surface", and use it as the high-likelihood traffic accident knowledge point corresponding to network A.

[0028] Similarly, network B and network C will also perform similar in-depth processing on the perception stream X and determine their corresponding high-likelihood traffic accident knowledge points respectively.

[0029] Step S120, based on the first prior traffic accident knowledge point of the template holographic road traffic perception flow and the comparative analysis of the high likelihood traffic accident knowledge point corresponding to each traffic accident knowledge point description network, extract the target template holographic road traffic perception flow from the basic template holographic road traffic perception flow sequence, and the extracted target template holographic road traffic perception flow constitutes the target template holographic road traffic perception flow sequence.

[0030] In step S110, the high likelihood traffic accident knowledge points of each traffic accident knowledge point description network for the perception stream X have been determined. At the same time, the server already knows the first a priori traffic accident knowledge point of the perception stream X. This a priori traffic accident knowledge point may be obtained by professional traffic analysts based on detailed on-site investigation and comprehensive judgment of multi-source data, and has high accuracy and authority.

[0031] Assume that the first prior traffic accident knowledge point of perception flow X is "collision risk caused by obstructed vision", and the high-likelihood traffic accident knowledge point corresponding to network A is "brake failure caused by slippery road surface", network B corresponds to "collision risk caused by obstructed vision", and network C corresponds to "degraded braking performance caused by overloading of large trucks".

[0032] Through comparative analysis, it can be found that the high-likelihood traffic accident knowledge points corresponding to network B match the first prior traffic accident knowledge points of perception stream X. Therefore, perception stream X is marked as the target template holographic road traffic perception stream.

[0033] The server continues to select the next perception stream from the basic template holographic road traffic perception stream sequence, such as perception stream Y. Perception stream Y describes a small car speeding on a curve, the driver is answering a phone call, there is a construction area beside the road and the warning signs are not obvious.

[0034] For the perception flow Y, the high-likelihood traffic accident knowledge points calculated by each network are: Network A is "lane deviation caused by driver distraction", Network B is "rollover danger caused by speeding on curves", and Network C is "failure to avoid due to insufficient warning in the construction area".

[0035] It is known that the first priori traffic accident knowledge point of perception flow Y is "the rollover risk caused by speeding on a curve". After comparison, it is found that the prediction of network B is consistent with it, so perception flow Y is also marked as the target template holographic road traffic perception flow.

[0036] The server continuously repeats the above process to extract one target template holographic road traffic perception flow after another from the basic template holographic road traffic perception flow sequence. Finally, these target template holographic road traffic perception flows constitute the target template holographic road traffic perception flow sequence.

[0037] Step S130, load the target template holographic road traffic perception flow into each of the traffic accident knowledge point description networks, generate a first positive prediction confidence corresponding to each traffic accident knowledge point, and calculate the first network training cost corresponding to each of the traffic accident knowledge point description networks based on the first prior traffic accident knowledge point of each of the traffic accident knowledge points and the first positive prediction confidence.

[0038] The server has obtained the target template holographic road traffic perception stream sequence composed of perception stream X, perception stream Y, etc.

[0039] First, the perception stream X is loaded into network A again. Network A regenerates the first positive prediction confidence for each traffic accident knowledge point, assuming that the confidence for "emergency braking error under obstructed vision" is 0.6, and the confidence for "rear-end collision hazard caused by low visibility at night" is 0.4, etc.

[0040] It is known that the first priori traffic accident knowledge point of perception flow X is “collision risk caused by obstructed vision”. By comparing the first priori traffic accident knowledge point and the generated new first positive prediction confidence, the prediction error of network A for perception flow X can be calculated.

[0041] Assume that the cross entropy loss function is used as an indicator for calculating the network training cost. If there is a large deviation between the confidence distribution predicted by network A and the actual first prior traffic accident knowledge point, the calculated cross entropy loss will be large, resulting in a high network training cost.

[0042] The above processing is also performed on the perception stream Y and other perception streams in the sequence, and the prediction error of each perception stream in the network A is calculated. These errors are combined to calculate the first network training cost corresponding to the network A.

[0043] For network B and network C, in the same way, for each perception stream in the target template holographic road traffic perception stream sequence, their respective first positive prediction confidences are calculated and compared with the corresponding first prior traffic accident knowledge points, and then the first network training cost of each network is calculated using an appropriate loss function.

[0044] Step S140: For each of the traffic accident knowledge point description networks, the traffic accident knowledge point description network is trained according to the first network training cost corresponding to the traffic accident knowledge point description network.

[0045] Taking network A as an example, the server uses optimization algorithms such as gradient descent to adjust many parameters of network A, including connection weights and bias values ​​between neurons in each layer, based on the calculated training cost of the first network.

[0046] The purpose of the adjustment is to enable network A to more accurately predict high-likelihood traffic accident knowledge points that match the first prior traffic accident knowledge points when processing the new target template holographic road traffic perception flow, thereby reducing the training cost of the first network.

[0047] For example, if network A makes inaccurate predictions about traffic accident knowledge points related to “obstructed vision,” resulting in high training costs, then during the training process, the parameters related to processing the obstructed vision feature will be adjusted more significantly to enhance the network’s sensitivity and predictive ability for such situations.

[0048] For network B and network C, a similar gradient-based training process is also performed to optimize the network parameters and improve performance according to their respective first network training costs.

[0049] Step S150, overlaying the basic template holographic road traffic perception flow sequence with the target template holographic road traffic perception flow sequence, and returning to the step of loading the template holographic road traffic perception flow in the basic template holographic road traffic perception flow sequence into the multiple traffic accident knowledge point description networks, until the training cost of the first network converges, and determining the converged traffic accident knowledge point description network from the multiple traffic accident knowledge point description networks.

[0050] In this embodiment, the server uses the previously extracted target template holographic road traffic perception flow sequence as new more targeted and representative training data to completely cover the original basic template holographic road traffic perception flow sequence.

[0051] Then, return to step S110, load the new template holographic road traffic perception flow (ie, the target template holographic road traffic perception flow) into multiple traffic accident knowledge point description networks, and execute the subsequent steps S120, S130 and S140 again.

[0052] During each cycle, the server continues to closely monitor the changes in the training cost of the first network. When the training cost of the first network gradually decreases with the increase of training rounds and does not fluctuate much within a certain range, it is considered to have converged.

[0053] Finally, from multiple traffic accident knowledge point description networks (such as network A, network B and network C), by comprehensively comparing their first network training cost in the converged state, the ability to handle complex and rare traffic accident scenarios, generalization performance and other indicators, the network with the best performance is selected as the final converged traffic accident knowledge point description network.

[0054] Assume that after a comprehensive evaluation, it is found that network B not only has the lowest training cost when processing various complex traffic accident knowledge points, but also can give accurate predictions when faced with unseen test data. In this case, network B is determined to be the converged traffic accident knowledge point description network, which is used for the subsequent accurate and reliable prediction and analysis of traffic accident knowledge points for actual, more diverse and complex road traffic perception flows.

[0055] In this way, through continuous iterative optimization and fine-tuning, the server can screen and cultivate a highly accurate and powerful network model from multiple initial traffic accident knowledge point description networks, providing solid technical support and intelligent decision-making basis for ensuring the safe and efficient operation of road traffic.

[0056] Based on the above steps, the embodiment of the present application realizes efficient and accurate detection of road traffic accidents by constructing and training multiple traffic accident knowledge point description networks. First, holographic perception technology is used to comprehensively capture multi-dimensional information in road traffic scenes, effectively improving the sensitivity and accuracy of accident detection. By repeatedly loading the target template holographic road traffic perception flow to the traffic accident knowledge point description network, and generating high-likelihood traffic accident knowledge points based on comparative analysis, it not only significantly enhances the network's recognition ability for complex traffic scenes, but also ensures the reliability of the prediction results. In addition, by calculating and training the network in a targeted manner based on the first network training cost, the network performance is effectively optimized, so that the converged traffic accident knowledge point description network can more accurately predict potential accident risks when facing actual traffic conditions, providing decision-making support for traffic management departments, thereby significantly improving the real-time and accuracy of road traffic accident detection, helping to reduce the occurrence of traffic accidents and ensure road traffic safety.

[0057] In a possible implementation, step S150 includes:

[0058] Step S151, obtaining a third verification holographic road traffic perception flow sequence, wherein the third verification holographic road traffic perception flow comprises a plurality of third verification holographic road traffic perception flows, and the third verification holographic road traffic perception flow has a second key priori traffic accident knowledge point.

[0059] Step S152, loading the third verified holographic road traffic perception stream into each of the traffic accident knowledge point description networks, and generating high-likelihood traffic accident knowledge point data for each of the traffic accident knowledge point description networks.

[0060] Step S153, determining a second performance indicator of each of the traffic accident knowledge point description networks based on a comparison between the second key a priori traffic accident knowledge point and the high likelihood traffic accident knowledge point data of each of the traffic accident knowledge point description networks.

[0061] Step S154, determining the converged traffic accident knowledge point description network according to the second performance indicator of each of the traffic accident knowledge point description networks.

[0062] In this embodiment, first, the server obtains the third verification holographic road traffic perception flow sequence. Assume that this third verification holographic road traffic perception flow sequence contains 1,000 third verification holographic road traffic perception flows. These perception flows contain various complex and diverse road traffic conditions. For example, some perception flows describe the driving conditions of vehicles on a highway in foggy weather, in which a car has a slight collision with the vehicle in front due to low visibility; some perception flows depict a busy intersection in the city during peak hours, where a motorcycle runs a red light and almost collides with a vehicle traveling laterally; and some perception flows record landslides on mountain roads due to continuous rainfall, part of the road is blocked, and a truck skids while avoiding falling rocks. In addition, each third verification holographic road traffic perception flow has a second key prior traffic accident knowledge point determined by a professional traffic analysis team, which accurately describes the potential accident risks and types in each traffic condition.

[0063] Next, the server loads these third-verified holographic road traffic perception streams into the three traffic accident knowledge point description networks, namely, network A, network B and network C.

[0064] Taking network A as an example, when loading a third-verified holographic road traffic perception stream describing "At night, a bus almost lost control due to excessive speed when making a sharp turn", network A generates prediction data about possible traffic accident knowledge points through internal calculation and analysis. Assume that the high-likelihood traffic accident knowledge point predicted by network A is "Risk of vehicle losing control due to excessive speed on a curve at night". For each perception stream in the entire third-verified holographic road traffic perception stream sequence, network A performs similar processing to generate a series of high-likelihood traffic accident knowledge point data.

[0065] Similarly, network B and network C also process the 1,000 third-verified holographic road traffic perception streams and each generates corresponding high-likelihood traffic accident knowledge point data.

[0066] Then, the server compares the high likelihood traffic accident knowledge point data generated by each traffic accident knowledge point description network based on the second key prior traffic accident knowledge point.

[0067] For example, for the perception stream mentioned above, "At night, a bus nearly lost control due to excessive speed when making a sharp turn", its second key prior traffic accident knowledge point is "the potential danger of loss of control caused by excessive speed on a curve at night". If the high-likelihood traffic accident knowledge point generated by network A completely matches it, then it is considered that the prediction of network A on this perception stream is accurate. By making such a comparison for each perception stream in the entire third verification holographic road traffic perception stream sequence, the server can calculate the proportion of each network in accurately predicting prior knowledge points, the degree of error and other indicators.

[0068] Assume that through comparison and calculation, it is found that the prediction of network A on 800 perception streams matches the second key prior traffic accident knowledge point with an accuracy of 80%; network B matches on 850 perception streams with an accuracy of 85%; network C matches on 750 perception streams with an accuracy of 75%. In addition, other indicators are also considered, such as the predicted recall rate (i.e., the proportion of actual prior knowledge points that are successfully predicted), F1 value (an indicator that comprehensively considers accuracy and recall), etc. Combining these indicators, the server determines a second performance indicator for each network.

[0069] Finally, the converged traffic accident knowledge point description network is determined based on the second performance index of each traffic accident knowledge point description network. In the above example, since network B has the best comprehensive performance in accuracy, recall rate, and F1 value, its second performance index is the highest. Therefore, the server determines network B as the converged traffic accident knowledge point description network.

[0070] In actual applications, the server will also consider more factors to evaluate the performance of the network. For example, the network's prediction ability for certain rare but serious traffic accident scenarios; the network's stability and adaptability when processing perception flows in different regions and different road types (such as urban roads, highways, and rural roads); and the network's computing efficiency and resource consumption. By comprehensively weighing these factors, the server can accurately select the most suitable convergence network for actual application needs from multiple traffic accident knowledge point description networks, providing reliable support for road traffic safety analysis and prediction.

[0071] In this way, the server goes through a series of complex and rigorous processing processes and finally determines the traffic accident knowledge point description network with the best performance, so as to improve the ability to accurately identify and predict potential accidents in road traffic and ensure the safety of public travel.

[0072] In a possible implementation, step S110 includes:

[0073] Step S111, for each of the traffic accident knowledge point description networks, the template holographic road traffic perception flow in the basic template holographic road traffic perception flow sequence is loaded into the traffic accident knowledge point description network to generate the first positive prediction confidence corresponding to each of the traffic accident knowledge points.

[0074] Step S112, determining the traffic accident knowledge point with the maximum first positive prediction confidence as the high-likelihood traffic accident knowledge point corresponding to the traffic accident knowledge point description network.

[0075] In this embodiment, it is assumed that there is a template holographic road traffic perception flow Z in the basic template holographic road traffic perception flow sequence, which describes the following scenario: On a snowy evening, a small passenger car is driving on a mountain road at a high speed. The driver frequently checks the navigation. The road is icy and the visibility is poor.

[0076] The server first loads the perception stream Z into the traffic accident knowledge point description network A. After its complex calculation and analysis process, the network A generates the corresponding first positive prediction confidence for a series of traffic accident knowledge points.

[0077] For example, for the knowledge point of "collision risk caused by poor visibility in snowy weather", the confidence of the first positive prediction generated is 0.7; for "the risk of loss of control caused by icy roads", the confidence is 0.85; for "operational errors caused by the driver's distraction in checking the navigation", the confidence is 0.6; for "accidents caused by complex road conditions in mountainous areas", the confidence is 0.5, etc.

[0078] Then, the server also loads the perception stream Z into the traffic accident knowledge point description network B. The network B also generates the corresponding first positive prediction confidence for each traffic accident knowledge point.

[0079] Assume that for the "collision risk caused by poor visibility in snowy weather", the confidence level is 0.65; for the "risk of loss of control caused by icy roads", the confidence level is 0.7; for the "operational errors caused by the driver's distraction in checking the navigation", the confidence level is 0.75; for the "accidents caused by complex road conditions in mountainous areas", the confidence level is 0.4, etc.

[0080] Next, the server needs to determine the traffic accident knowledge point with the maximum first positive prediction confidence in each network.

[0081] For network A, by comparing the confidence values, it is found that the confidence of "risk of loss of control caused by icy roads" is 0.85, which is the maximum value. Therefore, the server determines "risk of loss of control caused by icy roads" as a high-likelihood traffic accident knowledge point corresponding to network A.

[0082] For network B, after comparison, the confidence level of "operational error caused by the driver's distraction in checking the navigation" is 0.75, which is the largest, so the server determines it as a high-likelihood traffic accident knowledge point corresponding to network B.

[0083] In this way, the server continuously processes other perception streams in the basic template holographic road traffic perception stream sequence, and determines the corresponding high-likelihood traffic accident knowledge point for each traffic accident knowledge point description network.

[0084] This can provide key information for subsequent analysis and processing, so as to more accurately evaluate and optimize the performance of the traffic accident knowledge point description network and improve the ability to predict and judge potential road traffic accident risks.

[0085] In a possible implementation, step S112 includes:

[0086] Step S1121, obtaining threshold setting values ​​of the plurality of traffic accident knowledge points.

[0087] Step S1122: taking the traffic accident knowledge point whose first positive prediction confidence is greater than the threshold setting value as a reference traffic accident knowledge point.

[0088] Step S1123, determining the reference traffic accident knowledge point with the maximum first positive prediction confidence as the high-likelihood traffic accident knowledge point corresponding to the traffic accident knowledge point description network.

[0089] In this embodiment, when processing the output of the traffic accident knowledge point description network, the threshold setting values ​​of multiple traffic accident knowledge points are first obtained. Assume that the following threshold setting values ​​are set:

[0090] For the knowledge point “Risk of collision caused by vehicle speeding”, the threshold setting value is 0.7;

[0091] For the knowledge point of “the risk of loss of control caused by driver fatigue driving”, the threshold setting value is 0.6;

[0092] For the knowledge point "Hazards of obstructed vision under the influence of severe weather", the threshold setting value is 0.5;

[0093] For the knowledge point of “traffic accidents caused by damage to road facilities”, the threshold setting value is 0.4.

[0094] Then, the server obtains the first positive prediction confidence generated by the traffic accident knowledge point description network for a specific basic template holographic road traffic perception flow. Assume that for the above perception flow, the first positive prediction confidence of each knowledge point is:

[0095] The confidence level for “risk of collision caused by vehicle speeding” is 0.8;

[0096] The confidence level of “the risk of loss of control caused by driver fatigue driving” is 0.75;

[0097] The confidence level of “the hidden danger of obstructed vision under the influence of bad weather” is 0.45;

[0098] The confidence level for “traffic accidents caused by damaged road facilities” is 0.3.

[0099] Next, the server compares the first positive prediction confidence with the threshold setting value. Since the confidence of "collision risk caused by vehicle speeding" is 0.8, which is greater than its threshold setting value of 0.7, and the confidence of "loss of control caused by driver fatigue driving" is 0.75, which is greater than its threshold setting value of 0.6, these two traffic accident knowledge points are used as reference traffic accident knowledge points.

[0100] Finally, the first positive prediction confidences of the two reference traffic accident knowledge points are compared. Because 0.8 is greater than 0.75, the server determines that "collision risk caused by vehicle speeding" is a high-likelihood traffic accident knowledge point corresponding to the traffic accident knowledge point description network.

[0101] The prediction results of each basic template holographic road traffic perception flow in the traffic accident knowledge point description network are processed in this way, so as to accurately determine the high-likelihood traffic accident knowledge points and provide strong support for subsequent road traffic analysis and decision-making.

[0102] In a possible implementation, the traffic accident knowledge point description network includes a traffic accident semantic enhancement unit and a fully connected mapping unit.

[0103] After step S150, the method further includes:

[0104] Step A110, distinguish and identify the first prior traffic accident knowledge point of each traffic accident knowledge point of the target template holographic road traffic perception flow, and obtain first attribute knowledge point data, second attribute knowledge point data, and empty knowledge point data, wherein the network training distinction difficulty between the first attribute knowledge point data and the empty knowledge point data is less than the network training distinction difficulty between the second attribute knowledge point data and the empty knowledge point data.

[0105] Step A120, in the target template holographic road traffic perception flow sequence, obtaining a first template holographic road traffic perception flow annotated with the second attribute knowledge point data and a second template holographic road traffic perception flow annotated with the empty knowledge point data.

[0106] Step A130, obtaining a first difficult-to-distinguish template perception stream from the first template holographic road traffic perception stream, obtaining a second difficult-to-distinguish template perception stream from the second template holographic road traffic perception stream, and outputting other template holographic road traffic perception streams except the first difficult-to-distinguish template perception stream and the second difficult-to-distinguish template perception stream in the target template holographic road traffic perception stream sequence as non-difficult-to-distinguish template perception streams.

[0107] Step A140, loading the first difficult-to-distinguish template perception flow into the converged traffic accident knowledge point description network, generating a first prediction confidence corresponding to the first prior traffic accident knowledge point of each traffic accident knowledge point predicted by the converged traffic accident knowledge point description network.

[0108] Step A150, based on the first prior traffic accident knowledge point and the first prediction confidence corresponding to the first prior traffic accident knowledge point, determine the first derivative training supervision information of the traffic accident knowledge point, and based on the first derivative training supervision information, calculate the second network training sub-cost of the first difficult-to-distinguish template perception flow.

[0109] Step A160, for the second difficult-to-distinguish template perception flow, determine the second derivative training supervision information of each of the traffic accident knowledge points, and calculate the third network training sub-cost of the second difficult-to-distinguish template perception flow based on the second derivative training supervision information.

[0110] Step A170, calculating a fourth network training sub-cost of the non-difficult-to-distinguish template-perceived flow, and calculating a second network training cost based on the second network training sub-cost, the third network training sub-cost, and the fourth network training sub-cost.

[0111] Step A180: training the traffic accident semantic enhancement unit and the fully connected mapping unit in the converged traffic accident knowledge point description network according to the second network training cost.

[0112] In this embodiment, in the road traffic management system, the server has determined a converged traffic accident knowledge point description network from multiple traffic accident knowledge point description networks after a series of processing. This traffic accident knowledge point description network includes a traffic accident semantic enhancement unit and a fully connected mapping unit, aiming to more accurately analyze and predict accident risks in road traffic.

[0113] The server first distinguishes and identifies the first prior traffic accident knowledge point of each traffic accident knowledge point in the target template holographic road traffic perception stream. Assume that the target template holographic road traffic perception stream contains a series of rich descriptions of traffic accident scenes. For example, for a perception stream, it describes a motorcycle that runs a red light at a busy intersection, which is marked as "risk of collision caused by serious violations", which is the first prior traffic accident knowledge point. Through complex algorithms and feature analysis, the server identifies such knowledge points with clear characteristics and higher risks as second attribute knowledge point data. For another perception stream, it only describes the normal driving of the vehicle, the good road conditions, and no obvious signs of accident risk, which is marked as "normal driving without accidents", which is identified as empty knowledge point data. At the same time, there are some knowledge points in the intermediate state, whose characteristics are not obvious, and the network training is relatively easy to distinguish, such as "potential risk of minor vehicle scratches", which are identified as first attribute knowledge point data.

[0114] In the target template holographic road traffic perception flow sequence, the server obtains the first template holographic road traffic perception flow annotated with the second attribute knowledge point data. For example, there is a perception flow that describes a serious accident of multiple vehicles rear-ending on a highway, with clear details and clearly marked as "risk of major multi-vehicle rear-end collision", which is a typical first template holographic road traffic perception flow. At the same time, the server also obtains the second template holographic road traffic perception flow annotated with empty knowledge point data. For example, in a perception flow, vehicles are driving smoothly on an unobstructed road without any abnormalities, which is marked as "unobstructed driving without accidents".

[0115] Next, the server obtains the first difficult-to-distinguish template perception flow from the first template holographic road traffic perception flow. Suppose a first template holographic road traffic perception flow describes a complex intersection traffic accident involving multiple vehicles and pedestrians, and is labeled as "complex intersection multi-directional collision risk." When it is loaded into the converged traffic accident knowledge point description network, there is a large deviation between the network's prediction results and the actual annotations. For example, the network predicts "risk of vehicle collision at ordinary intersections" and the confidence of the prediction is low. This perception flow, which is significantly different from the actual annotation and has an unsatisfactory confidence, is determined by the server as the first difficult-to-distinguish template perception flow.

[0116] Similarly, the second difficult-to-distinguish template perception flow is obtained from the second template holographic road traffic perception flow. For example, a perception flow marked as empty knowledge point data, that is, vehicles on the road are driving normally without any signs of accidents, but the network gives a high confidence in the accident risk prediction, such as predicting that there is a "minor collision risk", such a perception flow is determined to be the second difficult-to-distinguish template perception flow.

[0117] For those perception streams that are neither the first-difficult-to-distinguish template-perceived stream nor the second-difficult-to-distinguish template-perceived stream, the server outputs them as non-difficult-to-distinguish template-perceived streams.

[0118] Then, the server reloads the first difficult-to-distinguish template perception flow into the converged traffic accident knowledge point description network. For example, for the complex intersection traffic accident perception flow that is determined to be the first difficult to distinguish, the network regenerates the first prediction confidence corresponding to the first prior traffic accident knowledge point of each traffic accident knowledge point. Assuming that the first prior traffic accident knowledge point actually marked is "multi-directional collision risk at complex intersections", the corresponding ideal confidence should be very high, but the first prediction confidence generated by the network is relatively low.

[0119] Based on the actual first prior traffic accident knowledge point and the generated first prediction confidence, the server determines the first derivative training supervision information of the traffic accident knowledge point. For example, it is found that the network's understanding and prediction of complex intersection conditions are insufficient, and it is necessary to strengthen the learning of intersection traffic rules and vehicle behavior interactions. Based on this first derivative training supervision information, the server calculates the second network training sub-cost of the first difficult-to-distinguish template perception flow. This cost reflects the errors and deficiencies of the network when processing such complex and difficult-to-distinguish perception flows.

[0120] For the second most difficult to distinguish template perception flow, the server also determines the second derivative training supervision information for each traffic accident knowledge point. For example, for an empty knowledge point data perception flow that is incorrectly predicted to have an accident risk, analyze why the network makes such a misjudgment, whether it is a deviation in the understanding of normal driving characteristics or other reasons. Based on this supervision information, calculate the third network training sub-cost of the second most difficult to distinguish template perception flow.

[0121] For the difficult-to-distinguish template perception flow, the server calculates the fourth network training sub-cost through the corresponding algorithm and model evaluation indicators. This cost usually reflects the performance of the network when processing relatively simple and clear perception flows.

[0122] Finally, the server calculates the second network training cost based on the second network training sub-cost, the third network training sub-cost, and the fourth network training sub-cost. This total cost comprehensively considers the performance of the network when processing various perception streams of different difficulties and types.

[0123] Using the calculated training cost of the second network, the server further trains and optimizes the traffic accident semantic enhancement unit and the fully connected mapping unit in the converged traffic accident knowledge point description network. For example, the parameters in the traffic accident semantic enhancement unit are adjusted to enable it to better extract and understand the semantic information in the perception stream, especially for complex and ambiguous accident scenes. At the same time, the parameters of the fully connected mapping unit are optimized to improve the prediction accuracy and generalization ability of the network.

[0124] The above steps are repeated throughout the process, processing a large amount of perception stream data, and continuously monitoring and evaluating the performance of the network. Through continuous training and optimization, the traffic accident knowledge point description network can more accurately identify and predict the accident risks in various road traffic situations, providing stronger support for traffic management and safety assurance.

[0125] In a possible implementation manner, the plurality of the first a priori traffic accident knowledge points include the empty knowledge point data and various a priori traffic accident knowledge points of various accident severity levels.

[0126] Step A110 includes:

[0127] Step A111, extracting the empty knowledge point data from the plurality of the first priori traffic accident knowledge points.

[0128] Step A112, for each prior traffic accident knowledge point of each accident severity level, the second verification holographic road traffic perception flow marked with the prior traffic accident knowledge point in the second verification holographic road traffic perception flow sequence is loaded into the traffic accident knowledge point description network to generate first traffic accident knowledge point data.

[0129] Step A113, based on the comparative analysis of the first traffic accident knowledge point data and the prior traffic accident knowledge point, the prior traffic accident knowledge point is distinguished and identified to obtain the first attribute knowledge point data or the second attribute knowledge point data.

[0130] Wherein, step A113 includes:

[0131] Step A1131, outputting the number of second verification samples in which the prior traffic accident knowledge point is marked in the second verification holographic road traffic perception flow sequence as a first statistical number.

[0132] Step A1132, outputting as a second statistical number the number of second verification samples in the second verification holographic road traffic perception stream in which the prior traffic accident knowledge point is marked in the second verification holographic road traffic perception stream sequence, in which the first traffic accident knowledge point data does not match the prior traffic accident knowledge point.

[0133] Step A1133: determining the network training differentiation difficulty based on the first statistical quantity and the second statistical quantity.

[0134] Step A1134: If the network training distinction difficulty is less than the first set threshold value, the prior traffic accident knowledge point is distinguished and identified to obtain the first attribute knowledge point data. If the network training distinction difficulty is not less than the first set threshold value, the prior traffic accident knowledge point is distinguished and identified to obtain the second attribute knowledge point data.

[0135] In this embodiment, first, the server faces a plurality of first priori traffic accident knowledge points. These first priori traffic accident knowledge points include empty knowledge point data, such as a description of a road condition without any signs of an accident, and priori traffic accident knowledge points of various accident severity levels, such as "minor accident", "moderate accident", "serious accident" and other accident categories of different degrees.

[0136] The server first extracts empty knowledge point data from these numerous first prior traffic accident knowledge points. For example, in a set of knowledge points, the server identifies those descriptions that clearly indicate that everything is normal on the road and no traffic accidents have occurred, and classifies them as empty knowledge point data.

[0137] Next, the server performs in-depth processing for each prior traffic accident knowledge point of each accident severity level. Assume that there are three prior traffic accident knowledge points of different severity levels: "minor scratch accident", "moderate collision accident" and "serious rear-end collision accident".

[0138] For the prior knowledge point of "minor scratch accident", the server loads all the perception streams marked as "minor scratch accident" in the second verification holographic road traffic perception stream sequence into the traffic accident knowledge point description network. After the network processes these perception streams, it generates the corresponding first traffic accident knowledge point data.

[0139] The server starts to count the relevant data. It records the number of second verification samples marked as "minor scratch accident" in the second verification holographic road traffic perception stream sequence as the first statistical number. Assume that this number is 200.

[0140] At the same time, the server finds out the number of samples in the second verification holographic road traffic perception stream of 200 labeled as "minor scratch accident" whose first traffic accident knowledge point data generated by the network does not match the prior traffic accident knowledge point of "minor scratch accident", that is, the second statistical number. Assume that this number is 30.

[0141] Then, the server determines the difficulty of network training based on the first statistical number (200) and the second statistical number (30). The server calculates the ratio of the number of mismatched samples to the total number of samples, that is, 30÷200=15%.

[0142] Assume that the first threshold value set in advance is 20%. Since the calculated network training distinction difficulty (15%) is less than the first threshold value, the server distinguishes and identifies the prior traffic accident knowledge point "minor scratch accident" as the first attribute knowledge point data. This means that it is relatively easy for the network to distinguish and predict such accidents.

[0143] Let’s look at the prior knowledge point of “moderate collision accident”. Assuming that the number of second verification samples is 150, of which the number of unmatched second verification samples is 40, the calculated mismatch ratio is 40÷150≈26.7%. If the first set threshold value is still 20%, since 26.7% is greater than 20%, the server will distinguish and identify the prior traffic accident knowledge point of “moderate collision accident” as the second attribute knowledge point data. This shows that the network has difficulty distinguishing when dealing with knowledge points of this type of accident, and requires more training and optimization.

[0144] For "serious rear-end collision", the above method is also used for processing and judgment. Assuming that the number of second verification samples is 180, the number of mismatches is 50, and the mismatch ratio is about 27.8%, which is greater than the first set threshold value, the server identifies it as the second attribute knowledge point data.

[0145] In this way, the server can accurately distinguish and identify the prior traffic accident knowledge points of each accident severity level. In the process of processing a large amount of data, the network processing difficulty characteristics of various types of accident knowledge points are continuously accumulated and analyzed.

[0146] For example, the server found that it was generally difficult for network training to distinguish accident knowledge points in certain types of road environments, such as "moderate collision accidents" on mountain roads. Therefore, in subsequent training and optimization, the server will pay more attention to targeted adjustments for such situations.

[0147] For those accident knowledge points identified as first-attribute knowledge point data, the server may appropriately reduce resource allocation in subsequent training, but still maintain a certain degree of monitoring and consolidation. For accident knowledge points marked as second-attribute knowledge point data, the server will focus more computing resources and optimization strategies to strive to improve the network's prediction accuracy for these complex and difficult-to-distinguish accident knowledge points.

[0148] By continuously repeating the above process, the characteristics of different accident knowledge points and the network processing difficulty can be identified more and more accurately, thereby continuously optimizing the traffic accident knowledge point description network, enabling it to more accurately respond to the analysis and prediction needs of various road traffic conditions, and provide stronger support for ensuring the safety and smoothness of road traffic.

[0149] In a possible implementation, step A130 includes:

[0150] Step A131, loads the first template holographic road traffic perception flow into the converged traffic accident knowledge point description network, generates the high likelihood traffic accident knowledge point predicted by the converged traffic accident knowledge point description network, and the first prediction confidence corresponding to the first prior traffic accident knowledge point of the first template holographic road traffic perception flow predicted by the converged traffic accident knowledge point description network.

[0151] Step A132: If the high likelihood traffic accident knowledge point predicted by the converged traffic accident knowledge point description network does not match the first prior traffic accident knowledge point of the first template holographic road traffic perception flow, and it is determined that the first prediction confidence and the first threshold setting value corresponding to the first prior traffic accident knowledge point of the first template holographic road traffic perception flow meet the first judgment condition, then the first template holographic road traffic perception flow is output as the first difficult-to-distinguish template perception flow.

[0152] Step A133, loading the second template holographic road traffic perception flow into the converged traffic accident knowledge point description network, generating a second prediction confidence corresponding to the first prior traffic accident knowledge point of the second template holographic road traffic perception flow predicted by the converged traffic accident knowledge point description network.

[0153] Step A134: If it is determined that the second prediction confidence and the second threshold setting value corresponding to the first prior traffic accident knowledge point of the second template holographic road traffic perception flow meet the second discrimination condition, the second template holographic road traffic perception flow is output as the second difficult-to-distinguish template perception flow.

[0154] In this embodiment, during the processing of the road traffic perception stream, the server first obtains the first template holographic road traffic perception stream and the second template holographic road traffic perception stream.

[0155] For the first template holographic road traffic perception stream, it is assumed that it describes a scenario where a car runs a red light at a busy intersection, resulting in a near collision with a vehicle traveling sideways. The first prior traffic accident knowledge point of this perception stream is labeled as "serious intersection violation leading to collision risk."

[0156] The server loads this first template holographic road traffic perception stream into the already converged traffic accident knowledge point description network. After calculation and analysis, the network predicts the high likelihood traffic accident knowledge point is "general intersection traffic chaos", which does not match the actual first prior traffic accident knowledge point "serious intersection violations leading to collision risks".

[0157] At the same time, the network generates a first prediction confidence corresponding to the first priori traffic accident knowledge point of this first template holographic road traffic perception flow. Assume that the first threshold setting value is 0.7, and the first prediction confidence actually calculated is 0.5. Since the first prediction confidence 0.5 is lower than the first threshold setting value 0.7, it meets the first discrimination condition. Therefore, the server outputs this first template holographic road traffic perception flow as the first difficult-to-distinguish template perception flow.

[0158] Next, let’s look at the second template holographic road traffic perception flow. Assume that it describes a motorcycle driving normally on a smooth road without any signs of an accident. Its first prior traffic accident knowledge point is marked as “driving normally without an accident”.

[0159] The server loads this second template holographic road traffic perception flow into the converged traffic accident knowledge point description network. The result of the network prediction is that there is a "minor accident risk" and generates a corresponding second prediction confidence. Assume that the second threshold setting value is 0.3, and the calculated second prediction confidence is 0.4. Since the second prediction confidence 0.4 is greater than the second threshold setting value 0.3, it meets the second discrimination condition. Therefore, the server outputs this second template holographic road traffic perception flow as a second difficult-to-distinguish template perception flow.

[0160] In this way, the server processes and judges a large number of first template holographic road traffic perception flows and second template holographic road traffic perception flows. For example, another first template holographic road traffic perception flow describes a vehicle tire burst on a highway but no accident occurs. Its first priori traffic accident knowledge point is "high-speed vehicle tire burst does not cause an accident", and the high-likelihood traffic accident knowledge point predicted by the network is "high-speed vehicle failure", which does not match, and the first prediction confidence is lower than the threshold setting value, and is determined to be the first difficult-to-distinguish template perception flow.

[0161] For another example, a second template holographic road traffic perception flow describes vehicles driving at a constant speed on an empty road at night. The first prior traffic accident knowledge point is "normal driving at night without accidents", and the network prediction is "there is a low probability accident risk". The second prediction confidence is higher than the threshold setting value, and it is judged to be the second difficult to distinguish template perception flow.

[0162] By continuously processing and screening these perception streams, we can accurately obtain the first difficult-to-distinguish template perception stream and the second difficult-to-distinguish template perception stream, providing targeted data support for the subsequent further optimization and training of the traffic accident knowledge point description network, thereby continuously improving the network's accurate judgment and prediction capabilities for various complex road traffic conditions.

[0163] In a possible implementation, step A132 includes:

[0164] Obtain a first priori weight, determine a first product result of the first prior weight and the first threshold setting value, and if the first prediction confidence is less than the first product result, determine that the first prediction confidence and the first threshold setting value meet the first judgment condition.

[0165] Step A134 includes:

[0166] Obtain a second prior weight, determine a second product result of the second prior weight and the second threshold setting value, and if the second prediction confidence is less than the second product result, determine that the second prediction confidence and the second threshold setting value meet the second judgment condition.

[0167] In this embodiment, when processing the first template holographic road traffic perception flow, the server first obtains the first priori weight, assuming that the value of the first priori weight is 0.8, and the first threshold setting value is 0.6.

[0168] The server calculates a first product result of the first priori weight and the first threshold setting value, ie, 0.8×0.6=0.48.

[0169] Then, the server obtains the first prediction confidence corresponding to the first a priori traffic accident knowledge point of the first template holographic road traffic perception flow generated by the converged traffic accident knowledge point description network. Assume that the first prediction confidence is 0.4.

[0170] Since the first prediction confidence of 0.4 is less than the product of the first priori weight and the first threshold setting value of 0.48, the server determines that the first prediction confidence and the first threshold setting value meet the first discrimination condition.

[0171] For example, the first template holographic road traffic perception flow describes a vehicle driving on a sharp turn road, and its first prior traffic accident knowledge point is "sharp turn driving has a high rollover risk." The first prediction confidence generated by the network shows that the estimate of the rollover risk is low, which is 0.4, while the judgment standard obtained by the above calculation is 0.48. Because 0.4 is less than 0.48, it meets the first discrimination condition and the perception flow is judged as the first difficult-to-distinguish template perception flow.

[0172] When processing the second template holographic road traffic perception flow, the server also first obtains the second priori weight. Assume that the second priori weight is 0.7 and the second threshold setting value is 0.5.

[0173] The server calculates a second product result of the second priori weight and the second threshold setting value, ie, 0.7×0.5=0.35.

[0174] Next, the server obtains the second prediction confidence corresponding to the first a priori traffic accident knowledge point of the second template holographic road traffic perception flow generated by the network. Assume that the second prediction confidence is 0.3.

[0175] Because the second prediction confidence of 0.3 is less than the product of the second priori weight and the second threshold setting value of 0.35, the server determines that the second prediction confidence and the second threshold setting value meet the second discrimination condition.

[0176] For example, the second template holographic road traffic perception flow describes a vehicle driving normally on a highway. Its first prior traffic accident knowledge point is "there is basically no accident risk when driving on a highway", but the second prediction confidence generated by the network shows that there is a certain risk of misestimation, which is 0.3, and the calculated judgment standard is 0.35. Since 0.3 is less than 0.35, it meets the second judgment condition and the perception flow is judged as the second difficult-to-distinguish template perception flow.

[0177] Through such precise calculation and comparison, a large number of perception streams can be accurately identified and classified, providing strong support for subsequent network optimization and training.

[0178] In a possible implementation, in the application stage, the method further includes:

[0179] Step S160, obtaining the target holographic road traffic perception flow.

[0180] Step S170, loading the target holographic road traffic perception flow into the traffic accident knowledge point description network, obtaining second traffic accident knowledge point data for each traffic accident knowledge point, wherein the second traffic accident knowledge point data characterizes whether the traffic accident characteristics of the target holographic road traffic perception flow conform to the traffic accident knowledge point.

[0181] In a possible implementation, the traffic accident knowledge point description network includes a semantic encoding unit, a traffic accident semantic enhancement unit, and a fully connected mapping unit.

[0182] Step S170 includes:

[0183] Step S171, using a Gaussian filtering algorithm to remove noise in the target holographic road traffic perception flow to generate a noise-cleaned perception flow, and adjusting the contrast of the noise-cleaned perception flow based on a preset contrast adjustment strategy to highlight the key elements in the traffic scene corresponding to the noise-cleaned perception flow, thereby generating a preprocessed perception flow.

[0184] Step S172, using multiple convolutional layers in a convolutional neural network to capture different features in the preprocessed perception stream, and using an activation function to perform nonlinear transformation on the captured different features, using a maximum pooling operation to downsample the output of the last convolutional layer, using the output of the last convolutional layer as a preliminary feature representation, performing a global average pooling operation on the preliminary feature representation to obtain the average value of each feature channel, using an attention mechanism to calculate the weight of each feature channel based on the average value of each feature channel, and using a softmax function to normalize the weight of each feature channel to obtain the attention weight of each feature channel.

[0185] Step S173, fusing the preliminary feature representation with the attention weight of each corresponding feature channel to obtain a weighted feature representation, and fusing the weighted feature representation to obtain a traffic perception attention feature.

[0186] Step S174, loading the traffic perception attention feature into the semantic coding unit to generate a semantic coding result, using the semantic coding result from the semantic coding unit as the input of the traffic accident semantic enhancement unit, generating an initial individual population, wherein each individual in the initial individual population is a transformed representation of the input semantic coding result.

[0187] Step S175, calculating the feature correlation parameter between the transformed representation feature generated by each individual and the known semantic feature of the traffic accident and the feature uniqueness parameter of each individual, wherein the feature uniqueness is used to characterize the degree of difference from the transformed representation features generated by other individuals in the initial individual population.

[0188] Step S176, weightedly fuse the feature correlation parameter and the feature uniqueness parameter to obtain the fitness value of each individual, randomly select multiple individuals from the initial individual population to compete, sort the individuals in the initial individual population according to the fitness value of each individual, select the target individual with a higher ranking as the parent individual based on the set ratio, and for the selected parent individual, uniformly exchange the corresponding vector element position with a first set probability to generate a new child individual, and generate vector elements of the child individual by linearly combining the parent individual vector elements, and add random noise to the vector elements of the child individual according to the second set probability to achieve mutation, wherein the second set probability is dynamically adjusted according to the fitness value of the child individual, and the child individual with a low fitness value has a higher second set probability.

[0189] Step S177, decoding the offspring individuals generated by the genetic operation and converting them into processable feature representations.

[0190] Step S178, use principal component analysis to decompose the decoded processable feature representation, extract the main component features and independent component features, and screen and filter the decomposed main component features and independent component features according to the pre-set traffic accident semantic rules to remove some component features that are irrelevant or weakly related to the traffic accident semantics. The screened component features are reorganized according to the target strategy, and the key components in the screened component features are enhanced through nonlinear transformation to obtain the final traffic accident semantic enhancement features.

[0191] Step S179: Load the traffic accident semantic enhancement features into the fully connected mapping unit, and generate the second traffic accident knowledge point data for each traffic accident knowledge point.

[0192] In this embodiment, it is assumed that the acquired target holographic road traffic perception flow depicts the traffic conditions of a complex urban intersection in the evening. This intersection connects multiple main roads and secondary roads, with a large traffic volume, including various types of vehicles, such as cars, buses, motorcycles and bicycles. At the same time, pedestrians are also shuttling through the intersection, and traffic lights are switching in an orderly manner.

[0193] First, the server uses a Gaussian filter algorithm to remove noise from the target holographic road traffic perception stream. In this actual scenario, noise may appear as momentary abnormal fluctuations in the speed data of some vehicles, or short-term erroneous records of pedestrian positions. For example, the speed data of a car suddenly jumps from 30 km / h to 50 km / h at a certain moment, and then quickly returns to normal. This may be due to noise caused by a short-term failure of the sensor. Through the Gaussian filter algorithm, these abnormal data points are smoothed to generate a noise-cleaned perception stream. After processing, the speed change of the above car becomes more stable and reasonable, which is consistent with the speed change trend of surrounding vehicles.

[0194] Next, the contrast of the noise-cleaned perception stream is adjusted based on a preset contrast adjustment strategy to highlight the key elements in the traffic scene corresponding to the noise-cleaned perception stream. In this intersection scene, the key elements may include the color change of traffic lights, the stop position of buses, and the density of pedestrians on the pedestrian crossing. The preset contrast adjustment strategy may enhance the brightness and color contrast of traffic lights in the perception stream to make them more eye-catching; for buses, the opening and closing status of their doors and the distance between the body and surrounding vehicles may be highlighted; for pedestrian crossings, the contrast between pedestrians and vehicles may be increased to more clearly show the relative position and interactive relationship between the two.

[0195] After contrast adjustment, the generated pre-processed perception stream allows the server to more intuitively focus on these elements that have a significant impact on traffic conditions and potential accidents. For example, the red light of a traffic light becomes more vivid and prominent in the perception stream, allowing the server to more easily capture changes in the state of the light, thereby determining whether a vehicle is at risk of running a red light; the bus stop position and door status are clearer, which helps analyze the impact of buses on surrounding traffic flow when passengers get on and off; the contrast between pedestrians and vehicles on pedestrian crosswalks is enhanced, allowing the server to more accurately assess the possibility of conflict between pedestrians and vehicles.

[0196] Next, the server uses multiple convolutional layers in the convolutional neural network to capture different features in the preprocessed perception stream. The first convolutional layer may capture some basic image features, such as the outline of the vehicle, the lines of the road, etc.; the second convolutional layer may capture the relative position relationship between vehicles and simple movement trends, such as whether the distance between vehicles is shrinking or increasing; the third convolutional layer may capture more complex scene features, such as the overall situation when multiple vehicles approach the intersection at the same time.

[0197] To illustrate more specifically, assume that the first convolutional layer captures the general shape of a motorcycle and its position on the road; the second convolutional layer captures that the distance between the motorcycle and a car next to it is gradually shortening, and the speed difference between the two is large; the third convolutional layer captures that in this intersection area, multiple motorcycles and cars show a similar trend of approaching, and there may be potential traffic conflicts.

[0198] Then, an activation function (such as the ReLU function) is used to perform nonlinear transformations on the different features captured. In the above example, after the nonlinear transformation, the originally small feature values ​​are amplified, highlighting the importance of the relative position and speed changes between the vehicles. For example, for the feature that the distance between the motorcycle and the car is shortened and the speed difference is large, after the nonlinear transformation, this feature will receive more attention in subsequent processing.

[0199] After that, the output of the last convolutional layer is downsampled using the max pooling operation to reduce the feature dimension while retaining the main features. Assuming that the output of the last convolutional layer contains a lot of detailed information about the vehicle position and speed, the max pooling operation only retains the most significant features in each local area. For example, in a local area, there are multiple vehicle speed features, and the max pooling operation will select the feature with the fastest speed as the representative feature of the area.

[0200] After taking the output of the last convolutional layer as the preliminary feature representation, perform a global average pooling operation on it to obtain the average value of each feature channel. Assume that the preliminary feature representation includes five feature channels: vehicle speed, vehicle spacing, number of pedestrians, signal light status, and road congestion. Through the global average pooling operation, the average value of each channel is obtained. For example, the average value of the vehicle speed channel is 40 kilometers per hour, the average value of the vehicle spacing channel is 5 meters, the average value of the pedestrian number channel is 10 people, the average value of the signal light status channel is 0.5 (indicating the proportion of green light time), and the average value of the road congestion channel is 0.7 (indicating moderate congestion).

[0201] The attention mechanism is used to calculate the weight of each feature channel based on the average value of each feature channel. In this specific scenario, if the average value of vehicle speed is high, it may mean that the traffic flow is fast, and the corresponding weight will be higher; if the average value of road congestion is high, it means that the traffic conditions are poor, and the corresponding weight will increase accordingly. If the average value of the number of pedestrians is low, it may have less impact on traffic, and the weight will be relatively low.

[0202] The softmax function is used to normalize the weight of each feature channel to obtain the attention weight of each feature channel. Assume that after calculation and normalization, the attention weight of the vehicle speed channel is 0.3, the weight of the vehicle spacing channel is 0.2, the weight of the pedestrian number channel is 0.1, the weight of the traffic light status channel is 0.2, and the weight of the road congestion channel is 0.2.

[0203] The preliminary feature representation is fused with the attention weight of each corresponding feature channel to obtain a weighted feature representation. For example, the preliminary feature representation value of vehicle speed is 50 km / h, which is multiplied by its attention weight of 0.3 to obtain a weighted value of 15 km / h; the preliminary feature representation value of vehicle spacing is 4 meters, which is multiplied by its attention weight of 0.2 to obtain a weighted value of 0.8 meters, and so on. These weighted feature representations are fused to obtain traffic perception attention features, which further highlight the features that have an important impact on traffic conditions.

[0204] Then, the traffic-aware attention features are loaded into the semantic encoding unit to generate semantic encoding results. The semantic encoding unit may convert the values ​​and patterns in the traffic-aware attention features into more semantically meaningful codes, such as encoding higher vehicle speeds and shorter vehicle spacing as "dense traffic flow and faster driving speed" and encoding higher road congestion levels as "moderate road congestion".

[0205] The semantic encoding results from the semantic encoding unit are used as the input of the traffic accident semantic enhancement unit to generate the initial individual population. Assume that there are 200 individuals in the initial individual population, and each individual is a unique transformation representation of the input semantic encoding results. For example, individual 1 may strengthen the semantics of "dense traffic flow and fast driving speed", individual 2 may pay more attention to the semantics of "moderate road congestion", and individual 3 may try to combine and reinterpret multiple semantics.

[0206] Next, the server calculates the feature correlation parameters between the transformed representation features generated by each individual and the known semantic features of traffic accidents. Known semantic features of traffic accidents may include "high-speed driving is prone to rear-end collisions" and "congested roads are prone to scratches". For individual 1, if its enhanced representation of "dense traffic flow and fast driving speed" has a strong correlation with the feature of "high-speed driving is prone to rear-end collisions", then its feature correlation parameter will be higher; for individual 2, if its attention to "moderate road congestion" is well matched with the feature of "congested roads are prone to scratches", the feature correlation parameter will also be correspondingly higher.

[0207] At the same time, the characteristic uniqueness parameter of each individual is calculated, that is, the degree of difference between the transformed representation characteristics generated by other individuals in the initial individual population. If the transformed representation of individual 1 is significantly different from most other individuals and has a unique understanding and emphasis on traffic semantics, then its characteristic uniqueness parameter will be high; if the representation of individual 2 is similar to many individuals, the characteristic uniqueness parameter will be relatively low.

[0208] The feature correlation parameter and the feature uniqueness parameter are weighted and fused to obtain the fitness value of each individual. Assume that the weight of the feature correlation parameter is 0.7 and the weight of the feature uniqueness parameter is 0.3. For individual 1, if its feature correlation parameter is 0.8 and its feature uniqueness parameter is 0.6, its fitness value is 0.7×0.8+0.3×0.6=0.74.

[0209] Randomly select multiple individuals from the initial individual population to compete. For example, randomly select 50 individuals. Sort the individuals in the initial individual population according to the fitness value of each individual. The individuals with higher fitness values ​​rank higher. Suppose the fitness value of individual 1 is 0.74, the fitness value of individual 2 is 0.85, the fitness value of individual 3 is 0.68, and so on, and sort them from high to low according to the fitness value.

[0210] Based on the set ratio (for example, the top 20%), the top-ranked target individuals are selected as parent individuals. Assuming the top 20% are the first 10 individuals, these 10 individuals are selected as parent individuals.

[0211] For the selected parent individuals, a uniform exchange is performed at the corresponding vector element position with a first set probability (such as 0.6) to generate new child individuals. For example, parent individual 1 and parent individual 2 are exchanged at the vector element position of "traffic density" and "vehicle speed", thereby generating some characteristics of the new child individuals.

[0212] Furthermore, the vector elements of the child individuals are generated by linearly combining the vector elements of the parent individuals. For example, the feature of "road congestion degree" of the child individuals may be obtained by multiplying the feature value of the parent individual 1 by 0.4 and the feature value of the parent individual 2 by 0.6.

[0213] Add random noise to the vector elements of the offspring individuals according to the second set probability to achieve mutation. Assume that the second set probability is dynamically adjusted according to the fitness value of the offspring individuals. Offspring individuals with low fitness values ​​have higher second set probabilities. If the fitness value of an offspring individual is 0.5 and the average fitness value is 0.7, then its second set probability may be higher, such as 0.7, to change its characteristics to a greater extent and promote its optimization; if the fitness value of an offspring individual is 0.8, which is close to the average fitness value, its second set probability may be reduced to 0.3.

[0214] Decode the offspring individuals generated by genetic operations and convert them into processable feature representations. For example, convert the encoding form of the offspring individuals after mutation and exchange operations back to understandable and processable feature representations such as "abnormal traffic flow on a specific road section" and "high risk of vehicle intersection".

[0215] Then, principal component analysis is used to decompose the decoded processable feature representation to extract the main component features and independent component features. The main component features may be comprehensive indicators reflecting the traffic conditions of the entire intersection, such as the overall traffic flow and average speed; the independent component features may be the flow of vehicles in certain specific directions, or the behavior of pedestrians in a specific time period.

[0216] According to the pre-set traffic accident semantic rules, the decomposed main component features and independent component features are screened and filtered. For example, if an independent component feature represents a branch road with few vehicles and has a low correlation with common traffic accident patterns, it will be removed.

[0217] The filtered component features are reorganized according to the target strategy, and the key components (such as high traffic flow and high vehicle intersection risk) in the filtered component features are strengthened through nonlinear transformation (such as Sigmoid function). For example, for the feature of large traffic flow, the Sigmoid function is used to strengthen it, so that it has a greater influence in subsequent processing.

[0218] Finally, the traffic accident semantic enhancement features are loaded into the fully connected mapping unit to generate the second traffic accident knowledge point data for each traffic accident knowledge point. For example, the probability of generating the traffic accident knowledge point "high risk of rear-end collision at intersection" is 0.7, the probability of generating the traffic accident knowledge point "low risk of collision between pedestrians and vehicles" is 0.2, and so on.

[0219] Assume that in another scenario, the target holographic road traffic perception stream obtained by the server is about the traffic conditions of a highway during holidays. The traffic volume is very large, the speed of vehicles varies greatly, and some sections of the road are under construction.

[0220] Again, noise removal and contrast adjustment are performed first. Noise may appear as a small range of offsets in the positioning data of some vehicles, which are processed to obtain more accurate vehicle location information. Contrast adjustment highlights the signs of construction sections and changes in vehicle speed.

[0221] The convolutional neural network’s multiple convolutional layers capture features such as vehicle distribution patterns, speed gradients, and the impact of construction sections on traffic flow. Activation functions and pooling operations further extract and simplify these features.

[0222] The attention mechanism assigns weights based on the importance of feature channels. For example, the vehicle speed gradient has a higher weight because it is closely related to potential rear-end collisions.

[0223] The semantic encoding unit converts these features into semantic codes, such as "high-speed traffic flow is unstable" and "there are obstacles on the construction section".

[0224] The traffic accident semantic enhancement unit generates an initial individual population, calculates the feature correlation and uniqueness parameters of the individuals, performs genetic operations to generate offspring individuals, and performs mutation and decoding according to the fitness values.

[0225] Principal component analysis extracts the main component features, such as overall traffic flow fluctuations and vehicle density on construction sections, and filters out features with low relevance to accidents, such as brand information of certain specific vehicles.

[0226] Finally, the fully connected mapping unit generates data for traffic accident knowledge points, such as the probability of "high-speed road section collision risk is high" is 0.85, and the probability of "construction road section scratch risk is low" is 0.15.

[0227] Consider another scenario where the server obtains a target holographic road traffic perception stream of a rural road at night. The road lighting conditions are poor, there are few vehicles but the speed is fast, and animals occasionally appear.

[0228] After noise processing and contrast adjustment, the lights of vehicles and the figures of animals are highlighted. The convolutional layer captures features such as the light tracks of vehicles and the locations where animals appear.

[0229] The attention mechanism focuses on important features such as vehicle speed and animal appearance frequency. The semantic encoding converts it into "vehicles driving at high speed on rural roads at night and animals disturbing".

[0230] The subsequent processing process is similar to the above, and finally the fully connected mapping unit generates traffic accident knowledge point data such as "the risk of accidents caused by animals on rural roads at night" with a probability of 0.6.

[0231] Through the above multiple detailed and specific scenario examples, the server can comprehensively and deeply process various types of target holographic road traffic perception streams, accurately generate valuable data for each traffic accident knowledge point, and provide strong support for traffic management and safety assurance.

[0232] Assume there is a new scenario, the target holographic road traffic perception stream received by the server is about the traffic conditions of roads around a school during the school and after school hours. At this time, there are a large number of students, parents and pick-up vehicles on the road, and the traffic order is relatively chaotic.

[0233] In the process of removing noise, some vehicle position and speed detection errors caused by dense crowds may be corrected. The contrast adjustment strategy will emphasize traffic signs at the school gate, crosswalks, and dense areas of students and vehicles.

[0234] The convolutional layer can capture features such as vehicles frequently slowing down to avoid pedestrians and students crossing the road in groups. The activation function and pooling operation further refine these features to make them more representative.

[0235] The attention mechanism gives a higher weight to the interaction features between vehicles and pedestrians because this is a key factor leading to potential traffic accidents.

[0236] The semantic encoding unit converts these complex features into clear semantic descriptions, such as "the traffic around the school is busy and there are frequent interactions between people and vehicles."

[0237] In the traffic accident semantic enhancement unit, the generated initial individual population will try various transformations and combinations of these semantics. By calculating the feature correlation and uniqueness parameters, more valuable individuals are screened out, and offspring individuals are generated through genetic operations. During the mutation and decoding process, the offspring individuals continuously optimize and adjust the description of traffic conditions.

[0238] Principal component analysis extracts the main traffic characteristics, such as the average deceleration frequency of vehicles and the density of pedestrians, and filters out some minor details, such as the color of individual students' clothing and other characteristics that are not related to traffic accidents.

[0239] Finally, the fully connected mapping unit generates probability data for specific traffic accident knowledge points, such as the probability of "the risk of vehicle scratches at the school gate is high" is 0.7, and the probability of "the risk of students being hit by vehicles is low" is 0.3.

[0240] Let's look at another scene. The target holographic road traffic perception stream obtained by the server depicts the traffic conditions on the roads near a large shopping mall on weekends. There are not only a large number of shoppers driving or walking, but also bus stations and taxi stops, and the traffic conditions are very complicated.

[0241] The noise removal process removes abnormal data fluctuations caused by the movement of people and the frequent start and stop of vehicles. Contrast adjustment highlights the entrance and exit of buses, the waiting area for taxis, and the concentrated areas where pedestrians cross the road.

[0242] The convolutional layers capture features such as vehicles queuing at parking lot entrances and exits, the impact of buses stopping on vehicles behind them, and pedestrians stopping in the middle of the road.

[0243] The activation function and pooling operation perform nonlinear transformation and dimensionality reduction on these features to extract key information.

[0244] The attention mechanism assigns weights to different features based on traffic rules and accident risks. For example, the feature of a pedestrian staying in a non-crosswalk area may be given a higher weight.

[0245] The semantic encoding unit converts the processed features into clear semantic expressions, such as "traffic congestion on roads around the shopping mall and pedestrians' behavior is irregular."

[0246] In the subsequent processing, the understanding and description of traffic conditions are continuously optimized through steps such as traffic accident semantic enhancement unit, genetic operation, and principal component analysis.

[0247] The fully connected mapping unit ultimately generates traffic accident knowledge point data such as "the risk of rear-end collision at the intersection near the shopping mall" has a probability of 0.55 and "the risk of pedestrians being hit by vehicles is high" has a probability of 0.45.

[0248] Assume that there is another scenario where the target holographic road traffic perception stream received by the server reflects the traffic conditions of a mountain road after rain. The road is slippery, the visibility is poor, and there are many bends and steep slopes.

[0249] When processing noise and adjusting contrast, emphasis is placed on the water on the road, the curvature of the curves and the slope of the steep slopes.

[0250] The convolutional layers capture features such as the vehicle skidding on a slippery road, the driver’s deceleration around a curve, and the change in power when the vehicle is climbing a hill.

[0251] After a series of processing and analysis steps, the fully connected mapping unit finally gives traffic accident knowledge point data such as "the risk of brake failure on mountain roads after rain is high" with a probability of 0.6 and "the vehicle is at risk of losing control on a steep slope" with a probability of 0.4.

[0252] The numerous detailed scenario examples above fully demonstrate the comprehensiveness and accuracy of the server in processing various complex road traffic perception flows, and can provide a reliable decision-making basis for traffic management and safety assurance.

[0253] Let's assume another scenario, the target holographic road traffic perception stream obtained by the server is about the traffic conditions of a city expressway in the early morning. At this time, the traffic volume gradually increases, but the overall traffic is relatively smooth, except for some sections of the road that are foggy.

[0254] In the process of removing noise and adjusting contrast, the vehicle detection errors caused by fog are corrected, and the areas with thicker fog and the lights of vehicles are highlighted.

[0255] The convolutional layer can capture features such as the vehicle's deceleration behavior on foggy roads and the changes in the safe distance between vehicles.

[0256] Subsequent processing steps are performed in sequence, and finally the fully connected mapping unit generates traffic accident knowledge point data such as "the risk of collision on foggy sections of expressways is low" with a probability of 0.2 and "the risk of vehicles having obstructed vision due to fog" with a probability of 0.6.

[0257] Figure 2 The hardware structure of the road traffic accident detection system 100 based on holographic perception for implementing the above-mentioned road traffic accident detection method based on holographic perception provided in the embodiment of the present application is shown as follows: Figure 2 As shown, the road traffic accident detection system 100 based on holographic perception may include a processor 110 , a machine-readable storage medium 120 , a bus 130 and a communication unit 140 .

[0258] In one possible design, the road traffic accident detection system 100 based on holographic perception can be a single server or a server group. The server group can be centralized or distributed (for example, the road traffic accident detection system 100 based on holographic perception can be a distributed system). In some embodiments, the road traffic accident detection system 100 based on holographic perception can be local or remote. For example, the road traffic accident detection system 100 based on holographic perception can access information and / or data stored in a machine-readable storage medium 120 via a network. For another example, the road traffic accident detection system 100 based on holographic perception can be directly connected to the machine-readable storage medium 120 to access the stored information and / or data. In some embodiments, the road traffic accident detection system 100 based on holographic perception can be implemented on a road traffic accident detection system based on holographic perception. As an example only, the road traffic accident detection system based on holographic perception can include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an internal cloud, a multi-layer cloud, etc. or any combination thereof.

[0259] The machine-readable storage medium 120 may store data and / or instructions. In some embodiments, the machine-readable storage medium 120 may store data obtained from an external terminal. In some embodiments, the machine-readable storage medium 120 may store data and / or instructions that the holographic perception-based road traffic accident detection system 100 uses to execute or use to complete the exemplary method described in this application.

[0260] During the specific implementation process, one or more processors 110 execute computer executable instructions stored in the machine-readable storage medium 120, so that the processor 110 can execute the road traffic accident detection method based on holographic perception in the above method embodiment. The processor 110, the machine-readable storage medium 120 and the communication unit 140 are connected through the bus 130, and the processor 110 can be used to control the sending and receiving actions of the communication unit 140.

[0261] The specific implementation process of the processor 110 can refer to the various method embodiments executed by the above-mentioned road traffic accident detection system 100 based on holographic perception. The implementation principles and technical effects are similar, and this embodiment will not be repeated here.

[0262] In addition, an embodiment of the present application also provides a readable storage medium, in which computer executable instructions are preset. When a processor executes the computer executable instructions, the above-mentioned road traffic accident detection method based on holographic perception is implemented.

[0263] It should be noted that in order to simplify the description disclosed in this application and thus help understand one or more embodiments of the invention, in the foregoing description of the embodiments of the present application, multiple features are sometimes combined into one embodiment, drawings, or descriptions thereof. Similarly, it should be noted that in order to simplify the description disclosed in this application and thus help understand one or more embodiments of the invention, in the foregoing description of the embodiments of the present application, multiple features are sometimes combined into one embodiment, drawings, or descriptions thereof.

Claims

1. A road traffic accident detection method based on holographic perception, characterized in that: The method comprises: Loading the template holographic road traffic perception stream in the basic template holographic road traffic perception stream sequence into multiple traffic accident knowledge point description networks, generating high-likelihood traffic accident knowledge points corresponding to each of the traffic accident knowledge point description networks; According to the first a priori traffic accident knowledge point of the template holographic road traffic perception flow and the comparative analysis of the high likelihood traffic accident knowledge point corresponding to each traffic accident knowledge point description network, the target template holographic road traffic perception flow is extracted from the basic template holographic road traffic perception flow sequence, and the extracted target template holographic road traffic perception flow constitutes the target template holographic road traffic perception flow sequence; Loading the target template holographic road traffic perception stream into each of the traffic accident knowledge point description networks to generate a first positive prediction confidence corresponding to each traffic accident knowledge point; Calculating a first network training cost corresponding to each of the traffic accident knowledge point description networks according to the first a priori traffic accident knowledge point of each of the traffic accident knowledge points and the first positive prediction confidence; For each of the traffic accident knowledge point description networks, training the traffic accident knowledge point description network according to the first network training cost corresponding to the traffic accident knowledge point description network; The target template holographic road traffic perception flow sequence is overwritten with the basic template holographic road traffic perception flow sequence, and the step of loading the template holographic road traffic perception flow in the basic template holographic road traffic perception flow sequence into the multiple traffic accident knowledge point description networks is returned until the first network training cost converges, and the converged traffic accident knowledge point description network is determined from the multiple traffic accident knowledge point description networks; The step of determining a converged traffic accident knowledge point description network from a plurality of traffic accident knowledge point description networks comprises: Acquire a third verification holographic road traffic perception stream sequence, wherein the third verification holographic road traffic perception stream comprises a plurality of third verification holographic road traffic perception streams, and the third verification holographic road traffic perception stream has a second key priori traffic accident knowledge point; Loading the third verification holographic road traffic perception stream into each of the traffic accident knowledge point description networks to generate high likelihood traffic accident knowledge point data for each of the traffic accident knowledge point description networks; Determining a second performance indicator of each of the traffic accident knowledge point description networks based on a comparison between the second key a priori traffic accident knowledge point and the high likelihood traffic accident knowledge point data of each of the traffic accident knowledge point description networks; The converged traffic accident knowledge point description network is determined based on the second performance indicator of each of the traffic accident knowledge point description networks.

2. The road traffic accident detection method based on holographic perception according to claim 1 is characterized in that: The step of loading the template holographic road traffic perception stream in the basic template holographic road traffic perception stream sequence into a plurality of traffic accident knowledge point description networks, and generating a high likelihood traffic accident knowledge point corresponding to each of the traffic accident knowledge point description networks, comprises: For each of the traffic accident knowledge point description networks, the template holographic road traffic perception stream in the basic template holographic road traffic perception stream sequence is loaded into the traffic accident knowledge point description network to generate the first positive prediction confidence corresponding to each of the traffic accident knowledge points; The traffic accident knowledge point with the maximum first positive prediction confidence is determined as the high-likelihood traffic accident knowledge point corresponding to the traffic accident knowledge point description network.

3. The road traffic accident detection method based on holographic perception according to claim 2 is characterized in that: The determining of the traffic accident knowledge point having the maximum first positive prediction confidence as the high likelihood traffic accident knowledge point corresponding to the traffic accident knowledge point description network includes: Obtaining threshold setting values ​​of a plurality of traffic accident knowledge points; Using the traffic accident knowledge point whose first positive prediction confidence is greater than the threshold setting value as a reference traffic accident knowledge point; Determine the reference traffic accident knowledge point with the maximum first positive prediction confidence as the high-likelihood traffic accident knowledge point corresponding to the traffic accident knowledge point description network.

4. The road traffic accident detection method based on holographic perception according to claim 1 is characterized in that: The traffic accident knowledge point description network includes a traffic accident semantic enhancement unit and a fully connected mapping unit; After determining a converged traffic accident knowledge point description network from the plurality of traffic accident knowledge point description networks, the method further includes: Distinguish and identify the first a priori traffic accident knowledge point of each traffic accident knowledge point of the target template holographic road traffic perception flow, and obtain first attribute knowledge point data, second attribute knowledge point data, and empty knowledge point data, wherein the network training distinction difficulty between the first attribute knowledge point data and the empty knowledge point data is less than the network training distinction difficulty between the second attribute knowledge point data and the empty knowledge point data; In the target template holographic road traffic perception flow sequence, a first template holographic road traffic perception flow annotated with the second attribute knowledge point data and a second template holographic road traffic perception flow annotated with the empty knowledge point data are obtained; Acquire a first difficult-to-distinguish template perception stream from the first template holographic road traffic perception stream, acquire a second difficult-to-distinguish template perception stream from the second template holographic road traffic perception stream, and output other template holographic road traffic perception streams except the first difficult-to-distinguish template perception stream and the second difficult-to-distinguish template perception stream in the target template holographic road traffic perception stream sequence as non-difficult-to-distinguish template perception streams; Loading the first difficult-to-distinguish template perception flow into the converged traffic accident knowledge point description network, generating a first prediction confidence corresponding to the first a priori traffic accident knowledge point for each of the traffic accident knowledge points predicted by the converged traffic accident knowledge point description network; Determine first derivative training supervision information of the traffic accident knowledge point based on the first a priori traffic accident knowledge point and the first prediction confidence corresponding to the first a priori traffic accident knowledge point, and calculate a second network training sub-cost of the first difficult-to-distinguish template perception flow based on the first derivative training supervision information; For the second difficult-to-distinguish template perception flow, determining second derivative training supervision information of each of the traffic accident knowledge points, and calculating a third network training sub-cost of the second difficult-to-distinguish template perception flow based on the second derivative training supervision information; Calculating a fourth network training sub-cost of the non-difficult-to-distinguish template-perceived flow, and calculating a second network training cost according to the second network training sub-cost, the third network training sub-cost, and the fourth network training sub-cost; The traffic accident semantic enhancement unit and the fully connected mapping unit in the converged traffic accident knowledge point description network are trained according to the second network training cost.

5. The road traffic accident detection method based on holographic perception according to claim 4 is characterized in that: The plurality of the first priori traffic accident knowledge points include the empty knowledge point data and various priori traffic accident knowledge points of various accident severity levels; The step of distinguishing and identifying the first a priori traffic accident knowledge point of each traffic accident knowledge point of the target template holographic road traffic perception flow to obtain first attribute knowledge point data, second attribute knowledge point data, and empty knowledge point data includes: Extracting the empty knowledge point data from the plurality of the first priori traffic accident knowledge points; For each prior traffic accident knowledge point of each accident severity level, the second verification holographic road traffic perception stream marked with the prior traffic accident knowledge point in the second verification holographic road traffic perception stream sequence is loaded into the traffic accident knowledge point description network to generate first traffic accident knowledge point data; According to the comparative analysis of the first traffic accident knowledge point data and the prior traffic accident knowledge point, the prior traffic accident knowledge point is distinguished and identified to obtain the first attribute knowledge point data or the second attribute knowledge point data; The step of distinguishing and identifying the prior traffic accident knowledge point based on the comparative analysis of the first traffic accident knowledge point data and the prior traffic accident knowledge point to obtain the first attribute knowledge point data or the second attribute knowledge point data includes: Outputting the number of second verification samples annotated with the priori traffic accident knowledge point in the second verification holographic road traffic perception stream sequence as a first statistical number; Outputting the number of second verification samples in the second verification holographic road traffic perception stream in which the priori traffic accident knowledge point is marked in the second verification holographic road traffic perception stream sequence, in which the first traffic accident knowledge point data and the priori traffic accident knowledge point do not match, as a second statistical number; Determining the network training differentiation difficulty according to the first statistical quantity and the second statistical quantity; If the network training distinction difficulty is less than the first set threshold value, the prior traffic accident knowledge point is distinguished and identified to obtain the first attribute knowledge point data; if the network training distinction difficulty is not less than the first set threshold value, the prior traffic accident knowledge point is distinguished and identified to obtain the second attribute knowledge point data.

6. The road traffic accident detection method based on holographic perception according to claim 4 is characterized in that: The step of acquiring a first difficult-to-distinguish template perception stream from the first template holographic road traffic perception stream, and acquiring a second difficult-to-distinguish template perception stream from the second template holographic road traffic perception stream comprises: Loading the first template holographic road traffic perception stream into the converged traffic accident knowledge point description network, generating the high likelihood traffic accident knowledge point predicted by the converged traffic accident knowledge point description network, and the first prediction confidence corresponding to the first a priori traffic accident knowledge point of the first template holographic road traffic perception stream predicted by the converged traffic accident knowledge point description network; If the high likelihood traffic accident knowledge point predicted by the converged traffic accident knowledge point description network does not match the first a priori traffic accident knowledge point of the first template holographic road traffic perception flow, and it is determined that the first prediction confidence and the first threshold setting value corresponding to the first a priori traffic accident knowledge point of the first template holographic road traffic perception flow meet the first discrimination condition, then the first template holographic road traffic perception flow is output as the first difficult-to-distinguish template perception flow; Loading the second template holographic road traffic perception stream into the converged traffic accident knowledge point description network, generating a second prediction confidence corresponding to the first a priori traffic accident knowledge point of the second template holographic road traffic perception stream predicted by the converged traffic accident knowledge point description network; If it is determined that the second prediction confidence and the second threshold setting value corresponding to the first prior traffic accident knowledge point of the second template holographic road traffic perception flow meet the second judgment condition, the second template holographic road traffic perception flow is output as the second difficult-to-distinguish template perception flow.

7. The road traffic accident detection method based on holographic perception according to claim 6 is characterized in that: The determining that the first prediction confidence and the first threshold setting value corresponding to the first priori traffic accident knowledge point of the first template holographic road traffic perception flow meet the first discrimination condition includes: Obtaining a first priori weight, determining a first product result of the first priori weight and the first threshold setting value, and if the first prediction confidence is less than the first product result, determining that the first prediction confidence and the first threshold setting value meet the first discrimination condition; The determining that the second prediction confidence and the second threshold setting value corresponding to the first priori traffic accident knowledge point of the second template holographic road traffic perception flow meet the second discrimination condition includes: Obtain a second prior weight, determine a second product result of the second prior weight and the second threshold setting value, and if the second prediction confidence is less than the second product result, determine that the second prediction confidence and the second threshold setting value meet the second judgment condition.

8. The road traffic accident detection method based on holographic perception according to claim 1 is characterized in that: The method further comprises: Obtain target holographic road traffic perception flow; The target holographic road traffic perception flow is loaded into a traffic accident knowledge point description network, and second traffic accident knowledge point data is obtained for each traffic accident knowledge point, wherein the second traffic accident knowledge point data represents whether the traffic accident characteristics of the target holographic road traffic perception flow conform to the traffic accident knowledge point; Wherein, the traffic accident knowledge point description network includes a semantic encoding unit, a traffic accident semantic enhancement unit and a fully connected mapping unit; The step of loading the target holographic road traffic perception stream into a traffic accident knowledge point description network and obtaining second traffic accident knowledge point data for each traffic accident knowledge point includes: A Gaussian filtering algorithm is used to remove noise in the target holographic road traffic perception stream to generate a noise-cleaned perception stream, and the contrast of the noise-cleaned perception stream is adjusted based on a preset contrast adjustment strategy to highlight key elements in the traffic scene corresponding to the noise-cleaned perception stream to generate a preprocessed perception stream; A multi-layer convolutional layer in a convolutional neural network is used to capture different features in the preprocessed perception stream, and an activation function is used to perform nonlinear transformation on the captured different features, and then a maximum pooling operation is used to downsample the output of the last convolutional layer, and the output of the last convolutional layer is used as a preliminary feature representation, and a global average pooling operation is performed on the preliminary feature representation to obtain the average value of each feature channel, and the weight of each feature channel is calculated using an attention mechanism according to the average value of each feature channel, and the weight of each feature channel is normalized using a softmax function to obtain the attention weight of each feature channel; fusing the preliminary feature representation with the attention weight of each corresponding feature channel to obtain a weighted feature representation, and fusing the weighted feature representation to obtain a traffic perception attention feature; Loading the traffic perception attention feature into the semantic coding unit to generate a semantic coding result, using the semantic coding result from the semantic coding unit as an input of the traffic accident semantic enhancement unit to generate an initial individual population, wherein each individual in the initial individual population is a transformed representation of the input semantic coding result; Calculating a feature correlation parameter between the transformed representation feature generated by each individual and the known semantic feature of the traffic accident and a feature uniqueness parameter of each individual, wherein the feature uniqueness parameter is used to characterize the degree of difference from the transformed representation features generated by other individuals in the initial individual population; The feature correlation parameter and the feature uniqueness parameter are weightedly fused to obtain the fitness value of each individual; A plurality of individuals are randomly selected from the initial individual population for competition, the individuals in the initial individual population are sorted according to the fitness value of each individual, a target individual with a higher sorting is selected as a parent individual based on a set ratio, and for the selected parent individual, a uniform exchange is performed at a corresponding vector element position with a first set probability to generate a new child individual, and a vector element of the child individual is generated by linearly combining the vector elements of the parent individual; Adding random noise to the vector elements of the offspring individuals according to a second set probability to achieve mutation, wherein the second set probability is dynamically adjusted according to the fitness value of the offspring individuals, and the offspring individuals with low fitness values ​​have a higher second set probability; Decode the offspring individuals generated by genetic operations and convert them into processable feature representations; The decoded processable feature representation is decomposed using principal component analysis to extract the main component features and independent component features. The decomposed main component features and independent component features are screened and filtered according to the pre-set traffic accident semantic rules to remove some component features that are irrelevant or weakly related to the traffic accident semantics. The screened component features are reorganized according to the target strategy, and the key components in the screened component features are enhanced through nonlinear transformation to obtain the final traffic accident semantic enhancement features. The traffic accident semantic enhancement feature is loaded into the fully connected mapping unit, and the second traffic accident knowledge point data is generated for each traffic accident knowledge point.

9. A road traffic accident detection system based on holographic perception, characterized in that: The road traffic accident detection system based on holographic perception includes a processor and a memory, the memory is connected to the processor, the memory is used to store programs, instructions or codes, and the processor is used to execute the programs, instructions or codes in the memory to implement the road traffic accident detection method based on holographic perception as described in any one of claims 1 to 8.

Images