Method, device and electronic equipment for identifying road accidents using swarm intelligence

Abstract

The present application provides a method, device, and electronic device for identifying roads using swarm intelligence. The method for identifying roads using swarm intelligence includes: obtaining a road traffic video; performing image recognition on video frames in the road traffic video to determine a suspected accident recognition result based on image recognition that appears on the road corresponding to the road traffic video; using the road traffic video to train a corresponding swarm intelligence trajectory simulation model; obtaining a mobile group feature vector based on the swarm intelligence trajectory simulation model, providing it to a pre-trained classification model to obtain a suspected accident classification result based on swarm intelligence; and analyzing and determining a road accident in the road traffic video by combining the suspected accident recognition result based on image recognition and the suspected accident classification result based on swarm intelligence.

Description

Technical Field

[0001] The present invention relates to the field of road accident identification, and specifically to a method and device for identifying road accidents using group intelligence. The present application also provides an electronic device and a computer storage medium; the present application also provides a road video display device. Background Art

[0002] With the continuous development of urban construction and the continuous improvement of people's living standards, the road mileage and the number of motor vehicles have also increased rapidly. Simply detecting road accidents by manpower will cause a lot of waste of manpower and material resources. On this basis, in order to conveniently and quickly detect road traffic conditions, the image recognition method of the target detection model is usually used to identify the images taken by security cameras in various corners of urban traffic roads (generally expressed as video frames captured by the camera) to determine road abnormalities. However, due to the influence of shooting angles, shooting environments and various other comprehensive factors on the road, it is impossible to accurately judge road abnormalities based solely on the images taken by the camera.

[0003] In addition, the road accident identification method under the existing technology has some defects: it lacks the ability to judge the relationship between vehicles, people and road traffic factors on the road. For example, vehicles waiting for traffic lights to change on the road are misjudged as long-term stranded vehicles, resulting in false reports of traffic accidents.

[0004] Therefore, how to accurately and quickly identify road accidents has become a problem that technical personnel in this field need to solve urgently. Summary of the invention

[0005] The present application provides a method, device, electronic device and computer storage medium for identifying road accidents using swarm intelligence to solve the problems existing in the above-mentioned prior art. The present application also provides a road video display device.

[0006] The present application embodiment provides a method for identifying road accidents using swarm intelligence, including:

[0007] Obtain road traffic videos;

[0008] Performing image recognition on video frames in the road traffic video to determine a suspected accident recognition result based on image recognition that occurs on a road corresponding to the road traffic video;

[0009] Using the road traffic video training to obtain a corresponding swarm intelligence trajectory simulation model;

[0010] According to the swarm intelligence trajectory simulation model, a mobile group feature vector is obtained, and provided to a pre-trained classification model to obtain a suspected accident classification result based on swarm intelligence;

[0011] The road accidents in the road traffic video are analyzed and determined by combining the suspected accident identification result based on image recognition and the suspected accident classification result based on swarm intelligence.

[0012] Optionally, performing image recognition on the video frames in the road traffic video to determine a suspected accident recognition result based on image recognition appearing on the road corresponding to the road traffic video includes:

[0013] The video frames of the road traffic video are input into a trained accident visual recognition model to obtain a recognition result output by the accident visual recognition model, wherein the recognition result at least includes: a suspected accident area in the video frame, a first confidence level of an accident occurring in the suspected accident area, and may also include: a suspected accident category of the suspected accident area.

[0014] Optionally, the method of obtaining a corresponding swarm intelligence trajectory simulation model by using the road traffic video training includes:

[0015] Extracting structured data of road traffic based on the road traffic video;

[0016] Accumulating the structured data of the road traffic until a predetermined time threshold is reached;

[0017] providing the structured data of the road traffic accumulated to the predetermined time threshold as training samples to an initial swarm intelligent trajectory simulation model, and training the initial swarm intelligent trajectory simulation model;

[0018] The swarm intelligence trajectory simulation model trained as above is used as the swarm intelligence trajectory simulation model.

[0019] Optionally, providing the structured data of the road traffic accumulated to the predetermined time threshold as training samples to an initial swarm intelligence trajectory simulation model includes:

[0020] Constructing a regional road plan corresponding to the road captured by the road traffic video;

[0021] According to each video frame in the road traffic video, identifying and acquiring attribute information of a mobile individual and attribute information of a traffic element therein; wherein the attribute information of the mobile individual and the attribute information of the traffic element at least include location information of the mobile individual and the traffic element;

[0022] Mapping the attribute information of the mobile individuals and the attribute information of the traffic elements in each video frame to the corresponding positions of the regional road plan to obtain a road mobile group trajectory map;

[0023] The mobile group trajectory diagram is used as the training sample and provided to an initial group intelligent trajectory simulation model.

[0024] Optionally, the identifying and obtaining the attribute information of the mobile group and the attribute information of the traffic elements in each video frame of the road traffic video includes:

[0025] Using a pre-trained detection model, identifying moving individuals and traffic elements contained in the road traffic video;

[0026] Using a pre-trained second classification model, classifying the moving individuals and traffic elements in the road traffic video to obtain the category of each moving individual and the category of each traffic element;

[0027] Corresponding to the categories of the mobile individuals and the categories of the traffic elements, attribute information of each of the mobile individuals and attribute information of the traffic elements are obtained accordingly.

[0028] Optionally, the attribute information further includes one or more of the following information: category information of the mobile individual, category information of the traffic element, moving speed information of the mobile individual, and retention time information of the mobile individual;

[0029] The moving speed information and residence time of the moving individual are obtained by:

[0030] According to the position information of each moving individual reflected in each video frame of the road traffic video, combined with the temporal relationship between the video frames, the moving speed information of each moving individual and the residence time information of the moving individual are obtained.

[0031] Optionally, obtaining a mobile group feature vector according to the swarm intelligence trajectory simulation model and providing it to a pre-trained classification model to obtain a suspected accident classification result based on swarm intelligence includes:

[0032] Extracting the logic parameters of the group intelligent trajectory model, using the logic parameters, or further using one or more of the moving speed information and the retention time information of each of the moving individuals in addition to the logic parameters, to construct a moving group feature vector;

[0033] The mobile group feature vector is input into the classification model to obtain a suspected accident classification result output by the classification model.

[0034] Optionally, combining the suspected accident identification result based on image recognition and the suspected accident classification result based on swarm intelligence to analyze and determine the road accident in the road traffic video includes:

[0035] Determining a second confidence threshold for the suspected accident according to the suspected accident classification result output by the classification model;

[0036] It is determined whether the first confidence level of the accident occurring in the suspected accident area output by the accident visual model is greater than the second confidence level threshold; if so, it is determined that a road accident has occurred in the road video.

[0037] Optionally, the suspected accident classification result output by the classification model includes: a determination result of whether an accident has occurred on the road, and a confidence level corresponding to the determination result, and may further include: a category of the suspected road accident.

[0038] Optionally, combining the suspected accident identification result and the suspected accident classification result based on swarm intelligence to analyze and determine the road accident in the road traffic video includes:

[0039] If a certain video frame is judged as suspected of having an accident in the output of the classification model, and the suspected accident area is also included in the recognition result of the accident visual recognition model, it is confirmed that a road accident has occurred in the area reflected by the video frame.

[0040] Optionally, the method further includes:

[0041] It is determined whether the first confidence level of an accident occurring in the suspected accident area is greater than a preset first confidence level threshold; if so, it is determined that a road accident has occurred in the road video.

[0042] The present application also provides a device for identifying road accidents using swarm intelligence, including:

[0043] An acquisition module, used for acquiring road traffic videos;

[0044] A recognition module, configured to perform image recognition on video frames in the road traffic video, and determine a suspected accident recognition result based on image recognition that occurs on a road corresponding to the road traffic video;

[0045] A training module, used to obtain a corresponding swarm intelligence trajectory simulation model using the road traffic video training;

[0046] A classification module, used to obtain a mobile group feature vector according to the group intelligence trajectory simulation model, and provide it to a pre-trained classification model to obtain a suspected accident classification result based on group intelligence;

[0047] The analysis module is used to combine the suspected accident identification result based on image recognition and the suspected accident classification result based on swarm intelligence to analyze and determine the road accident in the road traffic video.

[0048] The present application also provides an electronic device, which includes:

[0049] processor;

[0050] A memory for storing a method program, wherein when the program is read and executed by the processor, the following steps are performed:

[0051] Obtain road traffic videos;

[0052] Performing image recognition on video frames in the road traffic video to determine a suspected accident recognition result based on image recognition that occurs on a road corresponding to the road traffic video;

[0053] Using the road traffic video training to obtain a corresponding swarm intelligence trajectory simulation model;

[0054] According to the swarm intelligence trajectory simulation model, a mobile group feature vector is obtained, and provided to a pre-trained classification model to obtain a suspected accident classification result based on swarm intelligence;

[0055] The road accidents in the road traffic video are analyzed and determined by combining the suspected accident identification result based on image recognition and the suspected accident classification result based on swarm intelligence.

[0056] The present application also provides a computer storage medium, wherein the computer storage medium stores a computer program, and when the program is executed, the following steps are implemented:

[0057] Obtain road traffic videos;

[0058] Performing image recognition on video frames in the road traffic video to determine a suspected accident recognition result based on image recognition that occurs on a road corresponding to the road traffic video;

[0059] Using the road traffic video training to obtain a corresponding swarm intelligence trajectory simulation model;

[0060] According to the swarm intelligence trajectory simulation model, a mobile group feature vector is obtained, and provided to a pre-trained classification model to obtain a suspected accident classification result based on swarm intelligence;

[0061] The road accidents in the road traffic video are analyzed and determined by combining the suspected accident identification result based on image recognition and the suspected accident classification result based on swarm intelligence.

[0062] The present application also provides a road video display device, including:

[0063] Displaying the road display interface at the video display terminal, wherein the road display interface includes: real-time road traffic video, a moving group trajectory plane map corresponding to the real-time road traffic video, and a road accident analysis result display area;

[0064] The road accident analysis result display area displays real-time traffic status information of the road based on the analysis results of the swarm intelligence trajectory simulation model and the recognition results of image recognition of the video frames in the road traffic video, wherein the road traffic status information includes: road accident information.

[0065] Compared with the prior art, this application has the following advantages:

[0066] The method for identifying road accidents using swarm intelligence provided by the present application includes: obtaining a road traffic video; performing image recognition on the video frames in the road traffic video to determine the suspected accident recognition results based on image recognition that appear in the road corresponding to the road traffic video; using the road traffic video to train and obtain the corresponding swarm intelligence trajectory simulation model; according to the swarm intelligence trajectory simulation model, obtaining a mobile group feature vector, providing it to a pre-trained classification model to obtain a suspected accident classification result based on swarm intelligence; combining the suspected accident recognition result based on image recognition and the suspected accident classification result based on swarm intelligence, analyzing and determining the road accident in the road traffic video. The method obtains visual recognition results of suspected accident areas in the road traffic video by performing image recognition on the video frames in the road traffic video; at the same time, using swarm intelligence technology, the mobile group feature vector constructed by the swarm intelligence trajectory simulation model corresponding to the current road, and the classification model determine the classification results of suspected accidents, and further combining the visual recognition results and the classification results to determine the road accidents in the road traffic video, thereby achieving accurate recognition of road accidents and improving the timeliness of road accident recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 A flow chart of a method for identifying road accidents using swarm intelligence provided in the first embodiment of the present application;

[0068] Figure 1A A schematic diagram of an application scenario embodiment provided by this application;

[0069] Figure 2 A flow chart of another method for identifying road accidents using swarm intelligence provided in the first embodiment of the present application;

[0070] Figure 3 A schematic diagram of the structure of a device for identifying road accidents using swarm intelligence provided in the second embodiment of the present application;

[0071] Figure 4 A schematic diagram of a road video display interface provided in the third embodiment of the present application;

[0072] Figure 5 A schematic diagram of the structure of an electronic device provided in the fourth embodiment of the present application. DETAILED DESCRIPTION

[0073] Many specific details are described in the following description to facilitate a full understanding of the present application. However, the present application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar generalizations without violating the connotation of the present application. Therefore, the present application is not limited to the specific embodiments disclosed below.

[0074] The present application provides a method, device, electronic device and storage medium for identifying road accidents using swarm intelligence, and also provides a road video display device. The following embodiments will be described in detail.

[0075] The core of the method for identifying road accidents using group intelligence provided by the present application is to use image recognition technology and group characteristics of moving groups in road traffic videos for comprehensive analysis, and determine road accidents on the road shown in the road traffic video based on the analysis results. By combining image recognition technology and group characteristic analysis technology, the method can more accurately determine road accidents and improve the accuracy and timeliness of road accident identification.

[0076] In order to more clearly demonstrate the present application, the application scenario of the image processing method provided in the embodiment of the present application is first introduced.

[0077] Some embodiments provided in this application are applied to a monitoring system composed of a video acquisition device 1, a server 2 and a monitoring video terminal, typically, such as a road traffic monitoring system or a community monitoring system. The video acquisition device 12 generally refers to a camera installed at a road traffic scene.

[0078] like Figure 1AAs shown, the figure is a schematic diagram of an application scenario embodiment provided by the present application. The video acquisition device 1 is connected to the server 2, and the live video of the road is sent to the server 2 in real time. The server 2 is configured with a system that uses image recognition (or image visual recognition) and group intelligence technology to identify road accidents in the road. The system first obtains the road traffic video through the acquisition module 101a; then, the recognition module 102a performs image recognition on the video frames in the road traffic video to determine the suspected accident recognition results based on image recognition that appear in the road corresponding to the road traffic video; in the process of the recognition module recognizing the image, the training module 103a uses the road traffic video to train and obtain the corresponding group intelligence trajectory simulation model; then, the classification module 104a obtains the group intelligence trajectory simulation model, obtains the mobile group feature vector, and provides it to the pre-trained classification model to obtain the suspected accident classification result based on group intelligence; finally, the analysis module 105a combines the suspected accident recognition result based on image recognition and the suspected accident classification result based on group intelligence to analyze and determine the road accident in the road traffic video.

[0079] Specifically, road accidents include not only car accidents, vehicle rollovers, and other road accidents that may cause casualties, but also road accidents that may cause road congestion, large-scale spillage on the road surface, and other events that may pose safety hazards. It should be clear that the above application scenario is only a specific embodiment of the road accident identification method described in this application. The purpose of providing the application scenario embodiment is to facilitate understanding of the method of using swarm intelligence to identify roads in this application, and is not used to limit the scope of application of this application.

[0080] Please refer to Figure 1 , which is a flow chart of a method for identifying road accidents using swarm intelligence provided in the first embodiment of the present application. The method includes steps S101 to S105.

[0081] Step S101, obtaining road traffic video.

[0082] In general, the road traffic video refers to a video of a certain road captured in real time by a road traffic safety camera. In an embodiment of the present application, the video can be a live video of a certain road captured by a camera, or a live video of multiple angles of a road captured by multiple cameras, or a live video of multiple roads captured by multiple cameras, and the present application does not impose any restrictions on this.

[0083] After obtaining the road traffic video, the continuous video frames in the road traffic video are simultaneously subjected to image visual analysis and group intelligence analysis of the moving groups in the image. The above analysis process is introduced below with steps S102 to S105.

[0084] Step S102: performing image recognition on the video frames in the road traffic video to determine a suspected accident recognition result based on image recognition that occurs on the road corresponding to the road traffic video.

[0085] In this step, performing image recognition on the video frames in the traffic video refers to performing image visual analysis on the continuous video frames or discrete video frames in the road traffic video. Here, step S102 in the process of implementing the method of using swarm intelligence to identify roads is considered to be the image visual analysis stage.

[0086] The image visual analysis utilizes video frames to identify and track vehicles, pedestrians, and traffic elements on the road (for example, signboards and warning lights temporarily set up by traffic police or owners of faulty vehicles in the event of an accident, etc.), and further performs image processing based on the identification and tracking results.

[0087] Since the method for identifying roads using swarm intelligence provided in the first embodiment of the present application is specifically for identifying road accidents, the process of performing image recognition on the road traffic video frame is specifically implemented by using a pre-trained accident visual recognition model.

[0088] The accident visual recognition model refers to a target detection model for detecting abnormal road conditions. The object detection model (Object Detection) is a branch of computer technology closely related to computer vision and image processing. Its goal is to detect specific semantic target entities in digital images and videos, such as people, buildings, cars, etc., and output the result as a tightly wrapped rectangular frame of the target entity. Object detection has applications in many computer vision fields such as image retrieval and video monitoring. Further, the accident visual recognition model is a neural network model, which is a classification model containing multiple task networks, and the model can map unknown category data to one of the given categories according to the characteristics of the data. In the first embodiment of the present application, the input data of the accident visual recognition model is a continuous video frame or a discrete video frame in a road traffic video, and the output result is a suspected accident area in the video frame, a suspected accident category of the suspected accident area, and a first confidence level of an accident in the suspected accident area. According to the general usage scenario, continuous video frames are usually used for analysis, and discrete or a certain video frame is not excluded for analysis. How to use the video frame specifically is related to the construction and training method of the specific accident visual recognition model.

[0089] In an optional embodiment of the present application, the accident visual recognition model can be obtained by training through the following steps S102-1 to S102-3:

[0090] Step S102-1, obtaining a pre-prepared road traffic video frame, wherein at least a portion of the road traffic video frame reflects an image of a road accident.

[0091] The pre-prepared road traffic video frames may be road accident video frames uploaded on the Internet, or may be video frames of historical road traffic videos captured by road traffic safety cameras, and this application does not impose any limitation on this.

[0092] Step S102-2, annotating the accident area and the accident type in the road traffic video frame by manual annotation; for the annotated accident area and the accident type corresponding to the accident area, the confidence level of the annotated content is considered to be 1.

[0093] Step S102-3: input the road traffic video frames and the annotated road traffic video frames as training samples into the initial accident vision model to train the initial accident vision model and obtain the accident vision recognition model.

[0094] In order to facilitate understanding of the input and output of the above accident visual recognition model, the model is explained here in the form of a specific scenario. First, assume that a traffic accident in which a vehicle rolls over occurs on Road A captured by road traffic safety camera A. At this time, after the video captured by camera A is input into the accident visual recognition model, the accident visual recognition model outputs the suspected area where the accident occurred (for example: outputs the area in the form of a detection frame), the category of the accident that occurred in the suspected area (the category is: vehicle rollover), and the confidence level of the accident that a vehicle rollover occurred in the area. The numerical expression of the confidence level can be set according to the actual situation, and this application does not impose any restrictions.

[0095] In the first embodiment of the present application, it is considered that the output results of the accident visual recognition model are all suspected accident recognition results. Therefore, in an optional embodiment of the present application, it is necessary to further test the suspected accident recognition results in the above-mentioned visual analysis stage. The detection specifically refers to setting a first confidence threshold. The first confidence threshold is used as a criterion for determining whether the accident visual recognition model can directly identify a certain road accident. It can be understood that in a preferred embodiment, since the accident visual recognition model outputs different categories of suspected accidents and corresponding confidence levels. Therefore, different first confidence thresholds can be set for different categories of suspected accidents. For example: when the confidence range is between 0-1, the confidence threshold for accidents such as vehicle rollovers is set to 0.7, and the confidence threshold for accidents such as traffic guardrail damage is set to 0.9.

[0096] In this step S102, the above-mentioned trained accident visual recognition model is used to perform image recognition on the video frames in the road traffic video obtained in step S101, and determine the suspected accident recognition results based on image recognition that appear in the road corresponding to the road traffic video, which at least include the suspected accident area and the first confidence level of the accident in the suspected accident area, and preferably may also include the category of the suspected accident. If it is judged that the first confidence level of the suspected accident area output by the accident recognition model is greater than the above-mentioned first confidence threshold - when the suspected accidents have been divided into different categories and different first confidence thresholds are set for different categories, then the first confidence level is greater than the first confidence threshold of the suspected road accident corresponding to the category, then the suspected accident area can be directly considered to be the accident area, that is, a road accident of the corresponding accident type has occurred in the suspected accident area output by the accident visual model.

[0097] Although there is the first confidence threshold, and when the confidence of a suspected road accident is determined to be greater than the first confidence threshold, it is determined that a road accident has occurred in the suspected accident area, there are still a large number of suspected accident areas whose first confidence is lower than the first confidence threshold. In order to improve the recognition accuracy, this embodiment combines the swarm intelligence analysis technology to further analyze and distinguish it; this is the swarm intelligence analysis stage of this embodiment. The swarm intelligence analysis stage includes the following steps S103 and S104.

[0098] Step S103, using the road traffic video training to obtain a corresponding swarm intelligence trajectory simulation model.

[0099] The swarm intelligence trajectory simulation model, in this embodiment, refers to an artificial intelligence model that mainly uses a swarm intelligence algorithm to analyze swarm behavior.

[0100] The so-called swarm intelligence algorithm (Swarm intelligence) originally mainly refers to the simulation of the group behavior of insects, animal flocks, bird flocks and fish schools. These groups search for food in a cooperative way, and each member of the group constantly changes the direction of the search by learning from its own experience and the experience of other members. The outstanding feature of the swarm intelligence optimization algorithm is to use the group wisdom of the population for collaborative search, so as to find the optimal solution in a short time. Common swarm intelligence optimization algorithms include ant colony algorithm, particle swarm optimization algorithm, bacterial colony optimization algorithm, frog leaping algorithm and artificial bee colony algorithm. In this application, the swarm intelligence algorithm is understood as a type of machine intelligence method that gathers group wisdom and collaboratively solves large-scale complex problems.

[0101] The swarm intelligent trajectory simulation model is a machine model for predicting the moving trajectory of a mobile group. The model predicts the future moving trajectory of the mobile group based on the current moving trajectory of the mobile group. In an embodiment of the present application, the swarm intelligent trajectory simulation model can be used to predict the moving trajectory of mobile individuals such as vehicles and pedestrians on the road. However, it should be noted that the swarm intelligent trajectory simulation model in the first embodiment of the present application does not focus on the output results of the model, but focuses on the training stage of the model and the internal logic parameters of the model.

[0102] Specifically, the swarm intelligence trajectory simulation model is obtained through the following steps S103 - 1 to S103 - 4 .

[0103] Step S103-1, extracting structured data of road traffic according to the road traffic video;

[0104] The structured data can reflect the structured data that can reflect the attribute information of the mobile individual in each continuous traffic video frame, and the structured data that reflects the attribute information of the traffic elements. The so-called structured data refers to data with a relatively regular data form that can be recorded using a two-dimensional table or the like. Most of the attribute information of the mobile individual and the attribute information of the traffic elements can use structured data to represent their specific attribute values.

[0105] The structured data reflecting the attribute information of the mobile individual includes: the position information, movement speed information, residence time information, etc. of the mobile individual. The position information of the mobile individual can be directly reflected in the video frame, and the movement speed information and residence time information can be obtained by calculating the time corresponding to the continuous video frames and the position of each mobile individual in the continuous video frames.

[0106] In an optional embodiment of the present application, if the mobile individual is a person, the structured data of the mobile individual may also include information such as the person's shape, height, and gait.

[0107] The structured data reflecting the attribute information of traffic elements include: category information of traffic elements, location information of traffic elements, status information of traffic elements, etc. For example: status information of traffic lights, meaning of temporary traffic signs (such as: no driving during construction), etc.

[0108] It can be understood that the above examples of structured data are only for the purpose of facilitating the understanding of the structured data, and the data types of the structured data do not only include the several types of data in the above examples.

[0109] Specifically, in order to extract the structured data of the road traffic, the road traffic video frame needs to be recognized first. In an optional implementation of the present application, the road traffic video frame is recognized by a pre-trained detection model and a pre-trained second classification model.

[0110] The pre-trained detection model is used to identify mobile individuals (e.g., vehicles, pedestrians, etc.) and traffic elements (e.g., lane lines, traffic lights) on the road. In an optional embodiment of the present application, the pre-trained detection model can be multiple, which are used to identify mobile groups and traffic elements respectively, so as to avoid the problem of unbalanced data quantity of different categories in the process of using the above data to train the subsequent group intelligent trajectory simulation model.

[0111] Based on the output results of the detection model, the second classification model subdivides the categories of mobile individuals and traffic elements in the road traffic video to obtain the categories of each mobile individual and each traffic element; for example, the classification of the identified vehicles (for example, the classification results are private cars, buses, muck trucks, commercial concrete trucks, etc.), and the classification of traffic elements (for example, the classification results are: traffic lights are red lights, traffic signs are straight ahead signs, etc.). Similar to the above-mentioned detection model, the second classification model can also be multiple, respectively used to identify the categories of different mobile groups and the categories of different traffic elements. For example, the second classification model 1 inputs the vehicle detection results of the detection model to classify private cars, muck trucks and other vehicle categories, and the second classification model 2 only inputs the detection results of traffic lights to classify the status of traffic lights, etc.

[0112] On the basis of obtaining the classification results of different categories of mobile groups and traffic elements output by the above-mentioned second classification model, the classification results are further screened according to preset rules. In the actual application process, the classification results can be screened by setting the threshold size of the recognition coil and the recognition wireframe, and the confidence threshold rules to eliminate misjudgment results.

[0113] While determining the categories of the mobile group and traffic elements and marking each mobile individual and traffic element in the mobile group, it is also necessary to determine the position of each mobile individual and traffic element. Specifically, the above position determination method can be determined based on the shooting position of the traffic video and the position of the mobile individual and traffic element in the video frame. Corresponding to the category of the mobile individual and the category of the traffic element, the attribute information of each mobile group individual and the attribute information of the traffic element are obtained; different categories of mobile individuals or traffic elements have different attribute information, so it is necessary to obtain the corresponding attribute information based on their categories.

[0114] Step S103 - 2 , accumulating the structured data of the road traffic until a predetermined time threshold is reached.

[0115] In an optional implementation of the present application, the structured data of the road traffic of the predetermined time threshold is sequentially stored in a simulation queue according to a time series relationship between video frames to form a series of simulation queue structured data.

[0116] Since the above process of extracting structured data is aimed at the video frames in the road traffic video, and in order to reflect the movement trajectory, the structured data in the continuous video frames for a period of time is required. Therefore, after the structured data has accumulated for a certain period of time, according to the time series relationship between the video frames, according to the specific categories of the mobile individuals or traffic elements therein, it is converted to the plane space converted according to the on-site situation, and the movement trajectory information of each mobile individual on the road and the change information of the traffic elements can be further obtained. For example: according to the position change of a certain vehicle individual in the video frame at different time points, the average moving speed, movement direction, retention time at the intersection or road and other parameters of the vehicle are obtained; for another example: after obtaining the changes in the traffic lights for a period of time, the duration information of the red, green and yellow lights of the road traffic lights can be obtained.

[0117] Step S103 - 3 , providing the structured data of the road traffic accumulated to the predetermined time threshold as training samples to an initial swarm intelligent trajectory simulation model, so as to train the initial swarm intelligent trajectory simulation model.

[0118] After determining the structured data within the predetermined time period, the structured data can clearly express the trajectory of the mobile group and the change information of traffic elements within the predetermined time period. For example, when the east-west traffic light is red for 30 seconds, the east-west traffic trajectory is stagnant. After using this structured data as a training sample to train the initial intelligent trajectory simulation model, the intelligent trajectory simulation model obtained has the ability to predict the trajectory of the mobile group in the current road traffic video.

[0119] In the first embodiment of the present application, the structured data corresponding to the attribute may include one or more of the following data: location information of the mobile individual, location information of the traffic element, category information of the mobile individual, category information of the traffic element, moving speed information of the mobile individual, and retention time information of the mobile individual;

[0120] The moving speed information and residence time of the moving individual are obtained by:

[0121] According to the position information of the mobile individual in each video frame of the road traffic video, combined with the time sequence relationship between the video frames, the moving speed information and the retention time information of the mobile individual are obtained. For example, according to the changes of traffic lights over a period of time, the reasonable retention time that a vehicle should have at the intersection is obtained, and another example is: according to the position of the vehicle at different times on the driving section, the average speed of the vehicle is obtained.

[0122] Furthermore, the samples represented by the structured data are obtained in the following way:

[0123] First, a regional road plan corresponding to the road photographed by the road traffic video is constructed.

[0124] The regional road plan map refers to a plan map that can reflect the road conditions. The regional road plan map can be converted from road video frames shot by road traffic video. Of course, it can also be obtained by other means, and this application does not impose any restrictions on this.

[0125] Secondly, according to each video frame in the road traffic video, identify and obtain the attribute information of the mobile individual therein (the specific content of the attribute information is expressed as structured data corresponding to the specific attribute of the mobile individual, and the subsequent attribute information and structured data have this relationship), as well as the attribute information reflecting the traffic elements (the specific content of the attribute information is expressed as structured data corresponding to the specific attribute of the traffic element); wherein, the attribute information of the mobile individual and the attribute information of the traffic elements at least include the location information of the mobile individual and the traffic elements.

[0126] Finally, the attribute information of the mobile individuals and the attribute information of the traffic elements in each video frame are mapped to the corresponding positions of the regional road plan to obtain a road mobile group trajectory map.

[0127] The road mobile group trajectory diagram is the training sample, which can be provided to the initial group intelligent trajectory simulation model for training.

[0128] Step S103 - 4 , using the swarm intelligence trajectory simulation model trained as above as the swarm intelligence trajectory simulation model.

[0129] The group intelligent trajectory simulation model obtained after training has the ability to predict the current moving trajectory of the mobile group on the road, that is, the content logic parameters of the model also have the correct logic for predicting the current moving trajectory of the mobile group.

[0130] In an optional embodiment of the present application, the intelligent trajectory simulation model can be implemented using a typical swarm intelligence algorithm model, such as an ant colony system algorithm model. By using the above-mentioned accumulated structured data for training, an intelligent trajectory simulation model that can simulate the trajectory of the current mobile group in the current scene can be obtained, and based on the continuously collected road traffic videos, the moving trajectory of the mobile individuals on the road and the changes in traffic elements are analyzed, including the changes in the trajectories of the various mobile individuals and different traffic elements that have been previously reflected by the road traffic videos, and also includes the estimation of the trajectory and changes of each mobile group and traffic element in the next time period.

[0131] Specifically, the intelligent trajectory simulation model can allocate a traffic coefficient to each position on the road plan of the current area (the default traffic coefficient of the passable section is 0, the traffic coefficient of the section with obstacles is 1, and the traffic coefficient of the section with steps that affect traffic is a decimal in the range of 0 to 1. In addition, the traffic coefficient of the intersection can be changed according to the state of the traffic light), initialize information heuristic factor, expected value heuristic factor, detour coefficient and other parameters.

[0132] After obtaining the starting point and terminal position information of each mobile individual in the video frame, the moving trajectory, moving speed, etc. of the mobile individual are estimated through the initial intelligent trajectory simulation model, and then compared with the moving trajectory and moving speed in the structured data. During this period, the initial group intelligent trajectory simulation model iteratively solves each learnable internal logic parameter in the model through a numerical optimization method until the number of iterations reaches the set upper limit or the difference between the estimated value of the initial intelligent trajectory simulation model and the value in the above structured data is less than the preset threshold. Among them, the learnable logic parameters include the above information heuristic factor, expected value heuristic factor, detour coefficient and other parameters as well as some traffic coefficients (for example: traffic coefficients in areas where accidents may occur on the road).

[0133] Step S104, obtaining a mobile group feature vector according to the swarm intelligence trajectory simulation model, providing the feature vector to a pre-trained classification model, and obtaining a suspected accident classification result based on swarm intelligence.

[0134] After obtaining the group intelligent trajectory simulation model, the mobile group feature vector can be obtained according to the group intelligent trajectory simulation model. The specific acquisition method can be, for example, to obtain the relevant mobile individual attributes and traffic element attributes according to the simulation results to form the mobile group feature vector. In this embodiment, the civil engineering solution is to extract the logical parameters of the group intelligent trajectory simulation model, use the logical parameters, or further use one or more of the moving speed information and the retention time information of each of the moving individuals in addition to the logical parameters to construct the feature vector of the mobile group. Here, the moving speed feature vector and the retention time feature vector of the mobile group are obtained through the moving speed information and the retention time information of the mobile group in step S103.

[0135] It can be understood that in step S103 of the present application, different mobile individuals in the mobile group have been classified into categories through the second classification model. Therefore, in step S104 of the present application, the moving speed and residence time information of the mobile group refers to the moving speed information and residence time information of mobile individuals of different categories.

[0136] After obtaining a feature vector composed of the logic parameter, the moving speed information and the residence time of the moving individual, the feature vector is input into a pre-trained classification model.

[0137] In the first embodiment of the present application, the classification model can extract the moving speed information and the retention time information in the input feature vector and combine it with the logical parameters of the model through the attention mechanism and other forms to obtain the classification result. Here, the classification result of the classification model is specifically the judgment result of whether an accident occurred on the road, and the confidence corresponding to the judgment result. Further, it can also include the category of suspected accidents. For example, if two similar vehicles in the video are in a long-term detention state at the same time, there is no major damage to the vehicle, and there are stranded individuals around the vehicle, then the classification result is that the two vehicles have a scratch accident, and the confidence of the accident is 0.9; for example: the two vehicles in the video are in an overlapping state, then the classification result is that the two vehicles have a collision accident, and the confidence of the accident is 0.9. It can be understood that the above examples are only for a clearer explanation of the function of the classification model of the present application and its input and output content, and are not used to limit the specific usage scenarios and specific content of the output of the present application.

[0138] After obtaining the above recognition results, a second confidence threshold is further set for different types of road accidents and their confidence levels output by the classification model.

[0139] The following introduces a method for obtaining the second confidence threshold in conjunction with step S105.

[0140] Step S105 , analyzing and determining the road accident in the road traffic video by combining the suspected accident identification result based on image recognition and the suspected accident classification result based on swarm intelligence.

[0141] The suspected accident identification result based on image recognition refers to the suspected accidents of different categories and the first confidence of the suspected accidents obtained in step S102. The suspected accident classification result based on swarm intelligence refers to the second confidence threshold set based on the output result of the first classification model in step S104.

[0142] Specifically, the above-mentioned step S105 means that by comparing the first confidence of the above-mentioned different categories of suspected accidents with the second confidence threshold corresponding to the category, if the first confidence is greater than the second confidence threshold, it is judged that the suspected accident area obtained in step S102 is the accident area, and a road accident has occurred in the road traffic video.

[0143] In the first embodiment of the present application, the second confidence threshold is determined according to the confidence of the road accident category output by the classification model. The second confidence threshold is actually a threshold set for the first confidence according to the group intelligence recognition result. The greater the possibility of a road accident determined by the group intelligence recognition result, the lower the second confidence threshold is set to, and the smaller the possibility of a road accident determined by the group intelligence recognition result, the higher the second confidence threshold is set to.

[0144] First, when the confidence level of the classification model output is relatively high, the second confidence threshold is set to be relatively small, and even in some extreme cases, the second confidence threshold can be set to 0. For example: assuming that the accident category output by the classification model is a vehicle rollover on the road, and the confidence level of the accident is 1 (i.e., the probability of a vehicle rollover is 100%). Then the second confidence level is set to 0, that is, no matter what the confidence level in the suspected accident identification result is, it is determined that a road accident has occurred in the road traffic video.

[0145] Secondly, when the confidence of the classification model output is small, the confidence threshold is set too high. For example, assuming that the accident category output by the classification model is a vehicle scratch, and the confidence of the accident is 0.2, the second confidence threshold can be set to 0.8, that is, only when the confidence of the road traffic accident in the suspected accident identification result is greater than 0.8, it is determined that a road accident has occurred in the road traffic video.

[0146] In another optional implementation of the present application, other methods different from the above-mentioned construction of the second confidence threshold may be used to determine whether the suspected accident area output by the accident visual recognition model is an accident area. For example: extract the logical parameters of the group intelligence trajectory model, and construct a mobile group feature vector based on the logical parameters and the trajectory information of the road mobile group; input the mobile group feature parameters into the third classification model to obtain the suspected accident classification result based on group intelligence output by the third classification model, wherein the suspected accident classification result output by the third classification model includes: whether the suspected accident occurred in the video frame, and the confidence level corresponding to the suspected accident, and may further include an analogy of the suspected accident.

[0147] If a video frame indicates that an accident of a certain category has occurred in the output of the third classification model, and there is also an area suspected of having occurred an accident of this category in the recognition result of the accident visual recognition model, it is confirmed that a road accident has occurred in the area.

[0148] Since the method steps described in the first embodiment of the present application are relatively large, Figure 2 The method described in the first embodiment of the present application is introduced. Figure 2 Another flowchart for identifying roads using swarm intelligence is provided for the first embodiment of the present application. The flowchart introduces the method using the image input module 101, the visual recognition module 102, the swarm intelligence analysis module 103, and the comprehensive judgment module 104 as the implementing bodies of the method steps.

[0149] The image input module 101 is specifically used to distribute the video frames of the road traffic video to the visual recognition module 102 and the swarm intelligence analysis module 103 respectively.

[0150] First, after receiving the video frames in sequence, the visual recognition module 102 executes steps S201 to S204.

[0151] Step S201, through the accident visual recognition model detection, obtain the suspected accident area in the video frame, the first confidence that the accident occurred in the suspected accident area, and the category of the suspected accident area.

[0152] The accident visual recognition model receives the video frames input by the image input module 101 in the time sequence of the video frames. Here, step S201 is basically similar to step S102 in the first embodiment of the present application. For relevant parts, please refer to the description of step S102 above, and the relevant parts will not be repeated.

[0153] Step S202, determining whether the first confidence level of the suspected accident area having an accident is greater than a preset confidence level threshold, if so, determining that a road accident has occurred in the suspected accident area, and executing step S203; if not, executing step S204;

[0154] Step S203: Send road accident alarm information to the road accident identification platform.

[0155] Step S204, sending the suspected accident area in the video frame, the first confidence of the accident in the suspected accident area, and the category of the suspected accident area to the comprehensive determination module 104 to further determine whether a road accident has occurred in the suspected accident area in the road.

[0156] Secondly, while the visual recognition module 102 is receiving the video frames, the swarm intelligence analysis module 103 is also receiving the video frames.

[0157] Specifically, after receiving the video frame, the swarm intelligence analysis module 103 executes step S301 to step S307.

[0158] Step S301 , receiving a video frame input by the image input module 101 .

[0159] Here, the swarm intelligence analysis module 103 receives the video frames input by the image input module 101 according to the time sequence of the video frames.

[0160] Step S302, extracting structured data of the road traffic video, and saving the structured data of the road traffic video to a simulation queue, wherein the structured data includes: the speed of the moving individual in the road traffic video, and the retention time of the moving individual.

[0161] Step S303 : sending the speed of the moving individual and the retention time of the moving individual in the road traffic video in the structured data to the comprehensive determination module 104 .

[0162] Step S304, determining whether the queue length of the simulation queue is greater than a preset time threshold, if not, then step S301, continuing to accept the video frame output by the image input module 101, if so, executing step S305.

[0163] In the embodiment of the present application, step S302 and step S304 are substantially the same as step S102 in the first embodiment of the present application, except that the structured data is stored in the simulation queue to ensure that the accumulation of the structured data reaches a preset time threshold. For related details, please refer to the description of step S102 above, which will not be repeated here.

[0164] Step S305: Use the simulated queue as a training sample to train an initial swarm intelligent traffic simulation model to obtain a swarm intelligent simulation model.

[0165] In the embodiment of the present application, step S305 is substantially the same as step S103 in the first embodiment of the present application. For the relevant details, please refer to the partial description of the above step S103, which will not be repeated here.

[0166] Step S306 , extracting the logic parameters of the swarm intelligence simulation model, and sending the logic parameters to the comprehensive determination module 104 .

[0167] Finally, the comprehensive determination module 104 executes the following steps S401 to S406 .

[0168] Step S401, receiving the speed of the mobile individual, the residence time of the mobile individual and the logic parameter sent by the swarm intelligence analysis module 103, and combining the speed of the mobile individual, the residence time of the mobile individual and the logic parameter into a feature vector.

[0169] Step S402: input the feature vector into a first classification model to obtain an accident classification result output by the first classification model.

[0170] Step S403: Determine a second confidence threshold for the suspected accident according to the accident classification result.

[0171] Step S404: receiving the suspected accident area, the first confidence level of the accident occurring in the suspected accident area, and the category of the suspected accident area sent by the visual recognition module 201.

[0172] Step S405, determining whether the first confidence level is greater than the second confidence level threshold; if so, executing step S406; if not, determining that no road accident has occurred in the suspected accident area.

[0173] Step S406: determine whether a road accident has occurred in the suspected accident area, and send road accident alarm information to the road accident identification platform.

[0174] In the above embodiment, a method for identifying roads using swarm intelligence is provided. Correspondingly, the second embodiment of the present application also provides a device for identifying roads using swarm intelligence. Please refer to Figure 3 , which is a schematic diagram of the structure of the device for identifying roads using swarm intelligence provided in the second embodiment of the present application. Since the device embodiment is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0175] The device for identifying roads using swarm intelligence provided by the present application includes:

[0176] An acquisition module 301 is used to acquire a road traffic video;

[0177] The recognition module 302 is used to perform image recognition on the video frames in the road traffic video to determine the suspected accident recognition result based on image recognition appearing on the road corresponding to the road traffic video;

[0178] A training module 303 is used to obtain a corresponding swarm intelligence trajectory simulation model using the road traffic video training;

[0179] The classification module 304 is used to obtain a mobile group feature vector according to the group intelligence trajectory simulation model, and provide it to a pre-trained classification model to obtain a suspected accident classification result based on group intelligence;

[0180] The analysis module 305 is used to analyze and determine the road accident in the road traffic video by combining the suspected accident identification result based on image recognition and the suspected accident classification result based on swarm intelligence.

[0181] As an embodiment, performing image recognition on the video frames in the road traffic video to determine a suspected accident recognition result based on image recognition appearing on the road corresponding to the road traffic video includes:

[0182] The video frames of the road traffic video are input into a trained accident visual recognition model to obtain a recognition result output by the accident visual recognition model, wherein the recognition result at least includes: a suspected accident area in the video frame, a first confidence level of an accident occurring in the suspected accident area, and may also include: a suspected accident category of the suspected accident area.

[0183] As an embodiment, it is characterized in that the method of obtaining a corresponding swarm intelligence trajectory simulation model by using the road traffic video training includes:

[0184] Extracting structured data of road traffic based on the road traffic video;

[0185] Accumulating the structured data of the road traffic until a predetermined time threshold is reached;

[0186] providing the structured data of the road traffic accumulated to the predetermined time threshold as training samples to an initial swarm intelligent trajectory simulation model, and training the initial swarm intelligent trajectory simulation model;

[0187] The swarm intelligence trajectory simulation model trained as above is used as the swarm intelligence trajectory simulation model.

[0188] As an embodiment, providing the structured data of the road traffic accumulated to the predetermined time threshold as a training sample to an initial swarm intelligence trajectory simulation model includes:

[0189] Constructing a regional road plan corresponding to the road captured by the road traffic video;

[0190] According to each video frame in the road traffic video, identifying and acquiring attribute information of a mobile individual and attribute information of a traffic element therein; wherein the attribute information of the mobile individual and the attribute information of the traffic element at least include location information of the mobile individual and the traffic element;

[0191] Mapping the attribute information of the mobile individuals and the attribute information of the traffic elements in each video frame to the corresponding positions of the regional road plan to obtain a road mobile group trajectory map;

[0192] The mobile group trajectory diagram is used as the training sample and provided to an initial group intelligent trajectory simulation model.

[0193] As an embodiment, the identifying the attribute information of the mobile group and the attribute information of the traffic elements in each image of the road traffic video includes:

[0194] Using a pre-trained detection model, identifying moving individuals and traffic elements contained in the road traffic video;

[0195] Using a pre-trained second classification model, classifying the moving individuals and traffic elements in the road traffic video to obtain the category of each moving individual and the category of each traffic element;

[0196] Corresponding to the categories of the mobile individuals and the categories of the traffic elements, attribute information of each of the mobile individuals and attribute information of the traffic elements are obtained accordingly.

[0197] As an embodiment, the attribute information further includes one or more of the following information: category information of the mobile individual, category information of the traffic element, moving speed information of the mobile individual, and retention time information of the mobile individual;

[0198] The moving speed information and the retention time information of the moving individual are obtained by:

[0199] According to the position information of each moving individual reflected in each video frame of the road traffic video, combined with the temporal relationship between the video frames, the moving speed information of each moving individual and the residence time information of the moving individual are obtained.

[0200] As an embodiment, obtaining a mobile group feature vector according to the group intelligence trajectory simulation model and providing it to a pre-trained classification model to obtain a suspected accident classification result based on group intelligence includes:

[0201] Extracting the logic parameters of the group intelligent trajectory model, using the logic parameters, or further using one or more of the moving speed information and the retention time information of each of the moving individuals in addition to the logic parameters, to construct a moving group feature vector;

[0202] The mobile group feature vector is input into the classification model to obtain a suspected accident classification result output by the classification model.

[0203] As an embodiment, the combining the suspected accident identification result based on image recognition and the suspected accident classification result based on swarm intelligence to analyze and determine the road accident in the road traffic video includes:

[0204] Determining a second confidence threshold for the suspected accident according to the suspected accident classification result output by the classification model;

[0205] It is determined whether the first confidence level of the accident occurring in the suspected accident area output by the accident visual model is greater than the second confidence level threshold; if so, it is determined that a road accident has occurred in the road video.

[0206] As an embodiment, the obtaining of the mobile group feature vector including the logical parameters of the group intelligence trajectory simulation model and providing the vector to a pre-trained classification model to obtain a suspected accident classification result based on group intelligence includes:

[0207] Extracting the logical parameters of the group intelligent trajectory model, and constructing a mobile group feature vector according to the logical parameters and the trajectory information of each road moving individual;

[0208] The mobile group feature vector is input into a third classification model to obtain a suspected accident classification result based on swarm intelligence output by the third classification model, wherein the suspected accident classification result output by the third classification model includes: a judgment result of whether a suspected accident has occurred in a video frame, and a confidence level corresponding to the judgment result.

[0209] As an embodiment, combining the suspected accident identification result and the suspected accident classification result based on swarm intelligence to analyze and determine the road accident in the road traffic video includes:

[0210] If a certain video frame is judged as suspected of having an accident in the output of the third classification model, and the video frame also includes a suspected accident area in the recognition result of the accident visual recognition model, it is confirmed that a road accident has occurred in the area.

[0211] As an embodiment, the method further includes: determining whether a first confidence level that an accident has occurred in the suspected accident area is greater than a preset first confidence level threshold; if so, determining that a road accident has occurred in the road video.

[0212] Corresponding to the first and second embodiments described above, the third embodiment of the present application further provides a road video device, which obtains a road video display interface through the method described in the first embodiment or the device described in the second embodiment, and displays the road video. Figure 4 The road video display method is introduced, wherein: Figure 4 This is a schematic diagram of the display interface of the road video display device provided in the third embodiment of the present application. Figure 4 The road video display method provided in the embodiment of the present application is introduced.

[0213] Please refer to Figure 4 The road video display device is specifically applied to the video display terminal. The road display interface includes:

[0214] Real-time road traffic video 501 , mobile group trajectory plan 602 corresponding to the real-time road traffic video, and road accident analysis result display area 502 .

[0215] The real-time road traffic video 501 is specifically obtained by real-time shooting by a road traffic safety camera. In an optional implementation of the present application, the video screen of the real-time road traffic video includes at least one of the following information:

[0216] A location box marking the location of the suspected accident area;

[0217] The average speed and residence time information of each moving individual in the video screen;

[0218] The moving trajectory information of each moving individual in the video screen.

[0219] The mobile group trajectory plan view 502 corresponding to the real-time road traffic video is specifically the road mobile group trajectory map in step S103 - 3 of the first embodiment of the present application.

[0220] Specifically, the moving group trajectory plane map corresponding to the road traffic video is obtained by the following method:

[0221] Constructing a regional road plan corresponding to the road captured by the road traffic video;

[0222] Identify the attribute information of the mobile individuals and the attribute information of the traffic elements in each video frame of the road traffic video, and determine the structured data of each element in the road traffic video according to the attribute information of the mobile individuals and the attribute information of the traffic elements; wherein the attribute information of the mobile individuals and the traffic element information at least include the location information of the mobile individuals and the traffic elements;

[0223] The attribute information of the mobile individuals and the attribute information of the traffic elements in each image within the preset time threshold are mapped to the corresponding positions of the regional road plan to obtain a road mobile group trajectory map.

[0224] The road accident analysis result is the analysis result of the road accident in the road traffic video in step S105 of the first embodiment of the present application, that is, the analysis result based on the swarm intelligence trajectory simulation model and the recognition result of image recognition of the video frames in the road traffic video, which displays the real-time traffic status information of the road.

[0225] Specifically, the road traffic status information also includes: confidence information of the road accident information.

[0226] Corresponding to the first and second embodiments of the present application, the fourth embodiment of the present application also provides an electronic device. Since the electronic device embodiment is basically similar to the above-mentioned first and second embodiments, the description is relatively simple. For relevant matters, please refer to the partial description of the above-mentioned first and second embodiments. The electronic device embodiment described below is merely illustrative.

[0227] Please refer to Figure 5 , which is a schematic diagram of the structure of an electronic device provided in the fourth embodiment of the present application.

[0228] The electronic device comprises: a processor 601;

[0229] The memory 602 is used to store a method program. When the program is read and executed by the processor, the method described in the first embodiment is executed.

[0230] The fifth embodiment of the present application also provides a computer storage medium. Since the storage medium embodiment is basically similar to the first and second embodiments mentioned above, the description is relatively simple. For related matters, please refer to the partial description of the first and second embodiments mentioned above. The computer storage medium embodiment described below is merely illustrative.

[0231] The computer storage medium stores a computer program, and when the program is executed, the method described in the first embodiment is implemented.

[0232] It should be noted that the detailed description of the storage medium provided in the fifth embodiment of the present application can refer to the relevant description of the above-mentioned method embodiment provided in the present application, and will not be repeated here.

[0233] Although the present application is disclosed as above in the form of a preferred embodiment, it is not intended to limit the present application. Any technical personnel in this field may make possible changes and modifications without departing from the spirit and scope of the present application. Therefore, the scope of protection of the present application shall be based on the scope defined by the claims of the present application.

[0234] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0235] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0236] 1. Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined in this article, computer-readable media does not include non-transitor7 media such as modulated data signals and carrier waves.

[0237] 2. Those skilled in the art should understand that the embodiments of the present application can be provided as a system or electronic device. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

Claims

1. A method for identifying road accidents using swarm intelligence, characterized in that: include: Obtain road traffic videos; Performing image recognition on video frames in the road traffic video to determine a suspected accident recognition result based on image recognition that occurs on a road corresponding to the road traffic video; Using the road traffic video training to obtain a corresponding swarm intelligence trajectory simulation model; According to the swarm intelligence trajectory simulation model, a mobile group feature vector is obtained, and provided to a pre-trained classification model to obtain a suspected accident classification result based on swarm intelligence; Determining a second confidence threshold for the suspected accident according to the suspected accident classification result output by the classification model; Determine whether the first confidence level of the suspected accident area output by the accident visual recognition model is greater than the second confidence level threshold; if so, determine that a road accident has occurred in the road traffic video.

2. The method for identifying road accidents using swarm intelligence according to claim 1, characterized in that: The performing image recognition on the video frames in the road traffic video to determine the suspected accident recognition result based on image recognition appearing on the road corresponding to the road traffic video includes: The video frames of the road traffic video are input into a trained accident visual recognition model to obtain a recognition result output by the accident visual recognition model, wherein the recognition result at least includes: a suspected accident area in the video frame, a first confidence level of an accident occurring in the suspected accident area, and also includes: a suspected accident category of the suspected accident area.

3. The method for identifying road accidents using swarm intelligence according to claim 1, characterized in that: The method of obtaining a corresponding swarm intelligence trajectory simulation model by using the road traffic video training includes: Extracting structured data of road traffic based on the road traffic video; Accumulating the structured data of the road traffic until a predetermined time threshold is reached; providing the structured data of the road traffic accumulated to the predetermined time threshold as training samples to an initial swarm intelligent trajectory simulation model, and training the initial swarm intelligent trajectory simulation model; The swarm intelligence trajectory simulation model trained as above is used as the swarm intelligence trajectory simulation model.

4. The method for identifying road accidents using swarm intelligence according to claim 3, characterized in that: The step of providing the structured data of the road traffic accumulated to the predetermined time threshold as training samples to an initial swarm intelligence trajectory simulation model includes: Constructing a regional road plan corresponding to the road captured by the road traffic video; According to each video frame in the road traffic video, identifying and acquiring attribute information of a mobile individual and attribute information of a traffic element therein; wherein the attribute information of the mobile individual and the attribute information of the traffic element at least include location information of the mobile individual and the traffic element; Mapping the attribute information of the mobile individuals and the attribute information of the traffic elements in each video frame to the corresponding positions of the regional road plan to obtain a road mobile group trajectory map; The mobile group trajectory diagram is used as the training sample and provided to an initial group intelligent trajectory simulation model.

5. The method for identifying road accidents using swarm intelligence according to claim 4, characterized in that: The identifying and obtaining the attribute information of the mobile group and the attribute information of the traffic elements in each video frame of the road traffic video includes: Using a pre-trained detection model, identifying moving individuals and traffic elements contained in the road traffic video; Using a pre-trained second classification model, classifying the moving individuals and traffic elements in the road traffic video to obtain the category of each moving individual and the category of each traffic element; Corresponding to the categories of the mobile individuals and the categories of the traffic elements, attribute information of each of the mobile individuals and attribute information of the traffic elements are obtained accordingly.

6. The method for identifying road accidents using swarm intelligence according to claim 4, characterized in that: The attribute information further includes one or more of the following information: category information of the mobile individual, category information of the traffic element, moving speed information of the mobile individual, and retention time information of the mobile individual; The moving speed information and residence time of the moving individual are obtained by: According to the position information of each moving individual reflected in each video frame of the road traffic video, combined with the temporal relationship between the video frames, the moving speed information of each moving individual and the residence time information of the moving individual are obtained.

7. The method for identifying road accidents using swarm intelligence according to claim 4, characterized in that: The method of obtaining a mobile group feature vector according to the group intelligence trajectory simulation model and providing it to a pre-trained classification model to obtain a suspected accident classification result based on group intelligence includes: Extracting the logic parameters of the group intelligent trajectory simulation model, using the logic parameters, or further using one or more of the moving speed information and the retention time information of each of the moving individuals in addition to the logic parameters, to construct a moving group feature vector; The mobile group feature vector is input into the classification model to obtain a suspected accident classification result output by the classification model.

8. The method for identifying road accidents using swarm intelligence according to claim 7, characterized in that: The suspected accident classification result output by the classification model includes: a determination result of whether an accident has occurred on the road, and a confidence level corresponding to the determination result, and also includes: a category of the suspected road accident.

9. The method for identifying road accidents using swarm intelligence according to claim 8, characterized in that: The combining the suspected accident identification result and the suspected accident classification result based on swarm intelligence to analyze and determine the road accident in the road traffic video includes: If a certain video frame is judged as suspected of having an accident in the output of the classification model, and the suspected accident area is also included in the recognition result of the accident visual recognition model, it is confirmed that a road accident has occurred in the area reflected by the video frame.

10. The method for identifying road accidents using swarm intelligence according to claim 2, characterized in that: The method further comprises: It is determined whether the first confidence level of an accident occurring in the suspected accident area is greater than a preset first confidence level threshold; if so, it is determined that a road accident has occurred in the road traffic video.

11. A device for identifying road accidents using swarm intelligence, characterized in that: include: An acquisition module, used for acquiring road traffic videos; A recognition module, configured to perform image recognition on video frames in the road traffic video, and determine a suspected accident recognition result based on image recognition that occurs on a road corresponding to the road traffic video; A training module, used to obtain a corresponding swarm intelligence trajectory simulation model using the road traffic video training; A classification module, used to obtain a mobile group feature vector according to the group intelligence trajectory simulation model, and provide it to a pre-trained classification model to obtain a suspected accident classification result based on group intelligence; An analysis module, used to analyze and determine the road accident in the road traffic video by combining the suspected accident identification result based on image recognition and the suspected accident classification result based on swarm intelligence; The analysis module is specifically used to determine a second confidence threshold for the suspected accident based on the suspected accident classification result output by the classification model; determine whether the first confidence of the accident occurring in the suspected accident area output by the accident visual recognition model is greater than the second confidence threshold; if so, determine that a road accident has occurred in the road traffic video.

12. An electronic device, characterized in that: include: processor; A memory for storing a method program, wherein when the program is read and executed by the processor, the following steps are performed: Obtain road traffic videos; Performing image recognition on video frames in the road traffic video to determine a suspected accident recognition result based on image recognition that occurs on a road corresponding to the road traffic video; Using the road traffic video training to obtain a corresponding swarm intelligence trajectory simulation model; According to the swarm intelligence trajectory simulation model, a mobile group feature vector is obtained, and provided to a pre-trained classification model to obtain a suspected accident classification result based on swarm intelligence; Determining a second confidence threshold for the suspected accident according to the suspected accident classification result output by the classification model; Determine whether the first confidence level of the suspected accident area output by the accident visual recognition model is greater than the second confidence level threshold; if so, determine that a road accident has occurred in the road traffic video.

13. A computer storage medium, characterized in that: The computer storage medium stores a computer program, and when the program is executed, the following steps are implemented: Obtain road traffic videos; Performing image recognition on video frames in the road traffic video to determine a suspected accident recognition result based on image recognition that occurs on a road corresponding to the road traffic video; Using the road traffic video training to obtain a corresponding swarm intelligence trajectory simulation model; According to the swarm intelligence trajectory simulation model, a mobile group feature vector is obtained, and provided to a pre-trained classification model to obtain a suspected accident classification result based on swarm intelligence; Determining a second confidence threshold for the suspected accident according to the suspected accident classification result output by the classification model; Determine whether the first confidence level of the suspected accident area output by the accident visual recognition model is greater than the second confidence level threshold; if so, determine that a road accident has occurred in the road traffic video.

14. A road video display device, characterized in that: include: Displaying a road display interface at the video display terminal, wherein the road display interface includes: a real-time road traffic video, a moving group trajectory plane map corresponding to the real-time road traffic video, and a road accident analysis result display area; The road accident analysis result display area displays real-time traffic status information of the road based on the analysis results of the swarm intelligence trajectory simulation model and the recognition results of image recognition of video frames in the road traffic video, wherein the road traffic status information includes: road accident information; the swarm intelligence trajectory simulation model is obtained by training with road traffic videos; the swarm intelligence trajectory simulation model is used to obtain mobile group feature vectors and provide them to a pre-trained classification model to obtain suspected accident classification results based on swarm intelligence; the suspected accident classification results are used to determine a second confidence threshold for suspected accidents; the recognition results of image recognition of video frames in the road traffic video include a first confidence level of an accident occurring in a suspected accident area output by an accident visual recognition model; when the first confidence level of an accident occurring in a suspected accident area output by the accident visual recognition model is greater than the second confidence threshold, a road accident occurs in the road traffic video.

Images