Information generation method, device, electronic device and computer readable medium

Through the frame extraction processing and optical flow information analysis of road camera video, the problem of low vehicle abnormal detection efficiency during peak traffic is solved, and accurate identification and detection of abnormally driven vehicles is achieved.

CN118314535BActive Publication Date: 2024-11-12ADDX (BEIJING) TECH CO LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202410573672.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-05-10
Publication Date
2024-11-12
Estimated Expiration
2044-05-10

AI Technical Summary

Technical Problem

During peak traffic periods, road shooting videos require shooting and amplification of a large number of vehicles to judge abnormal driving conditions, resulting in large workloads, low efficiency and insufficient detection accuracy.

Method used

By obtaining the road camera video set, extracting frame processing to generate a road shooting image sequence, extracting vehicle object information, filtering candidate abnormal driving vehicles, and generating optical flow information sequences to determine the vehicle's driving route and intention, and then judging abnormal driving information.

Benefits of technology

Accurate and efficient detection of abnormally driven vehicles is achieved, and the detection accuracy and work efficiency are improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118314535B_ABST
    Figure CN118314535B_ABST
Patent Text Reader

Abstract

The embodiments of the present disclosure disclose information generation methods, devices, electronic devices, and computer-readable media. A specific implementation of the method includes: obtaining a road camera video set; determining a vehicle object information set; for each road camera video, executing a first generation step: performing frame extraction processing on the road camera video to generate a road shooting image sequence; generating at least one vehicle driving abnormality information; screening out at least one candidate abnormal driving vehicle object information; for each candidate abnormal driving vehicle object information, executing a second generation step: screening out a vehicle driving image frame sequence set corresponding to the candidate abnormal driving vehicle object information; for each vehicle driving image frame sequence, generating an optical flow information sequence; generating vehicle driving route information and vehicle driving intention information; generating abnormal driving information. This implementation can accurately and efficiently determine the vehicle object information of abnormal driving.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technology, and in particular to an information generation method, device, electronic device, and computer-readable medium. Background Art

[0002] At present, the detection of dangerous driving vehicles is still the focus of road driving monitoring. For the abnormal detection of driving vehicles, the method usually adopted is: through the method of comparing road shooting videos, manually watching the abnormal driving conditions of related vehicles.

[0003] However, when the above method is used to detect abnormalities in driving vehicles, the following technical problems often occur:

[0004] During peak traffic hours, road videos are often shot of a large number of vehicles, and during viewing, the vehicle needs to be zoomed in to determine the driving details of the vehicle. This results in a huge workload for judging abnormal driving of vehicles, low work efficiency, and inability to effectively guarantee the accuracy of abnormal driving detection.

[0005] The above information disclosed in this Background section is only for enhancement of understanding of the background of the inventive concept and therefore it may contain information that does not form the prior art that is already known in this country to a person of ordinary skill in the art. Summary of the invention

[0006] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0007] Some embodiments of the present disclosure propose information generation methods, devices, electronic devices, and computer-readable media to solve one or more of the technical problems mentioned in the above background technology section.

[0008] In a first aspect, some embodiments of the present disclosure provide an information generation method, comprising: obtaining a road camera video set corresponding to a road camera device set, wherein the road camera video set is a video set shot at a target road intersection; determining a vehicle object information set corresponding to the road camera video set; for each road camera video in the road camera video set, executing the following first generation step: performing frame extraction processing on the road camera video to generate a road shot image sequence; generating at least one vehicle driving abnormality information corresponding to at least one vehicle object information according to the road shot image sequence, wherein the vehicle object information set includes: the at least one vehicle object information information; according to the vehicle driving abnormality information group corresponding to each vehicle object information, at least one candidate abnormal driving vehicle object information is screened out from the above vehicle object information set; for each candidate abnormal driving vehicle object information in the above at least one candidate abnormal driving vehicle object information, the following second generation step is performed: generating an optical flow information sequence set for the above candidate abnormal driving vehicle object information; based on the above optical flow information sequence set, generating vehicle driving route information and vehicle driving intention information corresponding to the above candidate abnormal driving vehicle object information; based on the above vehicle driving route information and the above vehicle driving intention information, generating abnormal driving information for the above candidate abnormal driving vehicle object information.

[0009] In a second aspect, some embodiments of the present disclosure provide an information generating device, comprising: an acquiring unit, configured to acquire a road camera video set corresponding to a road camera device set, wherein the road camera video set is a video set shot at a target road intersection; a determining unit, configured to determine a vehicle object information set corresponding to the road camera video set; a first generating unit, configured to perform the following first generating step for each road camera video in the road camera video set: performing frame extraction processing on the road camera video to generate a road shooting image sequence; generating at least one vehicle driving abnormality information corresponding to at least one vehicle object information according to the road shooting image sequence, wherein the vehicle object information set includes: the at least one Vehicle object information; a screening unit, configured to screen out at least one candidate abnormal driving vehicle object information from the above vehicle object information set according to the vehicle driving abnormality information group corresponding to each vehicle object information; a second generating unit, configured to perform the following second generating step for each candidate abnormal driving vehicle object information in the above at least one candidate abnormal driving vehicle object information: generating an optical flow information sequence set for the above candidate abnormal driving vehicle object information; generating vehicle driving route information and vehicle driving intention information corresponding to the above candidate abnormal driving vehicle object information according to the above optical flow information sequence set; generating abnormal driving information for the above candidate abnormal driving vehicle object information according to the above vehicle driving route information and the above vehicle driving intention information.

[0010] In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner in the first aspect.

[0011] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method described in any implementation manner in the first aspect is implemented.

[0012] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: the information generation method of some embodiments of the present disclosure can accurately and efficiently determine the vehicle object information of abnormal driving. Specifically, the reason why the relevant abnormal driving results are not accurate and efficient is that during the peak traffic period, the road shooting video is often a driving video shot for a large number of vehicles, and during the viewing process, the vehicle needs to be enlarged to determine the driving details of the vehicle. The workload of judging abnormal driving of the vehicle is extremely large, the work efficiency is low, and the accuracy of abnormal driving detection cannot be effectively guaranteed. Based on this, the information generation method of some embodiments of the present disclosure first obtains the road camera video set corresponding to the road camera device set as a data basis to facilitate the subsequent determination of abnormal driving vehicles. Among them, the above-mentioned road camera video set is a video set shot for the target road intersection. Then, the vehicle object information set corresponding to the above-mentioned road camera video set is determined, so as to facilitate all-round detection of each vehicle object information and ensure the comprehensiveness of abnormal driving detection. Then, for each road camera video in the above-mentioned road camera video set, the following first generation step is performed: the first step is to perform frame extraction processing on the above-mentioned road camera video to generate a road shooting image sequence, so as to facilitate the subsequent analysis of vehicles with abnormal driving. The second step is to accurately generate at least one vehicle driving abnormality information corresponding to at least one vehicle object information based on the above-mentioned road shooting image sequence, so as to be used for the subsequent determination of the corresponding vehicle object with abnormal driving. Among them, the above-mentioned vehicle object information set includes: the above-mentioned at least one vehicle object information. Furthermore, according to the vehicle driving abnormality information group corresponding to each vehicle object information, at least one candidate abnormal driving vehicle object information that may have abnormal driving behavior can be accurately preliminarily screened out from the above-mentioned vehicle object information set. Finally, for each candidate abnormal driving vehicle object information in the above-mentioned at least one candidate abnormal driving vehicle object information, the following second generation step is performed: the first step is to accurately generate an optical flow information sequence set for the above-mentioned candidate abnormal driving vehicle object information, so as to facilitate the subsequent determination of the vehicle driving route information and the vehicle driving intention information. In the second step, based on the optical flow information sequence set, the vehicle driving route information and vehicle driving intention information corresponding to the above-mentioned candidate abnormal driving vehicle object information can be accurately generated, so as to subsequently use the vehicle driving route information and vehicle driving intention information to accurately further determine whether the object corresponding to the candidate abnormal driving vehicle object information is driving abnormally. In the third step, based on the above-mentioned vehicle driving route information and the above-mentioned vehicle driving intention information, abnormal driving information for the above-mentioned candidate abnormal driving vehicle object information can be accurately generated. In summary, through the two-stage determination of abnormal driving vehicle objects, and the generation of vehicle driving route information and vehicle driving intention, vehicle objects with abnormal driving behavior can be accurately determined. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0014] Figure 1 are flow charts of some embodiments of the information generation method according to the present disclosure;

[0015] Figure 2 is a schematic diagram of the structure of some embodiments of the information generating device according to the present disclosure;

[0016] Figure 3 It is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0017] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0018] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0019] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0020] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0021] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0022] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0023] refer to Figure 1, shows a process 100 of some embodiments of the information generation method according to the present disclosure. The information generation method comprises the following steps:

[0024] Step 101: Obtain a road camera video set corresponding to a road camera device set.

[0025] In some embodiments, the execution subject of the above information generation method can obtain the road camera video set corresponding to the road camera device set through a wired connection method or a wireless connection method. The above road camera video set is a video set shot at a target road intersection. The road camera device can be a camera device set at the target road intersection for supervising vehicle driving. There is a one-to-one correspondence between the road camera devices in the road camera device set and the road shot videos in the road camera video set.

[0026] Step 102: determine the vehicle object information set corresponding to the road camera video set.

[0027] In some embodiments, the execution subject may determine a vehicle object information set corresponding to the road camera video set. The vehicle object information may be a vehicle identification, representing vehicle identity information. The vehicle object information set may appear in each vehicle object information in the road camera video set.

[0028] In some optional implementations of some embodiments, the determining of the vehicle object information set corresponding to the road camera video set may include the following steps:

[0029] In the first step, for each road camera video in the road camera video set, the following third generation step is performed:

[0030] Sub-step 1: determining a road shooting image sequence corresponding to the road shooting video as a target road shooting image sequence.

[0031] As an example, the execution subject may perform frame extraction processing on the road shooting video to generate a target road shooting image sequence.

[0032] Sub-step 2: input each road shot image in the target road shot image sequence into a pre-trained object feature extraction model to generate an object feature extraction information set. The object feature extraction information may represent feature information corresponding to vehicle object information in the road shot image. The object feature extraction model may be a neural network model for extracting object feature information. For example, the object feature extraction model may be a multi-layer series convolutional neural network.

[0033] Sub-step 3, clustering the individual object feature extraction information in the obtained object feature extraction information set sequence to generate an object feature extraction information cluster set. The feature similarity between the individual object feature extraction information in each object feature extraction information cluster is higher than the target value. The individual object feature extraction information in the object feature extraction information cluster has similar feature information for the vehicle object.

[0034] Sub-step 4: verify each object feature extraction information cluster in the above object feature extraction information cluster set to generate a verification result. The verification result may include: a result indicating that the verification is correct and a result indicating that the verification is incorrect.

[0035] As an example, the execution entity may perform a time arithmetic check on each object feature extraction information in each object feature extraction information cluster to determine whether the time difference corresponding to each object feature extraction information exists in the form of an arithmetic progression to generate a check result.

[0036] Sub-step 5, in response to determining that the verification result indicates that the verification is correct, for each object feature extraction information cluster in the object feature extraction information cluster set, executing the following fourth generation step:

[0037] The first sub-step is to determine each target road shooting image sub-sequence corresponding to each object feature extraction information in the object feature extraction information cluster, wherein the object feature extraction information in each object feature extraction information and the target road shooting image sub-sequence in each target road shooting image sub-sequence have a one-to-one correspondence.

[0038] The second sub-step is to generate corresponding at least one vehicle object information according to each of the target road captured image sub-sequences.

[0039] As an example, the execution subject may input each target road shot image subsequence in each target road shot image subsequence into a pre-trained vehicle object detection information generation model to generate vehicle object detection information, and obtain at least one vehicle object detection information as at least one vehicle object information. In practice, the vehicle object detection information generation model may be a multi-layer serial convolutional neural network model.

[0040] In the second step, the obtained multiple vehicle object information are determined as the above-mentioned vehicle object information set.

[0041] In the process of adopting technical solutions to solve the above technical problems, the following technical problem is often accompanied: the inability to accurately and effectively extract the corresponding features of the vehicle object, resulting in the subsequent determination of the vehicle object information is not accurate enough.

[0042] In some optional implementations of some embodiments, the object feature extraction model includes: a vehicle object segmentation model, a vehicle object license plate extraction model, a vehicle object color extraction model, a vehicle object model extraction model, and a vehicle object overall extraction model. The vehicle object segmentation model may be a pre-trained neural network model for performing image segmentation on a vehicle object. For example, the vehicle object segmentation model may be a U-net model. The vehicle object license plate extraction model may be a pre-trained neural network model for generating a vehicle object license plate. In practice, the vehicle object license plate extraction model may be a classification model based on a convolutional neural network model. The vehicle object color extraction model may be a neural network model for generating the appearance color of a vehicle object. In practice, the vehicle object color extraction model may be a multi-layer series convolutional neural network model. The vehicle object overall extraction model may be a neural network model for extracting overall feature information of an image corresponding to a vehicle object. In practice, the vehicle object overall extraction model may be a pre-trained model. The vehicle object model extraction model may be a neural network model for generating vehicle object model information. In practice, the vehicle object model information may include: vehicle object model information and vehicle object model probability. The vehicle object model information may be a model model corresponding to a vehicle object. For example, the vehicle object model information may be model A of brand A. The vehicle object model probability may be a probability that the vehicle model corresponding to the vehicle object is the vehicle object model information. The greater the vehicle object model probability, the higher the probability that the vehicle model corresponding to the vehicle object is the vehicle object model information. In practice, the vehicle object model extraction model may be an attention mechanism model based on a model association table. The attention mechanism model may be a neural network model based on a convolutional neural network model that focuses on key feature points of the vehicle model.

[0043] Optionally, the step of inputting the road photographed image into a pre-trained object feature extraction model to generate an object feature extraction information set may include the following steps:

[0044] The first step is to input the road shot image into the vehicle object segmentation model to generate a vehicle object segmentation information set, wherein the vehicle object segmentation information may be coordinate range information representing the object content corresponding to the vehicle object in the road shot image.

[0045] The second step is to determine the vehicle object segmentation image set corresponding to the vehicle object segmentation information set. There is a one-to-one correspondence between the vehicle object segmentation information in the vehicle object segmentation information set and the vehicle object segmentation image in the vehicle object segmentation image set. The vehicle object segmentation image may be an image whose main body is the corresponding vehicle object.

[0046] As an example, first, the execution subject may expand the range information set corresponding to the vehicle object segmentation information set to generate an extended range information set. Then, the road images are segmented according to the extended range information to generate a segmented image set as the vehicle object segmentation image set.

[0047] The third step is to perform the following fifth generation step for each vehicle object segmentation image in the vehicle object segmentation image set:

[0048] Sub-step 1: input the vehicle object segmentation image into the vehicle object license plate extraction model to generate vehicle object license plate information.

[0049] Sub-step 2: inputting the vehicle object segmentation image into the vehicle object color extraction model to generate vehicle object color information.

[0050] Sub-step 3: input the vehicle object segmentation image into the vehicle object model extraction model to generate vehicle object model information. The vehicle object model information includes: vehicle object model information and vehicle object model probability.

[0051] Sub-step 4: inputting the above-mentioned vehicle object segmentation image into the above-mentioned vehicle object overall extraction model to generate vehicle object overall extraction information.

[0052] Sub-step 5, determining the degree of completeness of the license plate corresponding to the above vehicle object license plate information. The degree of completeness of the license plate can represent the degree of completeness of the license plate number of the recognizable vehicle object license plate information. The higher the degree of completeness of the license plate, the more complete the recognized license plate number is.

[0053] Sub-step 6, respectively encode the vehicle object license plate information, the vehicle object color information, and at least one target vehicle object model information in the vehicle object model information set to generate vehicle object license plate feature information, vehicle object color feature information, and at least one target vehicle object model feature information, wherein the vehicle object model probability corresponding to the target vehicle object model information is greater than a predetermined probability value. For example, the predetermined probability value may be 70%. The encoding process may be a vector conversion process. Specifically, it may be implemented by an encoding neural network model.

[0054] Sub-step 7, concatenate the vehicle object license plate feature information and the license plate integrity corresponding feature information to generate first concatenated feature information. The license plate integrity corresponding feature information can be vector information that represents the semantics of the vehicle integrity corresponding degree. The license plate object license plate feature information can be vector information. In practice, the generation of the license plate integrity corresponding feature information can be achieved by encoding a neural network model.

[0055] Sub-step 8, concatenating the first concatenated feature information, the vehicle object color feature information, the at least one target vehicle object model feature information and the vehicle object overall extraction information to generate second concatenated feature information as object feature extraction information.

[0056] The content in "some optional implementations of some embodiments" as an inventive point of the present disclosure solves the technical problem mentioned in the background technology that "it is not possible to accurately extract the corresponding features of the vehicle object, resulting in the vehicle object information subsequently determined to be inaccurate." Based on this, the present disclosure uses a vehicle object segmentation model, a vehicle object license plate extraction model, a vehicle object color extraction model, a vehicle object model extraction model, and a vehicle object overall extraction model to extract multi-dimensional feature information related to the vehicle object in multiple directions, so that the extracted feature information is richer and more complete, providing a sufficient feature information basis for the subsequent determination of the vehicle object information.

[0057] In some optional implementations of some embodiments, the above-mentioned clustering process of each object feature extraction information in the obtained object feature extraction information set sequence to generate an object feature extraction information cluster set may include the following steps:

[0058] The first step is to determine the vehicle object driving time and vehicle object driving position information corresponding to each object feature extraction information in the above object feature extraction information set sequence, and obtain a vehicle object driving time set sequence and a vehicle object driving position information set sequence. The vehicle object driving time can be the frame time of the road shooting image corresponding to the object feature extraction information in the corresponding road shooting video. The vehicle object driving position information can be the object pixel position of the vehicle object in the corresponding road shooting image.

[0059] The second step is to set the intra-cluster restriction information for the above-mentioned object feature extraction information cluster set. The intra-cluster restriction information may be the restriction information corresponding to each object feature extraction information cluster in the object feature extraction information cluster set. For example, the intra-cluster restriction information may include: time restriction information within the cluster and position restriction information within the cluster. The time restriction information within the cluster may be that the time difference between each object feature extraction information in the cluster may be an arithmetic sequence, that is, the corresponding times are continuous. The position restriction information within the cluster may also be that the position transformation of each vehicle object driving position information corresponding to each object feature extraction information in the cluster presents a certain regularity. For example, it may be a linear regularity or a curvilinear regularity.

[0060] The third step is to cluster the object feature extraction information in the obtained object feature extraction information set sequence according to the above-mentioned intra-cluster restriction information, the above-mentioned vehicle object travel time set sequence and the above-mentioned vehicle object travel position information set sequence to generate an object feature extraction information cluster set.

[0061] Step 103: for each road camera video in the road camera video set, perform the following first generation step:

[0062] Step 1031: perform frame extraction processing on the road camera video to generate a road camera image sequence.

[0063] In some embodiments, the execution entity may perform frame extraction processing on the road camera video to generate a road camera image sequence.

[0064] Step 1032: Generate at least one piece of vehicle driving abnormality information corresponding to at least one piece of vehicle object information according to the road shot image sequence.

[0065] In some embodiments, the execution subject may generate at least one vehicle driving abnormality information corresponding to at least one vehicle object information according to the road shooting image sequence. The vehicle driving abnormality information may be driving abnormality behavior information of the object corresponding to the vehicle object information during the vehicle driving process. For example, the vehicle driving abnormality information may be a vehicle running a red light. There is a one-to-one correspondence between the vehicle object information in the at least one vehicle object information and the vehicle driving abnormality information in the at least one vehicle driving abnormality information. The vehicle object information set includes: the at least one vehicle object information.

[0066] In some optional implementations of some embodiments, the generating of at least one vehicle driving abnormality information corresponding to at least one vehicle object information according to the road shot image sequence may include the following steps:

[0067] In the first step, for each road shot image in the road shot image sequence, the following sixth generation step is performed:

[0068] Sub-step 1, input the above-mentioned road shooting image into a pre-trained vehicle object recognition information generation model to generate a vehicle object recognition information set. Among them, the above-mentioned at least one vehicle object information includes: a vehicle object information set included in the vehicle object recognition information set. The above-mentioned vehicle object recognition information set also includes: a vehicle object position recognition information set. The vehicle object recognition information generation model can be a neural network model for generating vehicle object recognition information. In practice, the vehicle object recognition information generation model can be a YOLO model. The vehicle object information set included in the vehicle object recognition information set is a subset of at least one vehicle object information. The vehicle object position recognition information can be the pixel position of the vehicle object in the road shooting image. There is a one-to-one correspondence between the vehicle object recognition information in the vehicle object recognition information set and the vehicle object information in the vehicle object information set. There is a one-to-one correspondence between the vehicle object recognition information in the vehicle object recognition information set and the vehicle object position recognition information in the vehicle object position recognition information set.

[0069] Sub-step 2: annotating the road shot image with vehicle object information according to the vehicle object recognition information set to generate an annotated road shot image.

[0070] As an example, the execution subject may mark each vehicle object information and each vehicle object position identification information included in the vehicle object identification information set on the road shooting map to generate a marked road shooting image.

[0071] Sub-step 3: cropping the road image and combining the object images according to the vehicle object recognition information set to generate a vehicle object cropped image set.

[0072] As an example, the execution subject crops the road shot image according to each vehicle object position identification information included in the vehicle object identification information set to obtain a cropped image set as the vehicle object cropped image set.

[0073] In the second step, each vehicle cropped image in the obtained vehicle cropped image set sequence is reorganized into a vehicle object to generate a vehicle cropped image sequence set, wherein each vehicle cropped image in each vehicle cropped image sequence in the vehicle cropped image sequence set corresponds to the same vehicle object.

[0074] In the third step, each vehicle cropped image sequence in the above vehicle cropped image sequence set is input into a pre-trained vehicle driving feature extraction model to generate vehicle driving feature information and obtain a vehicle driving feature information set. The vehicle driving feature extraction model can be a neural network model for generating vehicle driving feature information. The vehicle driving feature information can represent the driving-related feature semantics of the vehicle. The vehicle driving feature extraction model can be a residual model with multiple layers in series. For example, the vehicle driving feature extraction model can be a residual model with 18 layers in series.

[0075] In the fourth step, each annotated road shot image in the obtained annotated road shot image sequence is input into the multi-head attention mechanism model to generate an attention feature information set. The attention feature information in the attention feature information set has a one-to-one correspondence with each vehicle object information in the at least one vehicle object information. The attention feature information can be driving feature information extracted from each annotated road shot image, which characterizes the corresponding vehicle object during driving.

[0076] The fifth step is to combine the attention feature information in the above attention feature information set with the vehicle driving feature information in the above vehicle driving feature information set to generate a combined feature information set.

[0077] In step 6, the combined feature information set is input into a pre-trained vehicle driving abnormality information generation model to generate at least one vehicle driving abnormality information corresponding to the at least one vehicle object information. The vehicle driving abnormality information generation model may be a neural network model for generating vehicle driving abnormality information. In practice, the vehicle driving abnormality information generation model may include: a convolutional neural network model + a fully connected layer.

[0078] Step 104 : According to the vehicle driving abnormality information group corresponding to each vehicle object information, at least one candidate abnormal driving vehicle object information is screened out from the vehicle object information set.

[0079] In some embodiments, the execution subject may filter out at least one candidate abnormal driving vehicle object information from the vehicle object information set according to the vehicle driving abnormality information group corresponding to each vehicle object information. The candidate abnormal driving vehicle object information may be vehicle information corresponding to a vehicle that may have abnormal driving. For each vehicle object information, the vehicle object information may appear in at least one road shooting video in the road shooting video set, so the corresponding vehicle driving abnormality information group may be vehicle driving abnormality information corresponding to at least one road shooting video and associated with the vehicle object information.

[0080] In some optional implementations of some embodiments, the filtering out at least one candidate abnormal driving vehicle object information from the vehicle object information set according to the vehicle driving abnormality information group corresponding to each vehicle object information may include the following steps:

[0081] In the first step, for each vehicle object information in the vehicle object information set, in response to determining that the vehicle driving abnormality information group corresponding to the vehicle object information contains vehicle driving abnormality information representing vehicle driving abnormality, the vehicle object information is determined as candidate abnormal driving vehicle object information.

[0082] In the second step, for each vehicle object information in the vehicle object information set, in response to determining that there is no vehicle driving abnormality information representing vehicle driving abnormality in the vehicle driving abnormality information group corresponding to the above vehicle object information, the above vehicle object information is determined as candidate abnormal driving vehicle object information.

[0083] Step 105: for each candidate abnormal driving vehicle object information in the at least one candidate abnormal driving vehicle object information, perform the following second generation step:

[0084] Step 1051: Generate a set of optical flow information sequences for the above-mentioned candidate abnormal driving vehicle object information.

[0085] In some embodiments, the execution subject may generate a set of optical flow information sequences for the candidate abnormal driving vehicle object information, wherein the optical flow information may be information between two adjacent vehicle driving image frames in the vehicle driving image frame sequence, characterizing the motion change of the object corresponding to the candidate abnormal driving vehicle object information.

[0086] In some optional implementations of some embodiments, the generating of the optical flow information sequence set for the candidate abnormal driving vehicle object information may include the following steps:

[0087] In the first step, a vehicle driving image frame sequence set corresponding to the candidate abnormal driving vehicle object information is selected from the road camera video set.

[0088] In the second step, for each vehicle driving image frame sequence in the vehicle driving image frame sequence set, an optical flow information sequence for the candidate abnormal driving vehicle object information is generated according to the vehicle driving image frame sequence.

[0089] In some optional implementations of some embodiments, generating the optical flow information sequence for the candidate abnormal driving vehicle object information according to the vehicle driving image frame sequence may include the following steps:

[0090] In the first step, a first candidate optical flow information sequence for the candidate abnormal driving vehicle object information is generated based on the vehicle driving image frame sequence using a gradient-based differentiation method. The gradient-based differentiation method can be a method of calculating the velocity vector of a pixel using the spatiotemporal differentiation (i.e., the spatiotemporal gradient function) of the time-varying image grayscale (or its filtered form).

[0091] In the second step, a feature- and region-based matching method is used to generate a second candidate optical flow information sequence for the above-mentioned candidate abnormal driving vehicle object information according to the above-mentioned vehicle driving image frame sequence.

[0092] The third step is to generate a third candidate optical flow information sequence for the candidate abnormal driving vehicle object information based on the vehicle driving image frame sequence using a dense optical flow-based generation method. The dense optical flow-based generation method may be an image registration method that performs point-by-point matching on an image or a specified area.

[0093] The fourth step is to generate a fourth candidate optical flow information sequence for the candidate abnormal driving vehicle object information based on the vehicle driving image frame sequence using a generation method based on sparse optical flow. The generation method based on sparse optical flow can be an image registration method that specifies a set of points for tracking.

[0094] The fifth step is to determine the first effective value of the gradient-based differential method relative to the target road intersection corresponding scene, the second effective value of the feature and region-based matching method relative to the target road intersection corresponding scene, the third effective value of the dense optical flow-based generation method relative to the target road intersection corresponding scene, and the fourth effective value of the sparse optical flow-based generation method relative to the target road intersection corresponding scene. Among them, the first effective value can characterize the degree of adaptability of the method for generating optical flow information using the gradient-based differential method under the target road intersection corresponding scene. The second effective value can characterize the degree of adaptability of the method for generating optical flow information using the feature and region-based matching method under the target road intersection corresponding scene. The third effective value can characterize the degree of adaptability of the method for generating optical flow information using the dense optical flow-based generation method under the target road intersection corresponding scene. The fourth effective value can characterize the degree of adaptability of the method for generating optical flow information using the sparse optical flow-based generation method under the target road intersection corresponding scene. The first significant value, the second significant value, the third significant value, and the fourth significant value are all values ​​between 0 and 1.

[0095] In practice, the first significant value, the second significant value, the third significant value, and the fourth significant value may be preset by relevant algorithm experts according to the application scenario of the corresponding method.

[0096] The sixth step is to perform weighted summation processing on the first candidate optical flow information sequence, the second candidate optical flow information sequence, the third candidate optical flow information sequence, the fourth candidate optical flow information sequence, the first valid value, the second valid value, the third valid value and the fourth valid value to generate a weighted sum optical flow information sequence as the optical flow information sequence.

[0097] Step 1052: Generate vehicle driving route information and vehicle driving intention information corresponding to the candidate abnormal driving vehicle object information according to the optical flow information sequence set.

[0098] In some embodiments, the execution subject may generate the vehicle driving route information and vehicle driving intention information corresponding to the candidate abnormal driving vehicle object information according to the optical flow information sequence set. The vehicle driving route information may be the vehicle driving route corresponding to the candidate abnormal driving vehicle. The vehicle driving intention information may be the vehicle driving intention corresponding to the candidate abnormal driving vehicle.

[0099] As an example, first, the execution subject may perform a series connection process on the corresponding optical flow coordinates of each optical flow information in the optical flow information sequence set to generate the series-connected optical flow information. Then, the vehicle driving route information corresponding to the candidate abnormal driving vehicle object information is generated through the coordinate change corresponding to the series-connected optical flow information. Next, according to the vehicle driving route information and the intersection environment information corresponding to the target road intersection, the intention information generation model is used to generate the vehicle driving intention information. In practice, the intention information generation model may be a Transformer model. Specifically, the road direction information, road width information and indicator light information included in the intersection environment information are extracted. The road direction information, road width information and indicator light information are encoded to generate a road direction vector, a road width vector and an indicator light information vector. At least one key vehicle driving point is determined from the vehicle driving route information. Each key vehicle driving point in the at least one key vehicle driving point is vector-converted to generate a key vehicle driving point vector, and at least one key vehicle driving point vector is obtained. Finally, at least one key vehicle driving point vector, a road direction vector, a road width vector and an indicator light information vector are input into the Transformer model to generate the vehicle driving intention information. Specifically, the intention information corresponding to the vehicle driving route information can be generated as the vehicle driving intention information by querying a pre-set association table representing the route and intention information. The association table can be generated based on the intersection environment information.

[0100] Step 1053: Generate abnormal driving information for the candidate abnormal driving vehicle object information based on the vehicle driving route information and the vehicle driving intention information.

[0101] In some embodiments, the execution subject may generate abnormal driving information for the candidate abnormal driving vehicle object information based on the vehicle driving route information and the vehicle driving intention information. The abnormal driving information may be information indicating whether the vehicle object corresponding to the candidate abnormal driving vehicle object information has abnormal behavior.

[0102] As an example, the execution subject may determine whether the vehicle driving route information matches the vehicle driving intention information. In response to determining a match, abnormal driving information is generated that indicates that the vehicle object corresponding to the candidate abnormal driving vehicle object information does not have an abnormal form of behavior. In response to determining a mismatch, abnormal driving information is generated that indicates that the vehicle object corresponding to the candidate abnormal driving vehicle object information has an abnormal form of behavior.

[0103] As another example, by matching the intention with the route, the execution subject can determine at least one candidate driving route corresponding to the vehicle driving intention information. Then, it is determined whether the at least one candidate driving route includes the vehicle driving route information. In response to the determination that it is included, abnormal driving information is generated that characterizes that the vehicle object corresponding to the candidate abnormal driving vehicle object information does not have abnormal form of behavior. In response to the determination that it is not included, abnormal driving information is generated that characterizes that the vehicle object corresponding to the candidate abnormal driving vehicle object information has abnormal form of behavior.

[0104] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: the information generation method of some embodiments of the present disclosure can accurately and efficiently determine the vehicle object information of abnormal driving. Specifically, the reason why the relevant abnormal driving results are not accurate and efficient is that during the peak traffic period, the road shooting video is often a driving video shot for a large number of vehicles, and during the viewing process, the vehicle needs to be enlarged to determine the driving details of the vehicle. The workload of judging abnormal driving of the vehicle is extremely large, the work efficiency is low, and the accuracy of abnormal driving detection cannot be effectively guaranteed. Based on this, the information generation method of some embodiments of the present disclosure first obtains the road camera video set corresponding to the road camera device set as a data basis to facilitate the subsequent determination of abnormal driving vehicles. Among them, the above-mentioned road camera video set is a video set shot for the target road intersection. Then, the vehicle object information set corresponding to the above-mentioned road camera video set is determined, so as to facilitate all-round detection of each vehicle object information and ensure the comprehensiveness of abnormal driving detection. Then, for each road camera video in the above-mentioned road camera video set, the following first generation step is performed: the first step is to perform frame extraction processing on the above-mentioned road camera video to generate a road shooting image sequence, so as to facilitate the subsequent analysis of vehicles with abnormal driving. The second step is to accurately generate at least one vehicle driving abnormality information corresponding to at least one vehicle object information based on the above-mentioned road shooting image sequence, so as to be used for the subsequent determination of the corresponding vehicle object with abnormal driving. Among them, the above-mentioned vehicle object information set includes: the above-mentioned at least one vehicle object information. Furthermore, according to the vehicle driving abnormality information group corresponding to each vehicle object information, at least one candidate abnormal driving vehicle object information that may have abnormal driving behavior can be accurately preliminarily screened out from the above-mentioned vehicle object information set. Finally, for each candidate abnormal driving vehicle object information in the above-mentioned at least one candidate abnormal driving vehicle object information, the following second generation step is performed: the first step is to accurately generate an optical flow information sequence set for the above-mentioned candidate abnormal driving vehicle object information, so as to facilitate the subsequent determination of the vehicle driving route information and the vehicle driving intention information. In the second step, based on the optical flow information sequence set, the vehicle driving route information and vehicle driving intention information corresponding to the above-mentioned candidate abnormal driving vehicle object information can be accurately generated, so as to subsequently use the vehicle driving route information and vehicle driving intention information to accurately further determine whether the object corresponding to the candidate abnormal driving vehicle object information is driving abnormally. In the third step, based on the above-mentioned vehicle driving route information and the above-mentioned vehicle driving intention information, abnormal driving information for the above-mentioned candidate abnormal driving vehicle object information can be accurately generated. In summary, through the two-stage determination of abnormal driving vehicle objects, and the generation of vehicle driving route information and vehicle driving intention, vehicle objects with abnormal driving behavior can be accurately determined.

[0105] Further references Figure 2As an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of an information generating device, and these device embodiments are Figure 1 Corresponding to the method embodiments shown, the information generating device can be specifically applied to various electronic devices.

[0106] like Figure 2 As shown, an information generating device 200 includes: an acquisition unit 201, a determination unit 202, a first generation unit 203, a screening unit 204, and a second generation unit 205. The acquisition unit 201 is configured to acquire a road camera video set corresponding to a road camera device set, wherein the road camera video set is a video set shot at a target road intersection; the determination unit 202 is configured to determine a vehicle object information set corresponding to the road camera video set; the first generation unit 203 is configured to perform the following first generation step for each road camera video in the road camera video set: perform frame extraction processing on the road camera video to generate a road shooting image sequence; generate at least one vehicle driving abnormality information corresponding to at least one vehicle object information according to the road shooting image sequence, wherein the vehicle object information set includes: the at least one vehicle object information; the screening unit 204 is configured to screen the vehicle driving abnormality information set from the vehicle object information set according to the vehicle driving abnormality information group corresponding to each vehicle object information. At least one candidate abnormal driving vehicle object information is selected; the second generation unit 205 is configured to perform the following second generation step for each candidate abnormal driving vehicle object information in the at least one candidate abnormal driving vehicle object information: filter out a vehicle driving image frame sequence set corresponding to the candidate abnormal driving vehicle object information from the road camera video set; for each vehicle driving image frame sequence in the vehicle driving image frame sequence set, generate an optical flow information sequence for the candidate abnormal driving vehicle object information according to the vehicle driving image frame sequence; generate vehicle driving route information and vehicle driving intention information corresponding to the candidate abnormal driving vehicle object information according to the obtained optical flow information sequence set; generate abnormal driving information for the candidate abnormal driving vehicle object information according to the vehicle driving route information and the vehicle driving intention information.

[0107] It is understandable that the units recorded in the information generating device 200 are similar to those in the reference Figure 1 Therefore, the operations, features and beneficial effects described above for the method are also applicable to the information generating device 200 and the units included therein, and will not be described in detail here.

[0108] Reference below Figure 3, which shows a structural schematic diagram of an electronic device (eg, an electronic device) 300 suitable for implementing some embodiments of the present disclosure. Figure 3 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0109] like Figure 3 As shown, the electronic device 300 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the electronic device 300 are also stored. The processing device 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0110] Typically, the following devices may be connected to the I / O interface 305: input devices 306 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 308 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 309. The communication devices 309 may allow the electronic device 300 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 3 The electronic device 300 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 3 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0111] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 309, or installed from the storage device 308, or installed from the ROM 302. When the computer program is executed by the processing device 301, the above-mentioned functions defined in the method of some embodiments of the present disclosure are executed.

[0112] It should be noted that the computer-readable medium in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, the computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0113] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0114] The computer-readable medium may be included in the electronic device; or it may exist independently without being installed in the electronic device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: obtains a road camera video set corresponding to a road camera device set, wherein the road camera video set is a video set shot for a target road intersection; determines a vehicle object information set corresponding to the road camera video set; for each road camera video in the road camera video set, performs the following first generation step: performs frame extraction processing on the road camera video to generate a road shooting image sequence; based on the road shooting image sequence, generates at least one vehicle driving abnormality information corresponding to at least one vehicle object information, wherein the vehicle object information set includes: the at least one vehicle object information; based on the vehicle driving abnormality information group corresponding to each vehicle object information, generates at least one vehicle object information corresponding to at least one vehicle object information; At least one candidate abnormal driving vehicle object information is screened out from the information set; for each candidate abnormal driving vehicle object information in the at least one candidate abnormal driving vehicle object information, the following second generation step is performed: a vehicle driving image frame sequence set corresponding to the candidate abnormal driving vehicle object information is screened out from the road camera video set; for each vehicle driving image frame sequence in the vehicle driving image frame sequence set, an optical flow information sequence for the candidate abnormal driving vehicle object information is generated according to the vehicle driving image frame sequence; based on the obtained optical flow information sequence set, vehicle driving route information and vehicle driving intention information corresponding to the candidate abnormal driving vehicle object information are generated; based on the vehicle driving route information and the vehicle driving intention information, abnormal driving information for the candidate abnormal driving vehicle object information is generated.

[0115] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0116] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0117] The units described in some embodiments of the present disclosure may be implemented by software or hardware. The units described may also be provided in a processor, for example, may be described as: a processor including an acquisition unit, a determination unit, a first generation unit, a screening unit, and a second generation unit. The names of these units do not, in some cases, constitute limitations on the units themselves, for example, the acquisition unit may also be described as a "unit for acquiring a road camera video set corresponding to a road camera device set".

[0118] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0119] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) and the technical solutions formed.

Claims

1. A method for generating information, include: Acquire a road camera video set corresponding to the road camera device set, wherein the road camera video set is a video set shot at a target road intersection; Determine a vehicle object information set corresponding to the road camera video set; For each road camera video in the road camera video set, the following first generation step is performed: Performing frame extraction processing on the road camera video to generate a road shooting image sequence; generating at least one vehicle driving abnormality information corresponding to at least one vehicle object information according to the road shooting image sequence, wherein the vehicle object information set includes: the at least one vehicle object information; According to the vehicle driving abnormality information group corresponding to each vehicle object information, at least one candidate abnormal driving vehicle object information is screened out from the vehicle object information set; For each candidate abnormal driving vehicle object information in the at least one candidate abnormal driving vehicle object information, the following second generating step is performed: Generate an optical flow information sequence set for the candidate abnormal driving vehicle object information; Generating vehicle driving route information and vehicle driving intention information corresponding to the candidate abnormal driving vehicle object information according to the optical flow information sequence set; Abnormal driving information for the candidate abnormal driving vehicle object information is generated according to the vehicle driving route information and the vehicle driving intention information.

2. The method according to claim 1, in, The step of generating a set of optical flow information sequences for the candidate abnormally traveling vehicle object information comprises: Filtering out a vehicle driving image frame sequence set corresponding to the candidate abnormal driving vehicle object information from the road camera video set; For each vehicle driving image frame sequence in the vehicle driving image frame sequence set, an optical flow information sequence for the candidate abnormal driving vehicle object information is generated according to the vehicle driving image frame sequence.

3. The method according to claim 1, in, The determining of the vehicle object information set corresponding to the road camera video set includes: For each road camera video in the road camera video set, the following third generation step is performed: Determine a road shooting image sequence corresponding to the road shooting video as a target road shooting image sequence; Input each road shot image in the target road shot image sequence into a pre-trained object feature extraction model to generate an object feature extraction information set; Performing clustering processing on each object feature extraction information in the obtained object feature extraction information set sequence to generate an object feature extraction information cluster set; Verifying each object feature extraction information cluster in the object feature extraction information cluster set to generate a verification result; In response to determining that the verification result indicates that the verification is correct, for each object feature extraction information cluster in the object feature extraction information cluster set, the following fourth generation step is performed: Determine each target road shot image subsequence corresponding to each object feature extraction information in the object feature extraction information cluster; generating corresponding at least one vehicle object information according to each target road captured image subsequence; The obtained plurality of vehicle object information are determined as the vehicle object information set.

4. The method according to claim 3, in, The clustering process of each object feature extraction information in the obtained object feature extraction information set sequence to generate an object feature extraction information cluster set includes: Determine the vehicle object travel time and vehicle object travel position information corresponding to each object feature extraction information in the object feature extraction information set sequence, and obtain a vehicle object travel time set sequence and a vehicle object travel position information set sequence; Setting intra-cluster restriction information for the object feature extraction information cluster set; According to the intra-cluster restriction information, the vehicle object travel time set sequence and the vehicle object travel position information set sequence, each object feature extraction information in the obtained object feature extraction information set sequence is clustered to generate an object feature extraction information cluster set.

5. The method according to claim 1, in, The generating, according to the road photographed image sequence, at least one vehicle driving abnormality information corresponding to at least one vehicle object information comprises: For each road shot image in the road shot image sequence, the following sixth generation step is performed: Inputting the road shot image into a pre-trained vehicle object recognition information generation model to generate a vehicle object recognition information set, wherein the at least one vehicle object information includes: a vehicle object information set included in the vehicle object recognition information set, and the vehicle object recognition information set also includes: a vehicle object position recognition information set; annotating the road shot image with vehicle object information according to the vehicle object recognition information set to generate an annotated road shot image; According to the vehicle object recognition information set, the road shot image is cropped and the object image is combined to generate a vehicle object cropped image set; Reorganizing the vehicle objects of each vehicle cropped image in the obtained vehicle cropped image set sequence to generate a vehicle cropped image sequence set; Inputting each vehicle cropped image sequence in the vehicle cropped image sequence set into a pre-trained vehicle driving feature extraction model to generate vehicle driving feature information, thereby obtaining a vehicle driving feature information set; Input each annotated road shot image in the obtained annotated road shot image sequence into a multi-head attention mechanism model to generate an attention feature information set; The attention feature information in the attention feature information set is combined with the vehicle driving feature information in the vehicle driving feature information set to generate a combined feature information set; the combined feature information set is input into a pre-trained vehicle driving abnormality information generation model to generate at least one vehicle driving abnormality information corresponding to the at least one vehicle object information.

6. The method according to claim 1, in, The step of selecting at least one candidate abnormal driving vehicle object information from the vehicle object information set according to the vehicle driving abnormality information group corresponding to each vehicle object information includes: For each vehicle object information in the vehicle object information set, in response to determining that the vehicle object information corresponds to a vehicle driving abnormality information group including vehicle driving abnormality information representing vehicle driving abnormality, determining the vehicle object information as candidate abnormal driving vehicle object information; For each vehicle object information in the vehicle object information set, in response to determining that the vehicle driving abnormality information group corresponding to the vehicle object information does not contain vehicle driving abnormality information representing vehicle driving abnormality, the vehicle object information is determined as candidate abnormal driving vehicle object information.

7. The method according to claim 2, in, The step of generating an optical flow information sequence for the candidate abnormal driving vehicle object information according to the vehicle driving image frame sequence comprises: Using a gradient-based differentiation method, a first candidate optical flow information sequence for the candidate abnormal driving vehicle object information is generated according to the vehicle driving image frame sequence; Using a feature- and region-based matching method, a second candidate optical flow information sequence for the candidate abnormal driving vehicle object information is generated according to the vehicle driving image frame sequence; Using a dense optical flow-based generation method, a third candidate optical flow information sequence for the candidate abnormal driving vehicle object information is generated according to the vehicle driving image frame sequence; Generate a fourth candidate optical flow information sequence for the candidate abnormal driving vehicle object information according to the vehicle driving image frame sequence by using a generation method based on sparse optical flow; Determine a first effective value of the gradient-based differential method relative to the target road intersection corresponding to the intersection scene, a second effective value of the feature-and-area-based matching method relative to the target road intersection corresponding to the intersection scene, a third effective value of the dense optical flow-based generation method relative to the target road intersection corresponding to the intersection scene, and a fourth effective value of the sparse optical flow-based generation method relative to the target road intersection corresponding to the intersection scene; The first candidate optical flow information sequence, the second candidate optical flow information sequence, the third candidate optical flow information sequence, the fourth candidate optical flow information sequence, the first valid value, the second valid value, the third valid value and the fourth valid value are weightedly summed to generate a weighted sum optical flow information sequence as the optical flow information sequence.

8. An information generating device, include: An acquisition unit is configured to acquire a road camera video set corresponding to a road camera device set, wherein the road camera video set is a video set shot at a target road intersection; a determination unit, configured to determine a vehicle object information set corresponding to the road camera video set; The first generating unit is configured to perform the following first generating step for each road camera video in the road camera video set: performing frame extraction processing on the road camera video to generate a road shooting image sequence; generating at least one vehicle driving abnormality information corresponding to at least one vehicle object information according to the road shooting image sequence, wherein the vehicle object information set includes: the at least one vehicle object information; A screening unit is configured to screen out at least one candidate abnormal driving vehicle object information from the vehicle object information set according to the vehicle driving abnormality information group corresponding to each vehicle object information; The second generating unit is configured to perform the following second generating step for each candidate abnormal driving vehicle object information in the at least one candidate abnormal driving vehicle object information: generate an optical flow information sequence set for the candidate abnormal driving vehicle object information; generate vehicle driving route information and vehicle driving intention information corresponding to the candidate abnormal driving vehicle object information based on the optical flow information sequence set; generate abnormal driving information for the candidate abnormal driving vehicle object information based on the vehicle driving route information and the vehicle driving intention information.

9. An electronic device, include: one or more processors; a storage device having one or more programs stored thereon, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

10. A computer readable medium having a computer program stored thereon, in, When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Citation Information

Patent Citations

  • Moving vehicle detecting method based on aerially photographed video images

    CN106683119A

  • A vehicle retrograde motion intelligent detection method based on tracking trajectory analysis

    CN109948582A

  • End-to-end vehicle behavior detection method, system and device and storage medium

    CN114241373A

  • Vehicle abnormal driving behavior detection method and device based on optical flow estimation

    CN117456479A

  • Systems and methods for analyzing the in-lane driving behavior of a road agent external to a vehicle

    US20220277647A1