Vehicle related information sending method, device, equipment and computer readable medium

Abstract

The embodiments of the present disclosure disclose a method, device, equipment and computer-readable medium for sending vehicle-related information. A specific implementation of the method includes: in response to receiving multiple driving videos, performing frame extraction processing on each driving video to obtain multiple frame image sequences; performing time placement on each frame image to generate a frame image group sequence; for each frame image group, determining the corresponding first vehicle object driving information set; determining the vehicle driving behavior information corresponding to each vehicle object; for each vehicle object, executing the first generation step: determining the corresponding vehicle object driving information sequence; generating driving compliance confirmation information; in response to determining that the driving compliance confirmation information indicates that the vehicle object has illegal driving behavior, sending the vehicle object driving information sequence and the corresponding vehicle object driving calibration image group sequence to the vehicle driving management terminal. This implementation can accurately and efficiently realize driving violation detection of driving vehicles.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technology, and in particular to a method, device, equipment, and computer-readable medium for sending vehicle-related information. Background Art

[0002] At present, illegal driving of vehicles often brings great harm to society. Therefore, illegal driving detection of vehicles has become an important detection direction in the current vehicle traffic field. For vehicle violation detection, the usual method is: through the vehicle driving video captured by the target camera, using the vehicle violation detection model, directly output the vehicle driving violation detection information.

[0003] However, when the above method is used to detect vehicle violations, the following technical problems often occur:

[0004] The vehicle driving video captured by the target camera is rather one-sided and cannot fully and accurately determine the actual driving operation of the vehicle, which results in the vehicle driving violation detection information output by the subsequent vehicle violation detection model being inaccurate.

[0005] In the process of adopting technical solutions to solve the above technical problem 1, the following technical problem 2 is often accompanied by "how to more accurately generate the object's driving action information based on the orientation information and shooting angle information". For the above technical problem 2, the conventional solution is generally to determine the object's action in images at different orientations and shooting angles by artificial means to obtain the corresponding object's driving action information. However, the above conventional solution still has the following problems: the generation efficiency is low, and the object's driving action information cannot be effectively guaranteed.

[0006] The above information disclosed in this Background section is only for enhancement of understanding of the background of the inventive concept and therefore it may contain information that does not form the prior art that is already known in this country to a person of ordinary skill in the art. Summary of the invention

[0007] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0008] Some embodiments of the present disclosure propose a method, apparatus, device and computer-readable medium for sending vehicle-related information to solve one or more of the technical problems mentioned in the above background technology section.

[0009] In a first aspect, some embodiments of the present disclosure provide a method for sending vehicle-related information, comprising: in response to receiving multiple driving videos shot by multiple road camera devices, performing frame extraction processing on each of the multiple driving videos to generate a frame image sequence, thereby obtaining multiple frame image sequences, wherein the multiple road camera devices are arranged at respective orientations of the corresponding roads; according to the shooting time corresponding to each frame image, performing time placement on each frame image in the multiple frame image sequences to generate a frame image group sequence, wherein the shooting time corresponding to each frame image in the frame image group is the same; for each frame image group in the frame image group sequence, determining a first vehicle object driving information set corresponding to the frame image group, wherein the first vehicle object driving information set in the first vehicle object driving information set corresponds to the road at the target orientation at the target shooting time. There is a one-to-one correspondence between the vehicle objects in the vehicle object set photographed by the road camera device; according to the above-mentioned frame image group sequence, the vehicle driving behavior information corresponding to each vehicle object in the above-mentioned vehicle object set is determined to obtain a vehicle driving behavior information set; for each vehicle object in the above-mentioned vehicle object set, the following first generation step is performed: according to the obtained first vehicle object driving information set sequence, the vehicle object driving information sequence corresponding to the above-mentioned vehicle object is determined; according to the above-mentioned vehicle object driving information sequence and the above-mentioned vehicle driving behavior information set, driving compliance confirmation information for the above-mentioned vehicle object is generated; in response to determining that the above-mentioned driving compliance confirmation information represents that the above-mentioned vehicle object has illegal driving behavior, the above-mentioned vehicle object driving information sequence and the corresponding vehicle object driving calibration image group sequence are sent to the vehicle driving management terminal to confirm the illegal driving of the vehicle.

[0010] In a second aspect, some embodiments of the present disclosure provide a vehicle-related information sending device, including: a frame extraction unit, configured to, in response to receiving a plurality of driving videos shot by a plurality of road camera devices, perform frame extraction processing on each of the plurality of driving videos to generate a frame image sequence, thereby obtaining a plurality of frame image sequences, wherein the plurality of road camera devices are arranged at respective positions of the corresponding roads; a placement unit, configured to perform time placement on each frame image in the plurality of frame image sequences according to the shooting time corresponding to each frame image to generate a frame image group sequence, wherein the shooting time corresponding to each frame image in the frame image group is the same; a first determination unit, configured to determine, for each frame image group in the frame image group sequence, a first vehicle object driving information set corresponding to the frame image group, wherein the first vehicle object driving information in the first vehicle object driving information set is consistent with the target vehicle at the target shooting time. There is a one-to-one correspondence between the vehicle objects in the vehicle object set photographed by the road camera device with the marked orientation; the second determination unit is configured to determine the vehicle driving behavior information corresponding to each vehicle object in the vehicle object set according to the frame image group sequence, and obtain the vehicle driving behavior information set; the execution unit is configured to perform the following first generation step for each vehicle object in the vehicle object set: determine the vehicle object driving information sequence corresponding to the vehicle object according to the obtained first vehicle object driving information set sequence; generate driving compliance confirmation information for the vehicle object according to the vehicle object driving information sequence and the vehicle driving behavior information set; in response to determining that the driving compliance confirmation information indicates that the vehicle object has illegal driving behavior, send the vehicle object driving information sequence and the corresponding vehicle object driving calibration image group sequence to the vehicle driving management terminal to confirm the illegal driving of the vehicle.

[0011] In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner in the first aspect.

[0012] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method described in any implementation manner in the first aspect is implemented.

[0013] The above-mentioned embodiments of the present disclosure have the following beneficial effects: through the vehicle-related information sending method of some embodiments of the present disclosure, the driving violation detection of the moving vehicle can be accurately and efficiently realized. Specifically, the reason for the inaccuracy of the relevant driving violation detection is that the vehicle driving video shot by the target camera is relatively one-sided and cannot fully and accurately determine the actual driving operation of the vehicle. As a result, the vehicle driving violation detection information output by the subsequent vehicle violation detection model is not accurate enough. Based on this, the vehicle-related information sending method of some embodiments of the present disclosure, first, in response to receiving multiple driving videos shot by multiple road camera devices, each of the multiple driving videos is subjected to frame extraction processing to generate a frame image sequence to obtain multiple frame image sequences. Among them, the multiple road camera devices are arranged at various positions of the corresponding road. Here, by shooting the driving conditions of the vehicle in all directions through multiple driving videos, the problem of inaccurate driving hazard detection caused by the one-sided shooting angle can be avoided. Then, according to the shooting time corresponding to each frame image, each frame image in the multiple frame image sequence is time-placed to generate a frame image group sequence. Among them, the shooting time corresponding to each frame image in the frame image group is the same. Here, by putting the frame images with the same shooting time in a group, a full range of driving behavior analysis can be performed on the vehicle object at the same time in the future, thereby ensuring the accuracy of the subsequent driving violation detection. Secondly, for each frame image group in the above frame image group sequence, the first vehicle object driving information set corresponding to the above frame image group can be accurately determined. Among them, there is a one-to-one correspondence between the first vehicle object driving information in the above first vehicle object driving information set and the vehicle object in the vehicle object set photographed by the road camera device at the target orientation at the target shooting time. Then, according to the above frame image group sequence, the vehicle driving behavior information corresponding to each vehicle object in the above vehicle object set can be accurately determined to obtain the vehicle driving behavior information set. Finally, for each vehicle object in the above vehicle object set, the following first generation step is performed: the first step is to accurately determine the vehicle object driving information sequence corresponding to the above vehicle object according to the obtained first vehicle object driving information set sequence, so as to facilitate the generation of subsequent driving compliance determination information. The second step is to accurately generate driving compliance confirmation information for the above vehicle object according to the above vehicle object driving information sequence and the above vehicle driving behavior information set. In the third step, in response to determining that the driving compliance confirmation information indicates that the vehicle object has illegal driving behavior, the vehicle object driving information sequence and the corresponding vehicle object driving calibration image group sequence are sent to the vehicle driving management terminal to confirm the vehicle's illegal driving. In summary, through all-round driving detection of multiple driving videos, driving violation detection of driving vehicles can be accurately and efficiently achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0015] Figure 1 is a flowchart of some embodiments of the method for sending vehicle-related information according to the present disclosure;

[0016] Figure 2 is a schematic diagram of the structure of some embodiments of the vehicle-related information sending device according to the present disclosure;

[0017] Figure 3 It is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0018] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0019] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0020] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0021] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0022] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0023] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0024] refer to Figure 1, shows a process 100 of some embodiments of the method for sending vehicle related information according to the present disclosure. The method for sending vehicle related information includes the following steps:

[0025] Step 101 , in response to receiving a plurality of driving videos shot by a plurality of road camera devices, performing frame extraction processing on each of the plurality of driving videos to generate a frame image sequence, thereby obtaining a plurality of frame image sequences.

[0026] In some embodiments, in response to receiving multiple driving videos captured by multiple road cameras, the execution subject of the above-mentioned vehicle-related information sending method can perform frame extraction processing on each of the above-mentioned multiple driving videos to generate a frame image sequence and obtain multiple frame image sequences. Multiple road cameras are arranged at various positions of the corresponding roads. For example, the multiple road camera devices include: a road camera located at the left side of the road, a road camera located above the center of the road, and a road camera located at the right side of the road. There is a one-to-one correspondence between the driving videos in the multiple driving videos and the road camera devices in the multiple road cameras. The driving videos in the multiple driving videos are road videos corresponding to the roads captured by the road cameras.

[0027] As an example, the execution entity may extract a frame of image from the driving video every 0.3 seconds to generate a frame image sequence.

[0028] Step 102 : Time-arrange each frame image in the plurality of frame image sequences according to the shooting time corresponding to each frame image, so as to generate a frame image group sequence.

[0029] In some embodiments, the execution subject may time-place each frame image in the plurality of frame image sequences according to the shooting time corresponding to each frame image, so as to generate a frame image group sequence. The shooting time corresponding to each frame image in the frame image group is the same. The shooting time may be the time point of the frame image in the corresponding driving video.

[0030] Step 103: for each frame image group in the frame image group sequence, determine a first vehicle object driving information set corresponding to the frame image group.

[0031] In some embodiments, the execution subject may determine the first vehicle object driving information set corresponding to the frame image group for each frame image group in the frame image group sequence. There is a one-to-one correspondence between the first vehicle object driving information in the first vehicle object driving information set and the vehicle object in the vehicle object set photographed by the road camera device at the target orientation at the target shooting time. The first vehicle object driving information may characterize the object driving condition information of the vehicle object. The object driving condition information may be driving condition information of whether the vehicle object has dangerous driving actions. The target orientation may be the orientation of the road camera device at the front of the road and located above the center. That is, the road camera device corresponding to the target orientation will first photograph the vehicle object traveling on the road. The target shooting time may be the shooting time corresponding to the first frame image in the frame image sequence (that is, the time point of the first frame image in the driving video).

[0032] In some optional implementations of some embodiments, the first vehicle object driving information in the above-mentioned first vehicle object driving information set includes: vehicle object driving action information and vehicle license plate. The vehicle object driving action information may be action information of the vehicle driving object during driving. For example, the vehicle object driving action information may be driving action information of turning the steering wheel, or driving and talking on the phone action information. The vehicle license plate may be the license plate of the vehicle driven by the vehicle object corresponding to the first vehicle object driving information.

[0033] Optionally, the determining of the first vehicle object driving information set corresponding to the frame image group may include the following steps:

[0034] In the first step, for each vehicle object in the above vehicle object set, the following second generation step is performed:

[0035] In the first sub-step, each frame image in the above-mentioned frame image group is input into the vehicle object recognition model to generate vehicle recognition information and obtain a vehicle recognition information group. The vehicle recognition information is the recognition information for the above-mentioned vehicle object. The vehicle recognition information includes: vehicle position information. The vehicle position information can be the pixel coordinate position information of the object content corresponding to the vehicle object in the frame image. In practice, the vehicle object recognition model can be a neural network model for generating vehicle recognition information. Specifically, the vehicle object recognition model can be an image recognition model for the vehicle object. For example, the vehicle object recognition model can be a YOLO v5 model.

[0036] In a second sub-step, the frame images corresponding to the vehicle identification information indicating that the vehicle object is not identified are removed from the frame image group to obtain a post-removal frame image group.

[0037] In the third sub-step, according to the obtained vehicle position information group, each image in the above-mentioned removed frame image group is subjected to vehicle object target magnification and cropping processing to generate a processed image group. The ratio of the number of pixels corresponding to the content of the above-mentioned vehicle object in the processed image is not less than the target ratio. The target ratio may be a preset ratio. For example, the target ratio is 0.7. The vehicle object target magnification may be pixel magnification of the object content of the vehicle object in the image. The cropping processing may be object cropping according to the object content of the vehicle object in the image.

[0038] The fourth sub-step is to generate a vehicle license plate detection information group for the above-mentioned processed image group using a pre-trained vehicle license plate detection model. The vehicle license plate detection model may be a neural network model for detecting vehicle license plates. For example, the vehicle license plate detection model may be a U-net model. There is a one-to-one correspondence between the processed images in the processed image group and the vehicle license plate detection information in the vehicle license plate detection information group. The vehicle license plate detection information may be detection information of the vehicle license plate.

[0039] The fifth sub-step is to generate the vehicle license plate corresponding to the vehicle object in various ways according to the vehicle license plate detection information group.

[0040] The sixth sub-step is to generate a vehicle object driving action detection information group for the above-mentioned processed image group using a pre-trained vehicle object driving action detection information generation model. The vehicle object driving action detection information generation model may be a neural network model for generating vehicle object driving action detection information. For example, the vehicle object driving action detection information generation model may be a convolutional neural network model with multiple layers of serial connections based on key points. There is a one-to-one correspondence between the vehicle object driving action information in the vehicle object driving action detection information group and the processed images in the processed image group. The vehicle object driving action detection information may be detection information of the vehicle object driving action.

[0041] In a seventh sub-step, based on the vehicle object driving action detection information group, the vehicle object driving action information corresponding to the vehicle object can be generated in various ways.

[0042] In some optional implementations of some embodiments, generating the vehicle object driving action information corresponding to the vehicle object according to the vehicle object driving action detection information group may include the following steps:

[0043] In the first step, in response to determining that the vehicle object driving action detection information group contains detection information indicating that the vehicle object has dangerous actions, the position information and shooting angle corresponding to each of the plurality of road camera devices are determined. The position information may represent the actual placement position of the road camera device. The shooting angle may be a shooting angle of the road camera device facing the road.

[0044] The second step is to establish a coordinate system with the road camera device at the target orientation as the coordinate origin.

[0045] In the third step, coordinate conversion is performed on the obtained multiple position information and multiple shooting angles to generate multiple converted position information and multiple converted shooting angles in the above coordinate system. There is a one-to-one correspondence between the converted position information in the multiple converted position information and the position information in the multiple position information. There is a one-to-one correspondence between the converted shooting angles in the multiple converted shooting angles and the shooting angles in the multiple shooting angles.

[0046] In the fourth step, the above-mentioned multiple converted orientation information, the above-mentioned multiple converted shooting angles and the above-mentioned processed image group are input into the object driving action omnidirectional detection information generation model to generate the object driving action omnidirectional detection information. Wherein, the above-mentioned object driving action omnidirectional detection information includes: a first probability information characterizing the existence of dangerous actions of the above-mentioned vehicle object and a second probability information characterizing the absence of dangerous actions of the above-mentioned vehicle object. Wherein, the sum of the first probability information and the second probability information is a value of 1. Wherein, the object driving action omnidirectional detection information generation model can be a neural network model for generating the object driving action omnidirectional detection information. The object driving action omnidirectional detection information can be determined by omnidirectional shooting angles. Therefore, the problem of inaccurate action recognition caused by one-sided shooting angles can be avoided by the object driving action omnidirectional detection information. In practice, the object driving action omnidirectional detection information generation model can be a plurality of parallel connected convolutional neural networks + an attention mechanism model based on convolutional neural networks + a fully connected layer.

[0047] Step 5: In response to determining that the value of the second probability information minus the first probability information is greater than the target value, generate vehicle object driving action information indicating that the vehicle object does not have a dangerous action. The target value may be a preset value. For example, the target value is 0.3.

[0048] In step 6, in response to determining that the value of the second probability information minus the first probability information is equal to or less than the target value, the vehicle object driving action detection information group, the object driving action omnidirectional detection information and the processed image group are sent to an action detection terminal. The action detection terminal may be a terminal for detecting actions performed by the vehicle object during driving.

[0049] The seventh step is to receive the vehicle object driving action information corresponding to the above-mentioned action detection terminal.

[0050] Considering the problem of the conventional solution above, "determining the object action in images at different orientations and shooting angles by artificial means to obtain the corresponding object driving action information", facing the above technical problem 2: low generation efficiency and the object driving action information cannot be effectively guaranteed. Combining the advantages / technical status, it can be decided to adopt the following solution:

[0051] In some optional implementations of some embodiments, the above-mentioned object driving action omnidirectional detection information generation model includes: an image feature extraction model for multi-orientation information and multi-shooting angles, a target orientation image generative and adversarial model for multi-orientation information and multi-shooting angles, an image feature fusion model, a first attention mechanism model, a second attention mechanism model, and an output layer. The image feature extraction model may be a neural network model for extracting image features. The target orientation image generative and adversarial models may be generative and adversarial models for generating images under the target orientation. In practice, the target orientation image generative model may be a convolutional neural network with multiple layers connected in series. The target orientation image discriminative model may also be a convolutional neural network with multiple layers connected in series. The number of network layers corresponding to the target orientation image generative model and the target orientation image discriminative model may be different. The first attention mechanism model may be a model for extracting key attention feature information of the image under the target orientation. The second attention mechanism model may be a model for extracting key attention feature information of the image under different orientations. The multiple converted orientation information includes: first orientation information, second orientation information, and third orientation information. In practice, the first orientation information may be left orientation information. The second orientation information may be center-up orientation information. The third orientation information may be right orientation information. The plurality of converted shooting angles may include: a left shooting angle, a middle shooting angle, and a right shooting angle.

[0052] Optionally, inputting the plurality of converted position information, the plurality of converted shooting angles and the processed image group into an object driving action omnidirectional detection information generation model to generate the object driving action omnidirectional detection information may include the following steps:

[0053] In the first step, for each converted position information in the above-mentioned plurality of converted position information, the following information generation steps are performed:

[0054] The first sub-step is to input each processed image in the processed image group into the image feature extraction model to generate image feature extraction information and obtain an image feature extraction information group.

[0055] The second sub-step, in response to determining that the converted orientation information is the first orientation information, determines the image feature extraction information in the image feature extraction information group corresponding to the second orientation information as the first image feature extraction information, and determines the image feature extraction information in the image feature extraction information group corresponding to the third orientation information as the second image feature extraction information.

[0056] The third sub-step is to input the second orientation information and the first image feature extraction information into the target orientation image generation and adversarial model to generate the first candidate image feature extraction information under the first orientation information.

[0057] The fourth sub-step is to input the third orientation information and the second image feature extraction information into the target orientation image generation and adversarial model to generate the second candidate image feature extraction information under the first orientation information.

[0058] The fifth sub-step is to determine the image feature extraction information in the image feature extraction information group corresponding to the first orientation information as the third image feature extraction information.

[0059] In a sixth sub-step, the first candidate image feature extraction information, the second candidate image feature extraction information and the third image feature extraction information are input into an image feature fusion model to generate image feature fusion information. The image feature fusion model may be a fully connected layer. In addition, the image feature fusion model may be a vector concatenation model.

[0060] The seventh sub-step is to input the above-mentioned image feature fusion information into the above-mentioned first attention mechanism model to generate first attention feature information.

[0061] In the second step, the obtained multiple first attention feature information are input into the second attention mechanism model to generate second attention feature information.

[0062] The third step is to input the third attention feature information into the output layer to output the all-round detection information of the object's driving action.

[0063] The above-mentioned content of "in some optional implementation methods of some embodiments" as one of the inventive points of the present disclosure solves the technical problem mentioned in the background technology that "the generation efficiency is low, and the object driving action information cannot be effectively guaranteed". Based on this, the present disclosure, through the object driving action omnidirectional detection information generation model including the image feature extraction model for multi-orientation information and multi-shooting angles, the target orientation image generation and adversarial model for multi-orientation information and multi-shooting angles, can realize the enhancement of image feature extraction information in different orientations and different shooting angles, so as to obtain enhanced image feature extraction information in different orientations and different shooting angles. Furthermore, through the image feature fusion model, the first attention mechanism model, the second attention mechanism model and the output layer, accurate object driving action omnidirectional detection information is generated for multiple enhanced image feature extraction information.

[0064] In some optional implementations of some embodiments, the license plate detection information corresponding to the vehicle license plate with the highest frequency is screened out from the above-mentioned vehicle license plate detection information group as the vehicle license plate corresponding to the above-mentioned vehicle object.

[0065] Step 104 : determining the vehicle driving behavior information corresponding to each vehicle object in the vehicle object set according to the frame image group sequence, and obtaining a vehicle driving behavior information set.

[0066] In some embodiments, the execution subject may determine the vehicle driving behavior information corresponding to each vehicle object in the vehicle object set according to the frame image group sequence to obtain a vehicle driving behavior information set. The vehicle driving behavior information may be behavior information of the vehicle object driving the vehicle. For example, the vehicle driving behavior information may be behavior information indicating that the vehicle is driving to the left.

[0067] In some optional implementations of some embodiments, determining the vehicle driving behavior information corresponding to each vehicle object in the vehicle object set according to the frame image group sequence may include the following steps:

[0068] The first step is to determine a frame image sequence group corresponding to the vehicle object. Each frame image sequence has a corresponding road shooting device. That is, each frame image sequence is an image sequence after the corresponding road shooting device shoots the vehicle object.

[0069] In the second step, for each frame image sequence in the above frame image sequence group, the following third generation step is performed:

[0070] The first sub-step is to obtain a behavior indicator set. The behavior indicator set may be an indicator set for vehicle driving behavior information. The indicator information set corresponding to the behavior indicator set may indirectly reflect the driving behavior of the vehicle object. Specifically, the behavior indicator set may include but is not limited to at least one of the following: a vehicle turning indicator and a vehicle spacing indicator.

[0071] The second sub-step is to input each frame image in the frame image sequence into a first behavior feature information generation model to generate first behavior feature information. The first behavior feature information is a first behavior indicator information set corresponding to the behavior indicator set. The first behavior feature information generation model can be a neural network model that generates the first behavior feature information. In practice, the first behavior feature information generation model can be a convolutional neural network model based on vehicle key points.

[0072] In the third sub-step, each frame image in the frame image sequence is input into an image local feature extraction model to generate image local feature extraction information. The image local feature extraction model may be a neural network model that extracts image feature information corresponding to the content of the driving object in the frame image. In practice, the image local feature extraction model may include: a driving object segmentation model and a convolutional neural network model.

[0073] The fourth sub-step is to vectorize each first behavior indicator information in the first behavior indicator information set corresponding to the first behavior characteristic information to generate a first behavior indicator vector set.

[0074] As an example, the execution subject may vectorize each first behavior indicator information in the first behavior indicator information set corresponding to the first behavior feature information through a word embedding model to generate a first behavior indicator vector set.

[0075] In a fifth sub-step, the obtained image local feature extraction information sequence and the obtained first behavior indicator vector set sequence are input into a vehicle driving behavior information generation model based on a time series neural network model to generate first vehicle driving behavior information. The vehicle driving behavior information generation model based on a time series neural network model may be a neural network model for generating vehicle driving behavior information. In practice, the vehicle driving behavior information generation model based on a time series neural network model may be an LSTM model.

[0076] The third step is to generate the above vehicle driving behavior information based on the obtained first vehicle driving behavior information group.

[0077] In some optional implementations of some embodiments, the generating of the vehicle driving behavior information according to the obtained first vehicle driving behavior information group may include the following steps:

[0078] In the first step, in response to determining that the first vehicle driving behavior information group contains first vehicle driving behavior information indicating that the vehicle is driving dangerously, at least one first target vehicle driving behavior information is obtained. The first target vehicle driving behavior information may be first vehicle driving behavior information indicating that the vehicle is driving dangerously.

[0079] The second step is to generate at least one frame image sequence corresponding to the at least one first target vehicle driving behavior information.

[0080] The third step is to time-arrange the frame images in the at least one frame image sequence to generate a frame image subgroup sequence, wherein the corresponding time of each frame image in the frame image subgroup is the same.

[0081] Step 4: for each frame image subgroup in the frame image subgroup sequence, perform the following fourth generation step:

[0082] In the first sub-step, each frame image included in the frame image subset is input into a second behavior feature information generation model based on an attention mechanism model to generate second behavior feature information. The second behavior feature information is a second behavior indicator information set corresponding to the behavior indicator set. In practice, the second behavior feature information generation model based on the attention mechanism model can be a convolutional network model based on a multi-head attention mechanism.

[0083] The second sub-step is to vectorize each second behavior indicator information in the second behavior indicator information set corresponding to the second behavior characteristic information to generate a second behavior indicator vector set. For specific implementation methods, refer to the generation of the first behavior indicator vector.

[0084] The third sub-step is to determine the image local feature extraction information sub-group corresponding to the frame image sub-group. The generation of the image local feature extraction information sub-group can refer to the generation of the image local feature extraction information sequence.

[0085] The fourth sub-step is to integrate the feature extraction information of the above-mentioned image local feature extraction information subgroups to generate integrated image local feature extraction information.

[0086] As an example, the execution subject may perform feature information splicing on each image local feature information in the image local feature extraction information subgroup to generate spliced ​​feature information as the integrated image local feature extraction information.

[0087] In the fifth step, the obtained integrated image local feature extraction information sequence and the obtained second behavior indicator vector set sequence are input into the vehicle driving behavior information generation model based on the temporal neural network model to generate the second vehicle driving behavior information.

[0088] The sixth step is to generate the above-mentioned vehicle driving behavior information based on the above-mentioned second vehicle driving behavior information.

[0089] As an example, the above-mentioned execution entity may directly determine the second vehicle driving behavior information as the vehicle driving behavior information.

[0090] Step 105: for each vehicle object in the vehicle object set, perform the following first generation step:

[0091] Step 1051: Determine a vehicle object driving information sequence corresponding to the vehicle object according to the obtained first vehicle object driving information set sequence.

[0092] In some embodiments, the execution subject may determine the vehicle object driving information sequence corresponding to the vehicle object according to the obtained first vehicle object driving information set sequence, wherein the vehicle object driving information sequence corresponding to the vehicle object may be the first vehicle object driving information sequence corresponding to the vehicle object.

[0093] Step 1052: Generate driving compliance confirmation information for the vehicle object based on the vehicle object driving information sequence and the vehicle driving behavior information set.

[0094] In some embodiments, the execution subject may generate driving compliance confirmation information for the vehicle object according to the vehicle object driving information sequence and the vehicle driving behavior information set, wherein the driving compliance confirmation information may be confirmation information indicating whether the vehicle object is driving in compliance with the vehicle.

[0095] As an example, first, the execution subject may compare the vehicle object driving information sequence and the vehicle driving behavior information set with the dangerous driving behavior information in the dangerous driving behavior information table to generate comparison information. Then, based on the comparison information, the driving compliance confirmation information is generated.

[0096] Step 1053, in response to determining that the driving compliance confirmation information indicates that the vehicle object has illegal driving behavior, the vehicle object driving information sequence and the corresponding vehicle object driving calibration image group sequence are sent to the vehicle driving management terminal to confirm the vehicle's illegal driving.

[0097] In some embodiments, in response to determining that the driving compliance confirmation information indicates that the vehicle object has illegal driving behavior, the execution entity may send the vehicle object driving information sequence and the corresponding vehicle object driving calibration image group sequence to the vehicle driving management terminal to confirm the vehicle's illegal driving. The vehicle object driving calibration image may be an image of the vehicle corresponding to the calibrated vehicle object in the frame image. The vehicle driving management terminal may be a management terminal for confirming the vehicle's illegal driving.

[0098] In some optional implementations of some embodiments, the sending of the vehicle object driving information sequence and the corresponding vehicle object driving calibration image group sequence to the vehicle driving management terminal to confirm the vehicle's illegal driving may include the following steps:

[0099] The first step is to determine a sequence of removed frame images and a sequence of vehicle identification information groups corresponding to the vehicle objects.

[0100] In the second step, according to the vehicle identification information group sequence, each image in the removed frame image group sequence is calibrated with a bounding box to generate a vehicle object driving calibration image group and obtain a vehicle object driving calibration image group sequence.

[0101] As an example, the above-mentioned execution entity performs bounding box calibration on each image in the above-mentioned removed frame image group sequence according to the vehicle position information group sequence included in the above-mentioned vehicle identification information group sequence to generate a vehicle object driving calibration image group and obtain a vehicle object driving calibration image group sequence.

[0102] The third step is to send the above vehicle object driving information sequence and the corresponding vehicle object driving calibration image group sequence to the vehicle driving management terminal to confirm the vehicle's illegal driving.

[0103] The above-mentioned embodiments of the present disclosure have the following beneficial effects: through the vehicle-related information sending method of some embodiments of the present disclosure, the driving violation detection of the moving vehicle can be accurately and efficiently realized. Specifically, the reason for the inaccuracy of the relevant driving violation detection is that the vehicle driving video shot by the target camera is relatively one-sided and cannot fully and accurately determine the actual driving operation of the vehicle. As a result, the vehicle driving violation detection information output by the subsequent vehicle violation detection model is not accurate enough. Based on this, the vehicle-related information sending method of some embodiments of the present disclosure, first, in response to receiving multiple driving videos shot by multiple road camera devices, each of the multiple driving videos is subjected to frame extraction processing to generate a frame image sequence to obtain multiple frame image sequences. Among them, the multiple road camera devices are arranged at various positions of the corresponding road. Here, by shooting the driving conditions of the vehicle in all directions through multiple driving videos, the problem of inaccurate driving hazard detection caused by the one-sided shooting angle can be avoided. Then, according to the shooting time corresponding to each frame image, each frame image in the multiple frame image sequence is time-placed to generate a frame image group sequence. Among them, the shooting time corresponding to each frame image in the frame image group is the same. Here, by putting the frame images with the same shooting time in a group, a full range of driving behavior analysis can be performed on the vehicle object at the same time in the future, thereby ensuring the accuracy of the subsequent driving violation detection. Secondly, for each frame image group in the above frame image group sequence, the first vehicle object driving information set corresponding to the above frame image group can be accurately determined. Among them, there is a one-to-one correspondence between the first vehicle object driving information in the above first vehicle object driving information set and the vehicle object in the vehicle object set photographed by the road camera device at the target orientation at the target shooting time. Then, according to the above frame image group sequence, the vehicle driving behavior information corresponding to each vehicle object in the above vehicle object set can be accurately determined to obtain the vehicle driving behavior information set. Finally, for each vehicle object in the above vehicle object set, the following first generation step is performed: the first step is to accurately determine the vehicle object driving information sequence corresponding to the above vehicle object according to the obtained first vehicle object driving information set sequence, so as to facilitate the generation of subsequent driving compliance determination information. The second step is to accurately generate driving compliance confirmation information for the above vehicle object according to the above vehicle object driving information sequence and the above vehicle driving behavior information set. In the third step, in response to determining that the driving compliance confirmation information indicates that the vehicle object has illegal driving behavior, the vehicle object driving information sequence and the corresponding vehicle object driving calibration image group sequence are sent to the vehicle driving management terminal to confirm the vehicle's illegal driving. In summary, through all-round driving detection of multiple driving videos, driving violation detection of driving vehicles can be accurately and efficiently achieved.

[0104] Further references Figure 2As an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a vehicle-related information sending device, and these device embodiments are Figure 1 Corresponding to the method embodiments shown, the vehicle-related information sending device can be specifically applied to various electronic devices.

[0105] like Figure 2 As shown, a vehicle-related information sending device 200 includes: a frame extraction unit 201, a placement unit 202, a first determination unit 203, a second determination unit 204 and an execution unit 205. The frame extraction unit 501 is configured to, in response to receiving a plurality of driving videos shot by a plurality of road camera devices, perform frame extraction processing on each of the plurality of driving videos to generate a frame image sequence, thereby obtaining a plurality of frame image sequences, wherein the plurality of road camera devices are arranged at respective positions of the corresponding road; the placement unit 502 is configured to perform time placement on each of the plurality of frame image sequences according to the shooting time corresponding to each frame image to generate a frame image group sequence, wherein the shooting time corresponding to each frame image in the frame image group is the same; the first determination unit 503 is configured to determine, for each frame image group in the frame image group sequence, a first vehicle object driving information set corresponding to the frame image group, wherein the first vehicle object driving information set in the first vehicle object driving information set is consistent with the first vehicle object driving information set shot by the road camera device at the target position at the target shooting time. There is a one-to-one correspondence between the vehicle objects in the vehicle object set; the second determination unit 504 is configured to determine the vehicle driving behavior information corresponding to each vehicle object in the vehicle object set according to the frame image group sequence, and obtain the vehicle driving behavior information set; the execution unit 505 is configured to perform the following first generation step for each vehicle object in the vehicle object set: determine the vehicle object driving information sequence corresponding to the above vehicle object according to the obtained first vehicle object driving information set sequence; generate driving compliance confirmation information for the above vehicle object according to the above vehicle object driving information sequence and the above vehicle driving behavior information set; in response to determining that the above driving compliance confirmation information indicates that the above vehicle object has illegal driving behavior, send the above vehicle object driving information sequence and the corresponding vehicle object driving calibration image group sequence to the vehicle driving management terminal to confirm the illegal driving of the vehicle.

[0106] It can be understood that the various units recorded in the vehicle related information sending device 200 are similar to those in the reference Figure 1 Therefore, the operations, features and beneficial effects described above for the method are also applicable to the vehicle-related information sending device 200 and the units contained therein, and will not be described in detail here.

[0107] Reference below Figure 3, which shows a structural schematic diagram of an electronic device (eg, an electronic device) 300 suitable for implementing some embodiments of the present disclosure. Figure 3 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0108] like Figure 3 As shown, the electronic device 300 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the electronic device 300 are also stored. The processing device 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0109] Typically, the following devices may be connected to the I / O interface 305: input devices 306 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 308 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 309. The communication devices 309 may allow the electronic device 300 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 3 The electronic device 300 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 3 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0110] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 309, or installed from the storage device 308, or installed from the ROM 302. When the computer program is executed by the processing device 301, the above-mentioned functions defined in the method of some embodiments of the present disclosure are executed.

[0111] It should be noted that the computer-readable medium in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, the computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0112] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0113] The computer-readable medium may be included in the electronic device; or it may exist independently without being installed in the electronic device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: in response to receiving multiple driving videos shot by multiple road camera devices, performs frame extraction processing on each of the multiple driving videos to generate a frame image sequence, thereby obtaining multiple frame image sequences, wherein the multiple road camera devices are arranged at respective positions of the corresponding roads; according to the shooting time corresponding to each frame image, time-arranges each frame image in the multiple frame image sequences to generate a frame image group sequence, wherein the shooting time corresponding to each frame image in the frame image group is the same; for each frame image group in the frame image group sequence, determines the first vehicle object driving information set corresponding to the frame image group, wherein the first vehicle object driving information in the first vehicle object driving information set is consistent with the target shooting time. There is a one-to-one correspondence between the vehicle objects in the vehicle object set photographed by the road camera device at the target orientation under the time; according to the above-mentioned frame image group sequence, the vehicle driving behavior information corresponding to each vehicle object in the above-mentioned vehicle object set is determined to obtain a vehicle driving behavior information set; for each vehicle object in the above-mentioned vehicle object set, the following first generation step is performed: according to the obtained first vehicle object driving information set sequence, the vehicle object driving information sequence corresponding to the above-mentioned vehicle object is determined; according to the above-mentioned vehicle object driving information sequence and the above-mentioned vehicle driving behavior information set, driving compliance confirmation information for the above-mentioned vehicle object is generated; in response to determining that the above-mentioned driving compliance confirmation information represents that the above-mentioned vehicle object has illegal driving behavior, the above-mentioned vehicle object driving information sequence and the corresponding vehicle object driving calibration image group sequence are sent to the vehicle driving management terminal to confirm the illegal driving of the vehicle.

[0114] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0115] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0116] The units described in some embodiments of the present disclosure may be implemented by software or by hardware. The described units may also be provided in a processor, for example, may be described as: a processor including a frame extraction unit, a placement unit, a first determination unit, a second determination unit, and an execution unit. Among them, the names of these units do not constitute limitations on the units themselves in certain circumstances, for example, the placement unit may also be described as "a unit that performs time placement on each frame image sequence in the above-mentioned multiple frame image sequences according to the shooting time corresponding to each frame image, so as to generate a frame image group sequence".

[0117] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0118] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) and the technical solutions formed.

Claims

1. A method for sending vehicle-related information, comprising: In response to receiving a plurality of driving videos shot by a plurality of road camera devices, performing frame extraction processing on each of the plurality of driving videos to generate a frame image sequence, thereby obtaining a plurality of frame image sequences, wherein the plurality of road camera devices are arranged at respective positions of the corresponding road; According to the shooting time corresponding to each frame image, each frame image in the plurality of frame image sequences is time-arranged to generate a frame image group sequence, wherein the shooting time corresponding to each frame image in the frame image group is the same; For each frame image group in the frame image group sequence, determining a first vehicle object driving information set corresponding to the frame image group, wherein the first vehicle object driving information in the first vehicle object driving information set has a one-to-one correspondence with a vehicle object in a vehicle object set photographed by a road camera device at a target orientation at a target shooting time, wherein the first vehicle object driving information in the first vehicle object driving information set includes: vehicle object driving action information and a vehicle license plate; and The determining of the first vehicle object driving information set corresponding to the frame image group includes: For each vehicle object in the vehicle object set, the following second generation step is performed: Input each frame image in the frame image group into a vehicle object recognition model to generate vehicle recognition information to obtain a vehicle recognition information group, wherein the vehicle recognition information is recognition information for the vehicle object, wherein the vehicle recognition information includes: vehicle position information; Removing frame images corresponding to vehicle identification information indicating that the vehicle object is not identified from the frame image group, to obtain a post-removal frame image group; According to the obtained vehicle position information group, each image in the removed frame image group is enlarged and cropped to generate a processed image group, wherein the ratio of the number of pixels corresponding to the vehicle object in the processed image is not less than the target ratio; Using a pre-trained vehicle license plate detection model, generating a vehicle license plate detection information group for the processed image group; Generating a vehicle license plate corresponding to the vehicle object according to the vehicle license plate detection information group; Using a pre-trained vehicle object driving action detection information generation model, a vehicle object driving action detection information group is generated for the processed image group; According to the vehicle object driving action detection information group, vehicle object driving action information corresponding to the vehicle object is generated, wherein: The step of generating vehicle object driving action information corresponding to the vehicle object according to the vehicle object driving action detection information group includes: In response to determining that the vehicle object driving action detection information group contains detection information indicating that the vehicle object has dangerous actions, determining the orientation information and shooting angle corresponding to each of the plurality of road camera devices; Establishing a coordinate system with the road camera device at the target orientation as the coordinate origin; Performing coordinate transformation on the obtained multiple position information and multiple shooting angles to generate multiple transformed position information and multiple transformed shooting angles in the coordinate system; Inputting the plurality of converted position information, the plurality of converted shooting angles and the processed image group into a model for generating object driving action omnidirectional detection information to generate object driving action omnidirectional detection information, wherein the object driving action omnidirectional detection information includes: first probability information characterizing that the vehicle object has a dangerous action and second probability information characterizing that the vehicle object does not have a dangerous action; In response to determining that a value obtained by subtracting the first probability information from the second probability information is greater than a target value, generating vehicle object driving action information indicating that the vehicle object does not have a dangerous action; In response to determining that the value of the second probability information minus the first probability information is equal to or less than the target value, sending the vehicle object driving action detection information group, the object driving action omnidirectional detection information and the processed image group to the action detection terminal; receiving the vehicle object driving action information corresponding to the action detection terminal; According to the frame image group sequence, the vehicle driving behavior information corresponding to each vehicle object in the vehicle object set is determined to obtain a vehicle driving behavior information set, wherein: The determining, according to the frame image group sequence, the vehicle driving behavior information corresponding to each vehicle object in the vehicle object set comprises: Determine a frame image sequence group corresponding to the vehicle object, wherein each frame image sequence has a corresponding road shooting device; For each frame image sequence in the frame image sequence group, the following third generating step is performed: Get the behavior indicator set; For each frame image in the frame image sequence, input the frame image into a first behavior feature information generation model to generate first behavior feature information, wherein the first behavior feature information is a first behavior indicator information set corresponding to the behavior indicator set; Inputting each frame image in the frame image sequence into an image local feature extraction model to generate image local feature extraction information; Vectorizing each first behavior indicator information in the first behavior indicator information set corresponding to the first behavior characteristic information to generate a first behavior indicator vector set; Inputting the obtained image local feature extraction information sequence and the obtained first behavior indicator vector set sequence into a vehicle driving behavior information generation model based on a temporal neural network model to generate first vehicle driving behavior information; generating the vehicle driving behavior information according to the obtained first vehicle driving behavior information group; For each vehicle object in the vehicle object set, the following first generation step is performed: Determining a vehicle object driving information sequence corresponding to the vehicle object according to the obtained first vehicle object driving information set sequence; Generating driving compliance confirmation information for the vehicle object according to the vehicle object driving information sequence and the vehicle driving behavior information set; In response to determining that the driving compliance confirmation information indicates that the vehicle object has illegal driving behavior, the vehicle object driving information sequence and the corresponding vehicle object driving calibration image group sequence are sent to the vehicle driving management terminal to confirm the vehicle's illegal driving.

2. The method according to claim 1, wherein: The generating the vehicle driving behavior information according to the obtained first vehicle driving behavior information group includes: In response to determining that the first vehicle driving behavior information group contains first vehicle driving behavior information indicating that the vehicle is driving dangerously, obtaining at least one first target vehicle driving behavior information; at least one frame image sequence corresponding to the at least one first target vehicle driving behavior information; Temporally arranging the frame images in the at least one frame image sequence to generate a frame image subgroup sequence, wherein the corresponding time of the frame images in the frame image subgroup is the same; For each frame image subset in the frame image subset sequence, the following fourth generation step is performed: Inputting each frame image included in the frame image subset into a second behavior feature information generation model based on an attention mechanism model to generate second behavior feature information, wherein the second behavior feature information is a second behavior indicator information set corresponding to the behavior indicator set; vectorizing each second behavior indicator information in the second behavior indicator information set corresponding to the second behavior characteristic information to generate a second behavior indicator vector set; Determine the image local feature extraction information subgroup corresponding to the frame image subgroup; integrating the image local feature extraction information subgroups to generate integrated image local feature extraction information; Inputting the obtained integrated image local feature extraction information sequence and the obtained second behavior indicator vector set sequence into a vehicle driving behavior information generation model based on a temporal neural network model to generate second vehicle driving behavior information; The vehicle driving behavior information is generated according to the second vehicle driving behavior information.

3. The method according to claim 2, wherein: The step of sending the vehicle object driving information sequence and the corresponding vehicle object driving calibration image group sequence to a vehicle driving management terminal to confirm the vehicle's illegal driving includes: Determine a post-removal frame image group sequence and a corresponding vehicle identification information group sequence corresponding to the vehicle object; According to the vehicle identification information group sequence, each image in the removed frame image group sequence is calibrated with a bounding box to generate a vehicle object driving calibration image group, thereby obtaining a vehicle object driving calibration image group sequence; The vehicle object driving information sequence and the corresponding vehicle object driving calibration image group sequence are sent to the vehicle driving management terminal to confirm the vehicle's illegal driving.

4. The method according to claim 3, wherein: The step of generating a vehicle license plate corresponding to the vehicle object according to the vehicle license plate detection information group includes: The license plate detection information with the highest frequency of corresponding vehicle license plates is screened out from the vehicle license plate detection information group as the vehicle license plate corresponding to the vehicle object.

5. A vehicle-related information sending device, comprising: a frame extraction unit configured to, in response to receiving a plurality of driving videos shot by a plurality of road camera devices, perform frame extraction processing on each of the plurality of driving videos to generate a frame image sequence, thereby obtaining a plurality of frame image sequences, wherein the plurality of road camera devices are arranged at respective positions of the corresponding road; a placement unit configured to perform time placement on each frame image in the plurality of frame image sequences according to a shooting time corresponding to each frame image, so as to generate a frame image group sequence, wherein the shooting time corresponding to each frame image in the frame image group is the same; The first determining unit is configured to determine, for each frame image group in the frame image group sequence, a first vehicle object driving information set corresponding to the frame image group, wherein the first vehicle object driving information in the first vehicle object driving information set has a one-to-one correspondence with a vehicle object in a vehicle object set photographed by a road camera device at a target orientation at a target shooting time, wherein the first vehicle object driving information in the first vehicle object driving information set includes: vehicle object driving action information and a vehicle license plate; and the determining of the first vehicle object driving information set corresponding to the frame image group includes: for each vehicle object in the vehicle object set, executing the following second generating step: converting the first vehicle object driving information in the frame image group into a vehicle object set; Each frame image is input into a vehicle object recognition model to generate vehicle recognition information to obtain a vehicle recognition information group, wherein the vehicle recognition information is recognition information for the vehicle object, wherein the vehicle recognition information includes: vehicle position information; removing frame images corresponding to the vehicle recognition information representing that the vehicle object is not recognized from the frame image group to obtain a removed frame image group; according to the obtained vehicle position information group, performing vehicle object target magnification and cropping processing on each image in the removed frame image group to generate a processed image group, wherein the ratio of the number of content pixels corresponding to the vehicle object in the processed image is not less than the target ratio; using a pre-trained vehicle license plate detection model, generating a vehicle license plate detection group for the processed image. The method comprises: generating a vehicle license plate detection information group for the processed image group; generating a vehicle license plate corresponding to the vehicle object according to the vehicle license plate detection information group; generating a vehicle object driving action detection information group for the processed image group using a pre-trained vehicle object driving action detection information generation model; generating vehicle object driving action information corresponding to the vehicle object according to the vehicle object driving action detection information group, wherein the generating vehicle object driving action information corresponding to the vehicle object according to the vehicle object driving action detection information group includes: in response to determining that the vehicle object driving action detection information group contains detection information indicating that the vehicle object has a dangerous action, determining that the plurality of road camera devices The method comprises the following steps: obtaining the position information and shooting angle corresponding to each road camera device of the target position; establishing a coordinate system with the road camera device of the target position as the coordinate origin; performing coordinate transformation on the obtained multiple position information and multiple shooting angles to generate multiple transformed position information and multiple transformed shooting angles in the coordinate system; inputting the multiple transformed position information, the multiple transformed shooting angles and the processed image group into the object driving action omnidirectional detection information generation model to generate the object driving action omnidirectional detection information, wherein the object driving action omnidirectional detection information includes: first probability information characterizing the existence of dangerous actions of the vehicle object and second probability information characterizing the absence of dangerous actions of the vehicle object;In response to determining that the value of the second probability information minus the first probability information is greater than a target value, generating vehicle object driving action information that indicates that the vehicle object does not have a dangerous action; in response to determining that the value of the second probability information minus the first probability information is equal to or less than the target value, sending the vehicle object driving action detection information group, the object driving action omnidirectional detection information and the processed image group to an action detection terminal; and receiving the vehicle object driving action information corresponding to the action detection terminal; The second determination unit is configured to determine the vehicle driving behavior information corresponding to each vehicle object in the vehicle object set according to the frame image group sequence, and obtain a vehicle driving behavior information set, wherein the determination of the vehicle driving behavior information corresponding to each vehicle object in the vehicle object set according to the frame image group sequence includes: determining a frame image sequence group corresponding to the vehicle object, wherein each frame image sequence has a corresponding road shooting device; for each frame image sequence in the frame image sequence group, performing the following third generation step: obtaining a behavior indicator set; for each frame image in the frame image sequence, inputting the frame image into a first behavior feature information generation model to generate a first behavior feature information, wherein the first behavior feature information is the first behavior indicator information set corresponding to the behavior indicator set; each frame image in the frame image sequence is input into an image local feature extraction model to generate image local feature extraction information; each first behavior indicator information in the first behavior indicator information set corresponding to the first behavior feature information is vectorized to generate a first behavior indicator vector set; the obtained image local feature extraction information sequence and the obtained first behavior indicator vector set sequence are input into a vehicle driving behavior information generation model based on a temporal neural network model to generate first vehicle driving behavior information; the vehicle driving behavior information is generated according to the obtained first vehicle driving behavior information group; The execution unit is configured to perform the following first generation step for each vehicle object in the vehicle object set: determine the vehicle object driving information sequence corresponding to the vehicle object based on the obtained first vehicle object driving information set sequence; generate driving compliance confirmation information for the vehicle object based on the vehicle object driving information sequence and the vehicle driving behavior information set; in response to determining that the driving compliance confirmation information represents the existence of illegal driving behavior of the vehicle object, send the vehicle object driving information sequence and the corresponding vehicle object driving calibration image group sequence to the vehicle driving management terminal to confirm the vehicle's illegal driving.

6. An electronic device comprising: one or more processors; a storage device having one or more programs stored thereon, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 4.

7. A computer readable medium having a computer program stored thereon, wherein: When the program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images