Data processing method, data processing device, electronic equipment and storage medium
Through the sensors and cameras of autonomous vehicles, they can determine the accident situation, identify the traffic police's actions, decide whether to wait for the traffic police to arrive, and collect and upload data at the accident site, solving the problem of low timeliness of autonomous vehicles accident handling and improving the accident Speed and accuracy of processing.
Patent Information
- Application Number
- CN202411086604.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-08
- Publication Date
- 2024-11-26
AI Technical Summary
After a traffic accident occurs in an unmanned vehicle, especially when there is no monitoring intersection or traffic police site, the timeliness of accident handling are low, and the unmanned vehicle may drive away from the accident site, resulting in difficulty in processing data and affecting the accuracy of the accident. and efficiency.
Provide a data processing method and device, including a state judgment module, an action recognition module and a data processing module, through the sensors and cameras of the unmanned vehicle itself, judge the accident situation, extract picture feature information, identify traffic police actions, calculate power, and determine Whether to wait for the traffic police to arrive and collect and upload data at the accident site to ensure the timely handling of accident information.
Improve the speed and efficiency of unmanned vehicle accident handling, ensure that videos and environmental data at the accident site are saved and uploaded in a timely manner, simplify the evidence collection process of traffic police, improve the accuracy of accident handling and the public's handling of unmanned vehicle accidents satisfaction.
Smart Images

Figure CN119028132A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data processing technology, and in particular to a data processing method and a data processing device, an electronic device, and a storage medium. Background Art
[0002] At present, with the continuous advancement of unmanned driving technology, various vehicles equipped with various laser sensors, cameras, sound recorders and other equipment can identify and detect static and dynamic objects around them based on various sensors, allowing vehicles to travel on urban roads.
[0003] For example, driverless cars and driverless delivery vehicles are running in many cities across the country, and accidents may follow, such as collisions between driverless delivery vehicles and people riding electric bikes, collisions between driverless cars and other vehicles, etc. However, from time to time, some driverless vehicles will ignore the instructions of traffic police comrades after accidents and drive away from the accident site.
[0004] At this time, it is very troublesome for some parties involved in traffic accidents to deal with the matter. In some sections of the road where there is no monitoring intersection, when there is no image of the parties involved in the accident, in order to ensure the correctness of the accident, the traffic police usually need to retrieve the accident image data from the unmanned vehicle deployment unit, and these additional procedures will greatly reduce the timeliness of accident handling. Especially when there are traffic police at the scene of the accident, the traffic police can obviously collect evidence on the spot, but if the unmanned vehicle drives away, it will be very troublesome.
[0005] Therefore, when an accident involving an unmanned vehicle occurs, timely processing of data becomes very important. Summary of the invention
[0006] To this end, the present invention provides a data processing method and a data processing device, an electronic device, and a storage medium to solve the above-mentioned problems in the prior art.
[0007] In order to achieve the above object, the present invention provides the following technical solutions:
[0008] A data processing method provided according to an embodiment of the present invention comprises the following steps:
[0009] S1. When the unmanned vehicle brakes suddenly, determine whether the unmanned vehicle has collided; if no collision occurs, continue to drive according to the established rules and tasks;
[0010] S2. When a vehicle collides, the unmanned vehicle will first perform a self-check and determine whether the vehicle can drive normally based on the self-check status; if it cannot drive normally, the accident information will be reported, an alarm will be notified to the backend staff, and the unmanned vehicle accident plan will be executed. At the same time, the corresponding data will be uploaded to the cloud or saved in the vehicle's own data storage disk;
[0011] S3. When the vehicle is able to drive normally, the unmanned vehicle will use its own sensors and cameras to determine whether the vehicle has collided with a vehicle, pedestrian or cyclist based on the images captured by the camera, and detect whether there are people lying on the ground and whether they are injured;
[0012] S4. Extract image feature information based on the frame image of the captured video, and determine whether there is a traffic police nearby based on the identity template feature information; at the same time, report the situation of injuries and notify the back-end staff to call the police and medical rescue;
[0013] S5. When it is determined that there is no traffic police nearby, directly extract the vehicle's driving route information, including the vehicle's location, the remaining mileage of the task route, and the task timeliness requirements, and read the vehicle's drivable mileage data in real time. Based on the accident scene waiting time template and the vehicle's power consumption when waiting at the scene, calculate the power consumption of waiting at the scene, and calculate the subsequent drivable mileage to determine whether the vehicle still has enough power to support the vehicle to travel the remaining task mileage after the waiting process at the accident scene is completed;
[0014] S6. When the vehicle detects the traffic police information, it will further identify the traffic police's actions. Here, it is mainly based on the action attention aggregation module MAA to identify the traffic police's actions;
[0015] S7. The vehicle staying at the accident scene cooperates with the traffic police to collect evidence, identifies the meaning of the traffic police's actions according to the action template, and simultaneously collects and identifies the traffic police's voice information;
[0016] S8. After the traffic police completes the evidence collection, they will leave the accident scene, archive the environmental data and video of the scene, and report the accident information.
[0017] Further: In step S3, if no one is injured, it is further determined whether the vehicle is only slightly scratched. If so, the vehicle directly leaves the accident scene, archives the scene environment data and video, and reports the accident information.
[0018] Further: in the step S4, the image features are extracted by first extracting the person's identity image, and then extracting the scale-invariant feature SIFT from the identity image. The SIFT algorithm includes constructing a scale space, detecting the extreme points of the Gaussian difference scale space DOG, removing the feature points with very asymmetric local curvature in the Gaussian difference scale space, assigning a 128-dimensional directional parameter to the feature points, and generating key point descriptors.
[0019] Further: in the step S5, when the remaining power does not support the vehicle to wait at the accident scene for a long time, the vehicle reports the accident information, executes the unmanned vehicle accident plan, and retains or uploads the corresponding data to the cloud;
[0020] When the remaining circuits can support the vehicle to wait at the scene for a long time, the vehicle stays at the accident scene.
[0021] Further: the MAA module in step S6 can adaptively adjust the direction and amplitude of the channel displacement. The MAA module replaces the 1D time convolution fixed filter in the time series shift TSM module with a 3×1×1 learnable filter.
[0022] The network structure of the MAA module first inputs a tensor as a motion-strong correlation feature sequence Y, Y= Split the channels into 4 equal groups, each of which has a shape of [T, C / 4, H, W], where represents the domain set, C is the number of channels, T is the time dimension, H and W are the spatial dimensions;
[0023] The 1D convolution and 2D convolution in the TSM module are decomposed into 4 sub-convolutions Y1, Y2, Y3 and Y4 respectively, and the 4 convolutions are assigned to 4 groups of channels respectively. The number of parameters of the sub-convolutions is 1 / 4 of the original;
[0024] No processing is done on the subconvolution Y1 of the first group, while the 1D time subconvolution and 2D space subconvolution on the channel dimension are applied to the 2nd to 4th groups in turn, and the features of the output of the previous group are added element by element before the subconvolution is applied to the 3rd to 4th groups, thus forming a pyramid structure, whose mathematical expression is:
[0025]
[0026] in, represents the output features of the i-th group, conv1 represents the 1D temporal sub-convolution of the channel dimension, and its convolution kernel size is 3×1×1, conv2 represents the 2D spatial sub-convolution, and its convolution kernel size is 1×3×3, W1 and W2 represent the convolution kernel parameters of 1D temporal sub-convolution and 2D spatial sub-convolution, respectively.
[0027] Furthermore, the output features of each group contain different spatiotemporal information. In order to fuse the feature information of all groups, different weights are assigned to different output features, and the attention mechanism is used to adaptively aggregate these features. The attention network structure is as follows: Figure 3 As shown, the output features of the four groups are input and And add the output features of each group element by element to get the aggregate feature
[0028]
[0029] A global average pool is applied along the channel to compress the spatial feature aggregation, and then a fully connected layer t is used to compress the channel. Then, a fully connected layer is used to expand the channel to the original number 4, and then the channel is evenly divided into 4 tensors A1, A2, A3 and A4. Finally, cross-channel soft attention is applied to these four tensors to dynamically select different spatial scales for each channel of each group of output features to obtain the attention coefficient of each group of outputs. After multiplying each group of feature tensors and the corresponding attention coefficient, the spatiotemporal feature Z output by the attention network is obtained through a cascade operation.
[0030]
[0031] Among them, α1, α2, α3 and α4 represent the attention coefficients of the four groups of channels respectively, and the obtained spatiotemporal feature Z dynamically integrates the spatiotemporal features of different time distances.
[0032] The present invention discloses a data processing device, including a state judgment module, an action recognition module and a data processing module, wherein the state judgment module can receive and extract the vehicle state information of the unmanned vehicle itself to judge whether the vehicle can travel normally, and at the same time, the state judgment module judges whether there are injured people and traffic police near the vehicle based on cameras and other sensors;
[0033] The action recognition module is used to recognize the actions of the traffic police so as to understand the meaning of the traffic police's gestures and cooperate with the traffic police to take photos or videos on the scene to collect evidence;
[0034] The data processing module is used to process the data processed by the action recognition module and compare the traffic police action template data, so as to further improve the meaning of the traffic police command or action; at the same time, the data processing module can also encapsulate the scene environment video and audio data, and store or upload them, so as to effectively preserve the evidence of the accident scene;
[0035] In addition, the data processing module can receive operation requests from back-end staff. For driverless taxis, the data processing module can also save data from the in-vehicle cameras.
[0036] An embodiment of the present invention application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the processor implements the steps in the data processing method described in any of the above embodiments when executing the program.
[0037] An embodiment of the present invention application provides a storage medium, in which a plurality of instructions are stored. The instructions are suitable for a processor to load to execute the steps in the data processing method described in any of the above embodiments.
[0038] The present invention has the following advantages: the data processing method of the present invention can process whether the unmanned vehicle is waiting for the arrival of the traffic police at the scene when a collision occurs, and extract and further process the image information in the video frame of the accident scene, and identify the meaning of the traffic police's action based on the traffic police's action template data; it can improve the public's favorability towards the handling of unmanned vehicle accidents and improve the speed of evidence collection in the process of traffic accident handling; it can greatly facilitate the traffic police to handle the accident problems of unmanned vehicles. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] Figure 1 The present invention provides a flowchart of a data processing method.
[0040] Figure 2 It is a network structure diagram of the MAA module in the data processing method of the present invention.
[0041] Figure 3 This is a diagram of the attention network structure in the data processing method of the present invention.
[0042] Figure 4 A block diagram of the module structure of a data processing device provided in an embodiment of the present invention.
[0043] Figure 5 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention.
[0044] In the figure: 10, processor; 20, memory; 30, bus; 40, communication interface; 50, input / output interface. DETAILED DESCRIPTION
[0045] The following is an explanation of the implementation of the present invention by specific specific examples. People familiar with this technology can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. Obviously, the described embodiments are part of the embodiments of the present invention, not all of them. It should be understood that these embodiments are only to further illustrate the present invention and cannot be understood as limiting the scope of protection of the present invention. Technical engineers in this field can make some non-essential improvements and adjustments to the present invention based on the contents of the above invention; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0046] like Figure 1 As shown, the data processing method specifically includes the following steps:
[0047] S1. When the unmanned vehicle brakes suddenly, analyze the reason for the sudden brake and determine whether the unmanned vehicle has collided; if no collision occurs, continue to drive according to the established rules and tasks.
[0048] S2. When a vehicle collides, the unmanned vehicle will first perform a self-inspection and determine whether the vehicle can drive normally based on the self-inspection status; if it cannot drive normally, the accident information will be reported, an alarm will be notified to the backend staff, and the unmanned vehicle accident plan will be executed. At the same time, the corresponding data will be uploaded to the cloud or saved in the vehicle's own data storage disk.
[0049] S3. When the vehicle is able to drive normally, the unmanned vehicle will use its own sensors and cameras to determine whether the vehicle has collided with a vehicle, pedestrian or cyclist based on the images captured by the camera, and detect whether there are people lying on the ground and whether they are injured;
[0050] If no one is injured, further determine whether the vehicle is only slightly scratched. If so, drive away from the accident scene directly, archive the on-site environmental data and video, and report the accident information.
[0051] S4. Extract image feature information based on the frame images of the captured video, and determine whether there is a traffic police nearby based on the identity template feature information; at the same time, report any injuries and notify the backend staff to call the police and medical assistance.
[0052] The extraction of the image features first extracts the identity image of the person, and then extracts the scale-invariant feature SIFT from the identity image. SIFT is an algorithm for detecting local features. The algorithm obtains features by finding feature points in an image and their related scale and orientation descriptors, and performs a process of matching image feature points. The SIFT feature is not only scale-invariant, but also can achieve good detection results even if the rotation angle, image brightness or shooting angle is changed. The SIFT algorithm mainly includes constructing a scale space, detecting extreme points of the Gaussian difference scale space DOG, removing feature points with very asymmetric local curvature in the Gaussian difference scale space, assigning a 128-dimensional direction parameter to the feature points, and generating key point descriptors.
[0053] When the vehicle is not slightly scratched, the first step is to determine whether there is a traffic police at the scene.
[0054] S5. When it is determined that there is no traffic police nearby, directly extract the vehicle's driving route information, including the vehicle's location, the remaining mileage of the task route, the task timeliness requirements, etc., and read the vehicle's drivable mileage data in real time. Based on the accident scene waiting time template and the vehicle's power consumption when waiting at the scene, calculate the power consumption of waiting at the scene, and calculate the subsequent drivable mileage to determine whether the vehicle still has enough power to support the vehicle to travel the remaining task mileage after the waiting process at the accident scene is completed;
[0055] When the remaining power does not allow the vehicle to wait at the accident scene for a long time, the vehicle reports the accident information, executes the unmanned vehicle accident plan, and retains or uploads the corresponding data to the cloud;
[0056] When the remaining circuits can support the vehicle to wait at the scene for a long time, the vehicle stays at the accident scene.
[0057] S6. When the vehicle detects traffic police information, it will further identify the traffic police's actions. Here, it is mainly based on the Motion Attention Aggregation (MAA) network, that is, the MAA module, to identify the traffic police's actions.
[0058] In the current computational tasks of action recognition, many methods use a local temporal convolution to process the information of adjacent picture frames. In order to model long-distance motion features, it is necessary to further increase the network depth and stack more local convolutions. However, after stacking more local convolutions, the important information transmitted by the long-span picture frames will be weakened, making it impossible to efficiently process the long-distance motion feature information.
[0059] The MAA module can adaptively adjust the direction and amplitude of channel displacement. The MAA module replaces the 1D time convolution fixed filter in the Temporal Shift Module (TSM) module with a 3×1×1 learnable filter.
[0060] like Figure 2 The network structure of the MAA module shown in the figure first inputs a tensor as a sequence of motion-strongly correlated features. Split the channel into 4 equal groups, each of which has a shape of [T, C / 4, H, W], where represents the domain set, C is the number of channels, T is the time dimension, H and W are the spatial dimensions;
[0061] The 1D convolution and 2D convolution in the TSM module are decomposed into 4 sub-convolutions Y1, Y2, Y3 and Y4 respectively, and the 4 convolutions are assigned to 4 groups of channels respectively. The number of parameters of the sub-convolutions is 1 / 4 of the original;
[0062] No processing is done on the subconvolution Y1 of the first group, while the 1D time subconvolution and 2D space subconvolution on the channel dimension are applied to the 2nd-4th groups in turn, and the features of the output of the previous group are added element by element before the subconvolution is applied to the 3rd-4th groups, thus forming a pyramid structure. As the features of the previous groups are fused by the lateral connection of the latter groups, their temporal receptive field will increase exponentially. This pyramid structure not only expands the temporal receptive field, but also strengthens the information fusion of different groups. Its mathematical expression is:
[0063]
[0064] in, represents the output features of the i-th group, conv1 represents the 1D temporal sub-convolution of the channel dimension, and its convolution kernel size is 3×1×1, conv2 represents the 2D spatial sub-convolution, and its convolution kernel size is 1×3×3, W1 and W2 represent the convolution kernel parameters of 1D temporal sub-convolution and 2D spatial sub-convolution, respectively.
[0065] The output features of each group contain different spatiotemporal information. In order to fuse the feature information of all groups, different weights are assigned to different output features, and the attention mechanism is used to adaptively aggregate these features. The attention network structure is as follows: Figure 3 As shown, the output features of the four groups are input and And add the output features of each group element by element to get the aggregate feature
[0066] A global average pool is applied along the channel to compress the spatial feature aggregation, and then a fully connected layer t is used to compress the channel. Then, a fully connected layer is used to expand the channel to the original number 4, and then the channel is evenly divided into 4 tensors A1, A2, A3 and A4. Finally, cross-channel soft attention is applied to these four tensors to dynamically select different spatial scales for each channel of each group of output features to obtain the attention coefficient of each group of outputs. After multiplying each group of feature tensors and the corresponding attention coefficient, the spatiotemporal feature Z output by the attention network is obtained through a cascade operation. Among them, α1, α2, α3 and α4 represent the attention coefficients of the four groups of channels respectively, and the obtained spatiotemporal feature Z dynamically integrates the spatiotemporal features of different time distances.
[0067] The attention mechanism used by the MAA module aggregates the spatiotemporal features of four groups. The temporal receptive fields of the features of each group are inconsistent, and the span of the temporal information concerned is also different. The span of long-distance spatiotemporal modeling required for different video actions is also different. The attention mechanism can rely on data characteristics to learn more accurate spatiotemporal features for different video actions and achieve better recognition effects.
[0068] S7. Vehicles staying at the accident scene cooperate with the traffic police to collect evidence, identify the meaning of the traffic police's actions according to the action template, and simultaneously collect and identify the traffic police's voice information, so as to easily understand the traffic police's command information, making the handling of accidents easier.
[0069] S8. After the traffic police completes the evidence collection, they will leave the accident scene, archive the environmental data and video of the scene, and report the accident information.
[0070] In actual operation, especially when there is no monitoring or road maintenance, when a collision occurs, the environment around the vehicle will be collected and the data will be uploaded directly to the cloud. Because driverless vehicles will strictly enforce local traffic rules and keep records before going on the road, the accidents handled are all emergencies that cannot be avoided. Therefore, when the vehicle is severely damaged and the above operation steps cannot be performed, the back-end monitoring will display signal loss or terminal, triggering a background alarm.
[0071] like Figure 4 As shown, a data processing device in this embodiment includes a state judgment module, an action recognition module and a data processing module, wherein the state judgment module can receive and extract the vehicle state information of the unmanned vehicle itself to judge whether the vehicle can travel normally, and the state judgment module can also judge whether there are injured people or traffic police near the vehicle based on cameras and other sensors;
[0072] The action recognition module is used to recognize the actions of the traffic police so as to understand the meaning of the traffic police's gestures and cooperate with the traffic police to take photos or videos on the scene to collect evidence;
[0073] The data processing module is used to process the data processed by the action recognition module and compare it with the traffic police action template data, so as to further improve the meaning of the traffic police's command or action; at the same time, the data processing module can also encapsulate the on-site environment video, audio and other data, and perform operations such as storage or uploading, which can effectively preserve the evidence at the accident scene.
[0074] In addition, the data processing module can also receive operation requests from back-end staff. For driverless taxis, the data processing module can also save data from the in-vehicle cameras.
[0075] like Figure 5 As shown, an electronic device provided in this embodiment includes a processor 10, a memory 20, a bus 30, a communication interface 40 and an input / output interface 50; the processor 10, the memory 20, the communication interface 40 and the input / output interface 50 are connected to each other through the bus 30 within the device.
[0076] The processor 10 may be implemented by using an application-specific integrated circuit or one or more integrated circuits to implement the technical solution provided in the embodiment of the present application;
[0077] The memory 20 is suitable for storing instructions or programs that are known to the processor 10. The memory 20 can store software organizations, such as operating systems, communication modules, interaction modules, and application programs.
[0078] The communication interface 40 is used to realize communication interaction between the device and other devices, and communication can be realized in a wired manner or a wireless manner.
[0079] The input / output interface 50 is used to implement information input and output.
[0080] The above-mentioned flowchart and / or block diagram of the method, device / system and computer program product according to the embodiment of the present invention describe various aspects of the present invention.It should be understood that each block of the flowchart and / or block diagram and the combination of the blocks in the flowchart legend and / or block diagram can be realized by computer program instructions.These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer or other programmable data processing device to produce a machine so that the instruction executed via the processor of a computer or other programmable data processing device is created to realize the flowchart and / or block diagram block or block.
[0081] As a non-transitory computer-readable storage medium, the memory can be used to store non-transitory software programs and non-transitory computer executable programs. In addition, the memory may include a high-speed random access memory and may also include a non-transitory memory; such as at least one disk storage device, a flash memory device or other non-transitory solid-state storage device. The memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network and a combination thereof.
[0082] A storage medium provided in an embodiment of the present invention is a computer-readable storage medium used for computer-readable storage. The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the above-mentioned data processing method.
[0083] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.
Claims
1. A data processing method, characterized in that: The following steps are involved: S1. When the unmanned vehicle brakes suddenly, determine whether the unmanned vehicle has collided; if no collision occurs, continue to drive according to the established rules and tasks; S2. When a vehicle collides, the unmanned vehicle first performs a self-check and determines whether the vehicle can drive normally based on the self-check status; If the vehicle cannot drive normally, it will report the accident information, alert the back-end staff, and execute the unmanned vehicle accident plan. At the same time, the corresponding data will be uploaded to the cloud or saved in the vehicle's own data storage disk; S3. When the vehicle is able to drive normally, the unmanned vehicle will use its own sensors and cameras to determine whether the vehicle has collided with a vehicle, pedestrian or cyclist based on the images captured by the camera, and detect whether there are people lying on the ground and whether they are injured; S4. Extract image feature information based on the frame image of the captured video, and determine whether there is a traffic police nearby based on the identity template feature information; at the same time, report the situation of injuries and notify the back-end staff to call the police and medical rescue; S5. When it is determined that there is no traffic police nearby, directly extract the vehicle's driving route information, including the vehicle's location, the remaining mileage of the task route, and the task timeliness requirements, and read the vehicle's drivable mileage data in real time. Based on the accident scene waiting time template and the vehicle's power consumption when waiting at the scene, calculate the power consumption of waiting at the scene, and calculate the subsequent drivable mileage to determine whether the vehicle still has enough power to support the vehicle to travel the remaining task mileage after the waiting process at the accident scene is completed; S6. When the vehicle detects the traffic police information, it will further identify the traffic police's actions. Here, it is mainly based on the action attention aggregation module MAA to identify the traffic police's actions; S7. The vehicle staying at the accident scene cooperates with the traffic police to collect evidence, identifies the meaning of the traffic police's actions according to the action template, and simultaneously collects and identifies the traffic police's voice information; S8. After the traffic police completes the evidence collection, they will leave the accident scene, archive the environmental data and video of the scene, and report the accident information.
2. The data processing method according to claim 1, characterized in that: In step S3, if no one is injured, it is further determined whether the vehicle is slightly scratched. If so, the vehicle directly leaves the accident scene, archives the scene environment data and video, and reports the accident information.
3. The data processing method according to claim 1, characterized in that: In step S4, the image features are extracted by first extracting the identity image of the person, and then extracting the scale-invariant feature SIFT from the identity image. The SIFT algorithm includes constructing a scale space, detecting the extreme points of the Gaussian difference scale space DOG, removing the feature points with very asymmetric local curvature in the Gaussian difference scale space, assigning a 128-dimensional directional parameter to the feature points, and generating key point descriptors.
4. The data processing method according to claim 1, characterized in that: In step S5, when the remaining power does not allow the vehicle to wait at the accident scene for a long time, the vehicle reports the accident information, executes the unmanned vehicle accident plan, and retains or uploads the corresponding data to the cloud; When the remaining circuits can support the vehicle to wait at the scene for a long time, the vehicle stays at the accident scene.
5. The data processing method according to claim 1, characterized in that: The MAA module in step S6 can adaptively adjust the direction and amplitude of the channel displacement. The MAA module replaces the 1D time convolution fixed filter in the time series shift TSM module with a 3×1×1 learnable filter.
6. The data processing method according to claim 5, characterized in that: The network structure of the MAA module first inputs a tensor as a sequence of motion-strongly correlated features. Split the channels into 4 equal groups, each of which has a shape of [T, C / 4, H, W], where represents the domain set, C is the number of channels, T is the time dimension, H and W are the spatial dimensions; The 1D convolution and 2D convolution in the TSM module are decomposed into 4 sub-convolutions Y1, Y2, Y3 and Y4 respectively, and the 4 convolutions are assigned to 4 groups of channels respectively. The number of parameters of the sub-convolutions is 1 / 4 of the original; No processing is done on the subconvolution Y1 of the first group, while the 1D time subconvolution and 2D space subconvolution on the channel dimension are applied to the 2nd to 4th groups in turn, and the features of the output of the previous group are added element by element before the subconvolution is applied to the 3rd to 4th groups, thus forming a pyramid structure, whose mathematical expression is: in, represents the output features of the i-th group, conv1 represents the 1D temporal sub-convolution of the channel dimension, and its convolution kernel size is 3×1×1, conv2 represents the 2D spatial sub-convolution, and its convolution kernel size is 1×3×3, W1 and W2 represent the convolution kernel parameters of 1D temporal sub-convolution and 2D spatial sub-convolution, respectively.
7. The data processing method according to claim 6, characterized in that: The output features of each group contain different spatiotemporal information. In order to fuse the feature information of all groups, different weights are assigned to different output features, and the attention mechanism is used to adaptively aggregate these features. The attention network structure is shown in Figure 3. The output features of the four groups are input and And add the output features of each group element by element to get the aggregate feature A global average pool is applied along the channel to compress the spatial feature aggregation, and then a fully connected layer t is used to compress the channel. Then, a fully connected layer is used to expand the channel to the original number 4, and then the channel is evenly divided into 4 tensors A1, A2, A3 and A4. Finally, cross-channel soft attention is applied to these four tensors to dynamically select different spatial scales for each channel of each group of output features to obtain the attention coefficient of each group of outputs. After multiplying each group of feature tensors and the corresponding attention coefficient, the spatiotemporal feature Z output by the attention network is obtained through a cascade operation. Among them, α1, α2, α3 and α4 represent the attention coefficients of the four groups of channels respectively, and the obtained spatiotemporal feature Z dynamically integrates the spatiotemporal features of different time distances.
8. A data processing device, characterized in that: It includes a state judgment module, an action recognition module and a data processing module. The state judgment module can receive and extract the vehicle state information of the unmanned vehicle to determine whether the vehicle can drive normally. At the same time, the state judgment module determines whether there are injured people or traffic police near the vehicle based on cameras and other sensors. The action recognition module is used to recognize the actions of the traffic police so as to understand the meaning of the traffic police's gestures and cooperate with the traffic police to take photos or videos on the scene to collect evidence; The data processing module is used to process the data processed by the action recognition module and compare the traffic police action template data, so as to further improve the meaning of the traffic police command action; at the same time, the data processing module can also encapsulate the scene environment video and audio data, and perform storage and upload operations, which can effectively preserve the evidence of the accident scene; In addition, the data processing module can receive operation requests from back-end staff. For driverless taxis, the data processing module can also save data from the in-vehicle cameras.
9. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and capable of running on the processor, wherein the processor implements the steps in the data processing method according to any one of claims 1 to 7 when executing the program.
10. A storage medium, characterized in that: The storage medium stores a plurality of instructions, which are suitable for loading by a processor to execute the steps in the data processing method according to any one of claims 1 to 7.
