Traffic anomaly identification method, device and equipment, and storage medium

Abstract

Embodiments of the present application disclose a traffic anomaly recognition method, device and equipment, and a storage medium, and specifically disclose the following: acquiring real-time traffic video collected by a vehicle-mounted driving recorder; inputting the real-time traffic video into a pre-trained lightweight deep neural network model to obtain a traffic anomaly recognition result output by the model; wherein the lightweight deep neural network model comprises a feature extraction network, a feature fusion and enhancement network and a classification recognition head connected in sequence; the feature extraction network is used to extract multi-scale feature maps from input images; the feature fusion and enhancement network receives the multi-scale feature maps output by the feature extraction network, performs feature fusion from top to bottom through a feature pyramid network, and fuses multi-scale context information through a dilated spatial pyramid pooling operation to output enhanced multi-scale fusion feature maps; and the classification recognition head outputs a recognition result of a traffic anomaly based on the enhanced multi-scale fusion feature maps.

Description

Technical Field

[0001] This invention relates to the field of anomaly recognition technology, and in particular to a method, apparatus, device, and storage medium for traffic anomaly recognition. Background Technology

[0002] Currently, the identification of abnormal traffic situations on highways typically relies on fixed-installation monitoring equipment. Specifically, this involves deploying cameras or imaging systems at intervals along highway sections to capture vehicle speed, lane-changing behavior, or road events using visible or infrared light. The captured information from localized areas is then processed by an intelligent analysis module to identify abnormal traffic situations.

[0003] Existing identification methods relying on fixed monitoring equipment have significant drawbacks: First, the initial investment in hardware is high, and its identification range is strictly limited by physical location, only able to capture anomalies in localized areas near the equipment. This prevents continuous, full-coverage real-time monitoring of the entire highway section, resulting in numerous blind spots and low identification efficiency. Second, it lacks sufficient analytical and identification capabilities for distant, small-scale targets, leading to low accuracy. Furthermore, the ability of existing solutions to detect anomalies is limited by the spatial layout of the equipment and the frequency of inspections, making it unable to effectively and promptly identify and warn of sudden incidents such as collisions, rockfalls, and tire blowouts on highways. Summary of the Invention

[0004] The main objective of this invention is to provide a traffic anomaly identification method, device, equipment, and storage medium, which aims to solve the problems of high hardware deployment costs, limited identification range, and insufficient ability to identify and warn of remote small targets and emergencies caused by the complete reliance on fixed-installation monitoring equipment in existing high-speed anomaly identification methods.

[0005] In a first aspect, embodiments of the present invention provide a method for identifying traffic anomalies, including: Acquire real-time traffic video captured by a vehicle dashcam; The real-time traffic video is input into a pre-trained lightweight deep neural network model to obtain the traffic anomaly recognition result output by the model. The lightweight deep neural network model includes a feature extraction network, a feature fusion and enhancement network, and a classification head connected in sequence. The feature extraction network consists of convolutional layers, inverted residual block layers, and dilated convolutional-attention layers, used to extract multi-scale feature maps from the input image. The inverted residual block layer adopts a depthwise separable convolutional structure, including a first inverted residual block sub-layer with an embedded attention mechanism module and a second inverted residual block sub-layer without an embedded attention mechanism module. The feature fusion and enhancement network receives the multi-scale feature maps output by the feature extraction network, performs top-down feature fusion through a feature pyramid network, and fuses multi-scale contextual information through dilated spatial pyramid pooling to output an enhanced multi-scale fused feature map. The classification head outputs the recognition result of the traffic anomaly based on the enhanced multi-scale fused feature map.

[0006] Secondly, embodiments of the present invention provide a traffic anomaly identification device, comprising: The acquisition module is used to acquire real-time traffic video collected by the vehicle-mounted dashcam; The recognition module is used to input the real-time traffic video into a pre-trained lightweight deep neural network model to obtain the traffic anomaly recognition result output by the model. The lightweight deep neural network model includes a feature extraction network, a feature fusion and enhancement network, and a classification recognition head connected in sequence. The feature extraction network consists of convolutional layers, inverted residual block layers, and dilated convolutional-attention layers, used to extract multi-scale feature maps from the input image. The inverted residual block layer adopts a depthwise separable convolutional structure, including a first inverted residual block sub-layer with an embedded attention mechanism module and a second inverted residual block sub-layer without an embedded attention mechanism module. The feature fusion and enhancement network receives the multi-scale feature maps output by the feature extraction network, performs top-down feature fusion through a feature pyramid network, and fuses multi-scale contextual information through dilated spatial pyramid pooling to output an enhanced multi-scale fused feature map. The classification recognition head outputs the recognition result of the traffic anomaly based on the enhanced multi-scale fused feature map.

[0007] Thirdly, embodiments of the present invention provide an electronic device, including: a processor; and a memory configured to store computer-executable instructions, which, when executed, cause the processor to perform the steps of the method described in the first aspect above.

[0008] Fourthly, embodiments of the present invention provide a computer-readable storage medium for storing computer-executable instructions that, when executed by a processor, implement the steps of the method described in the first aspect above.

[0009] Fifthly, embodiments of the present invention provide a computer program product, the computer program product including a computer program, which, when executed by a processor, implements the steps of the method described in the first aspect above.

[0010] The at least one technical solution provided by the embodiments of the present invention can achieve the following technical effects: In this embodiment of the invention, real-time traffic video collected by a vehicle-mounted dashcam can be acquired first, and then input into a pre-trained lightweight deep neural network model for processing. This model sequentially includes a feature extraction network, a feature fusion and enhancement network, and a classification and recognition head. The feature extraction network consists of convolutional layers, inverted residual block layers, and dilated convolutional attention layers, used to extract multi-scale feature maps from the input image. The inverted residual block layers employ a depthwise separable convolutional structure and include two types of sublayers: one with an embedded attention mechanism module and one without. The feature fusion and enhancement network receives these multi-scale feature maps and performs top-down feature fusion through a feature pyramid network, effectively combining the rich semantic information of deep features with the precise location information of shallow features. Furthermore, it fuses multi-scale contextual information through dilated spatial pyramid pooling, ultimately outputting an enhanced multi-scale fused feature map. The classification and recognition head, based on the enhanced multi-scale fused feature map, outputs the recognition result for traffic anomalies.

[0011] This invention effectively solves the core problems of existing technologies that rely on fixed monitoring equipment, such as limited recognition range, numerous blind spots, high deployment costs, and insufficient ability to identify and warn of distant small targets and emergencies, by constructing an automated recognition link that uses video captured by a mobile vehicle dashcam as input and an end-to-end lightweight deep neural network as its core. This invention utilizes widely available vehicles as mobile sensing units to achieve continuous and dynamic monitoring of the entire highway, fundamentally solving the geographical limitations of fixed cameras. Simultaneously, by employing a lightweight network model, real-time processing capabilities on vehicle-mounted embedded devices are ensured, significantly reducing deployment costs. Furthermore, the model internally uses a carefully designed feature extraction network to acquire multi-scale features and utilizes a feature fusion and enhancement mechanism combining feature pyramid networks and hollow spatial pyramid pooling to significantly improve the perception and recognition accuracy of distant small-scale abnormal targets, thereby overcoming the technical shortcomings of existing solutions in complex scenarios, such as insufficient analysis and recognition capabilities and low accuracy. Attached Figure Description

[0012] Figure 1 This is one of the flowcharts illustrating a traffic anomaly identification method provided in an embodiment of the present invention; Figure 2 This is one of the scenario diagrams illustrating a traffic anomaly identification method provided in an embodiment of the present invention; Figure 3 This is a second scenario illustration of a traffic anomaly identification method provided in an embodiment of the present invention; Figure 4 The third scenario diagram illustrates a traffic anomaly identification method provided in an embodiment of the present invention. Figure 5 The fourth scenario diagram illustrates a traffic anomaly identification method provided in an embodiment of the present invention. Figure 6 Fifth scenario illustration of a traffic anomaly identification method provided in an embodiment of the present invention; Figure 7 A sixth scenario illustration of a traffic anomaly identification method provided in an embodiment of the present invention; Figure 8 This is the seventh scenario illustration of a traffic anomaly identification method provided in an embodiment of the present invention; Figure 9 This is the eighth scenario illustration of a traffic anomaly identification method provided in an embodiment of the present invention; Figure 10 A schematic diagram of a traffic anomaly identification method provided in an embodiment of the present invention, number nine; Figure 11 A schematic diagram of the module composition of a traffic anomaly identification device 1100 provided in one embodiment of the present invention; Figure 12 This is a schematic diagram of the hardware structure of an electronic device provided in one embodiment of the present invention. Detailed Implementation

[0013] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0014] The technical solutions provided by the various embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0015] Please see Figure 1 , Figure 1This is one of the flowcharts illustrating a traffic anomaly identification method provided in an embodiment of the present invention, such as... Figure 1 As shown, the method includes the following steps: Step 102: Obtain real-time traffic video captured by the vehicle's dashcam.

[0016] Step 104: Input the real-time traffic video into a pre-trained lightweight deep neural network model to obtain the traffic anomaly recognition result output by the model. The lightweight deep neural network model includes a feature extraction network, a feature fusion and enhancement network, and a classification head connected sequentially. The feature extraction network consists of convolutional layers, inverted residual block layers, and dilated convolutional-attention layers, used to extract multi-scale feature maps from the input image. The inverted residual block layer adopts a depthwise separable convolutional structure, including a first inverted residual block sub-layer with an embedded attention mechanism module and a second inverted residual block sub-layer without an embedded attention mechanism module. The feature fusion and enhancement network receives the multi-scale feature maps output by the feature extraction network, performs top-down feature fusion through a feature pyramid network, and fuses multi-scale contextual information through dilated spatial pyramid pooling operations, outputting an enhanced multi-scale fused feature map. The classification head, based on the enhanced multi-scale fused feature map, outputs the recognition result for the traffic anomaly.

[0017] In one embodiment of the invention, real-time traffic video collected by a vehicle-mounted dashcam can be acquired first. This video is typically continuously recorded by a dashcam installed in a moving vehicle, capturing real-time traffic information on highways. The video content can cover the entire field of view directly in front of the vehicle, including lane markings, surrounding vehicles, road conditions, road signs, and any unusual objects such as fallen cargo, potholes, or accident scenes. The dashcam can capture visual data of traffic ahead, covering a range from tens to over one hundred meters, providing raw input for subsequent anomaly detection. In practical applications, the video data can be transmitted in a streaming manner to ensure real-time performance.

[0018] After acquiring real-time traffic video from a vehicle-mounted dashcam, the pre-processed real-time traffic video frames can be input into a pre-trained lightweight deep neural network model for inference calculations, thereby obtaining the identification results of traffic anomalies. This lightweight deep neural network model fully considers the requirements of the in-vehicle environment for computing resources, power consumption, and real-time performance, and adopts a carefully designed lightweight architecture, which consists of three core parts connected in series: a feature extraction network, a feature fusion and enhancement network, and a classification and recognition head.

[0019] Among them, the feature extraction network, as the basic front-end of the model, is mainly responsible for extracting multi-level feature maps with rich semantic information from the input single-frame image. For example... Figure 2As shown on the left, this feature extraction network has a profound hierarchical structure, consisting of various types of network layers such as convolutional layers, IRB (Inverted Residual Block) layers, and dilated convolutional-attention layers. Through this complex structure with alternating depths and complementary functions, it can gradually extract abstract feature representations from the original input image from low to high levels.

[0020] The input image first enters the first convolutional layer, which uses a 3×3 kernel with a stride of 2 to perform dense convolution operations on the input image. Its main purpose is to initially extract basic visual features from the image, such as edges, corners, and textures. Simultaneously, the stride of 2 significantly reduces the spatial size of the feature map, decreasing subsequent computation. Immediately after convolution processing, batch normalization is performed. This operation normalizes each mini-batch of data, effectively accelerating the convergence speed during model training and improving the model's generalization ability on unknown data. Then, a hard-swish activation function is used in this layer to introduce non-linear transformation capabilities.

[0021] The core feature extraction part of the feature extraction network consists of multiple stacked IRB layers. The IRB layers employ an advanced depthwise separable convolutional structure to significantly reduce the number of model parameters and floating-point operations. Each depthwise separable convolution operation can be clearly decomposed into two sequential steps: first, depthwise convolution, which independently applies a spatial convolution kernel to each input channel of the input feature map, specifically responsible for capturing the unique spatial feature patterns within each channel; then, pointwise convolution, which uses a 1×1 convolution kernel to perform cross-channel information fusion and dimensionality transformation on the feature map processed by depthwise convolution. This ingenious way of decomposing standard convolution into two independent steps greatly reduces computational cost and model complexity.

[0022] In this embodiment of the invention, the inverted residual block layer can specifically include two subtypes: a first inverted residual block sublayer with an embedded attention mechanism module, which can be called an IRB-SE (Squeeze and Excitation) layer, and a second inverted residual block sublayer without an embedded attention mechanism module, which can be called a standard IRB layer. The SE attention module is an efficient channel attention mechanism. Its working principle is to compress the spatial information of each two-dimensional feature channel into a scalar with a global receptive field through a global average pooling operation. Then, this scalar sequence is passed through a small neural network composed of two fully connected layers to learn the importance weight of each feature channel. Finally, the weights are normalized to a reasonable range between 0 and 1 through the Sigmoid function, and then multiplied with the original feature map at the channel level. This allows the model to adaptively emphasize information-rich feature channels and suppress channels with less information, thereby achieving feature recalibration.

[0023] To achieve the best balance between performance and efficiency, the model uses different activation functions for different types of IRB layers: the first inverted residual block sublayer IRB-SE uses the Hard-swish activation function; while the second inverted residual block sublayer standard IRB uses the HardMish activation function.

[0024] In one example, such as Figure 3 As shown in the figure, the differences in characteristics and derivative behavior between the HardMish activation function and the classic ReLU activation function are illustrated in detail through curve comparison graphs. Figure 3 The graph shows the output curves of HardMish and ReLU in two key regions: the negative and positive regions. It's clear that the HardMish function is continuous and smooth throughout its domain, especially near the origin, where its curve transitions smoothly and its gradient is stable. In the negative region, unlike ReLU which directly outputs zero, HardMish retains the small negative response, and its derivative is continuously non-zero in the negative region. The derivative curve on the right further reveals the essential difference: ReLU's derivative abruptly drops to zero when the input is less than zero, meaning negatively activated neurons cannot update their weights during training; while HardMish's derivative smoothly descends from a positive value to zero in the negative region. This characteristic provides a path for gradient backpropagation, significantly mitigating the vanishing gradient problem common in deep neural networks, thus making the model training process more stable and efficient, while maintaining a computational speed comparable to ReLU, making it ideal for deployment in automotive embedded devices with limited computing power.

[0025] Furthermore, the feature extraction network cleverly incorporates dilated convolutional attention layers at specific key layers, such as layers 5, 9, and 13. This layer is a core module specifically designed to address the challenging problem of motion blur caused by the rapid movement of targets in high-speed driving environments.

[0026] In one example, such as Figure 4 and Figure 5 As shown in the diagram, the internal data flow and component composition of the dilated convolution-attention layer are clearly illustrated in the form of a structural block diagram. This layer is a composite module, consisting of two core sub-modules: a dilated convolution module and a spatial attention module, connected sequentially. The structure of the dilated convolution module is as follows: Figure 4 As shown, the structure of the spatial attention module is as follows: Figure 5 As shown. The input feature map first flows into the dilated convolution module, which performs dilated convolution processing on the input feature map with a dilation rate greater than 1. Dilated convolution is an advanced convolution variant that inserts spaces (controlled by the dilation rate d) between standard convolution kernel elements, thereby significantly expanding the receptive field of the convolution kernel without increasing the number of additional parameters or computational cost. The actual size of the receptive field can be quantized using the following formula:

[0027] Where k is the basic size of the convolution kernel; d is the set dilation rate. By increasing the dilation rate d, each convolution operation can perceive more contextual information in the input image without losing the spatial resolution of the feature map. This helps to understand the trajectory of high-speed moving targets, infer their intentions, and suppress the single-frame blur effect caused by motion.

[0028] Then, the feature maps with larger receptive fields after dilated convolution are immediately fed into the spatial attention module for further optimization.

[0029] In one example, such as Figure 6The diagram illustrates the internal workings of the spatial attention module: two global pooling operations are performed in parallel—global average pooling (calculating the average value at each position in the feature map) and global max pooling (extracting the maximum value at each position in the feature map)—resulting in two descriptor vectors that represent global spatial context information at different levels. These two descriptors are then concatenated along the channel dimension to form a more comprehensive spatial context representation. This concatenated vector is then passed through a one-dimensional convolutional layer for feature fusion and dimensionality reduction. Finally, a sigmoid activation function generates a spatial attention weight mask with values ​​ranging from 0 to 1, corresponding to the spatial dimension of the input feature map. The spatial attention module receives the feature map after dilated convolution and generates a spatial attention weight mask based on it. This spatial attention weight mask is then weighted and fused (element-wise multiplication) with the dilated convolution-processed feature map to output an enhanced feature map. This process enables the model to mimic the human visual attention mechanism, adaptively focusing on areas in the image that are more critical to the recognition task, such as the area where moving vehicles are located or potential obstacles, while suppressing responses to irrelevant or interfering backgrounds, thereby significantly improving the model's target recognition accuracy and robustness in high-speed, dynamic, and blurred scenes.

[0030] Through this hierarchical processing, the feature extraction network gradually decreases the spatial size of the feature maps while gradually increasing the number of channels from beginning to end, ultimately outputting a set of feature maps containing multiple scales. Shallow feature maps have higher resolution and retain rich details such as edges, corners, and textures, which are very helpful for the accurate localization of small objects. While deep feature maps have lower spatial resolution, they contain stronger high-level semantic information, which helps the model understand complex scene contexts and accurately identify large objects.

[0031] The Feature Fusion and Enhancement Network (FPN), as the middle part of the model, plays a crucial role in efficiently integrating multi-scale information. It receives multi-scale feature maps output from the front-end of the feature extraction network and performs two key operations. First, it performs top-down feature fusion through the Feature Pyramid Network (FPN). The FPN employs a top-down path structure with lateral connections. The specific fusion process is as follows: In the top-down path, the feature map from the deepest layer (richest in semantic information) of the feature extraction network is upsampled. This upsampling is typically done using nearest neighbor interpolation or transposed convolution to double its spatial resolution. The upsampled feature map is then fused with the corresponding shallow feature map of the same spatial resolution in the bottom-up path. This fusion is usually done element-wise. The fused feature map is then upsampled again to further increase its resolution and fused with the next-level feature map of the corresponding resolution in the bottom-up path. The resolution of the next-level feature map is higher than that of the adjacent shallow feature maps used in the previous fusion step. Repeat the upsampling and fusion steps, fusing the feature maps with progressively increasing resolutions along the bottom-up path, until the feature maps at the bottom layer (i.e., the highest resolution) are also fused.

[0032] This recursive fusion process can be formally represented as:

[0033] in, This represents the feature map of the i-th layer after fusion; This represents the result of upsampling the feature map of the (i+1)th deeper layer of the feature map. represents the feature map of the i-th layer in the bottom-up path; Conv represents a convolutional operation used to smooth the fusion result. In this way, FPN successfully combines high-resolution but shallow features containing weak semantic information (good at localization) with low-resolution but deep features containing strong semantic information (good at recognition).

[0034] To further capture rich contextual information, the fused feature map is input into ASPP (Atrous Spatial Pyramid Pooling). ASPP fuses multi-scale contextual information through atrous spatial pyramid pooling operations. It uses multiple parallel dilated convolutional branches with different dilation rates to simultaneously capture contextual information at different scales, from local details to the global scene. Finally, the outputs of each branch are aggregated to obtain a more comprehensive understanding of the target's surrounding environment.

[0035] In one example, such as Figure 7As shown in the diagram, the detailed architecture combining FPN and ASPP is clearly illustrated. Figure 7 The left side clearly depicts the top-down, step-by-step fusion process of FPN with shallow features, while the right side details how the ASPP module extracts multi-scale features through four parallel branches, including three dilated convolution branches with different dilation rates and a global average pooling branch. Finally, the outputs of these branches are concatenated along the channel dimension and integrated through a 1×1 convolution to output an enhanced multi-scale fused feature map. This powerful combination of FPN and ASPP significantly improves the perception and recognition accuracy of distant, small-scale anomalies, such as obstacles hundreds of meters away.

[0036] As the final stage of the model, the classification and recognition head can output the recognition results of traffic anomalies based on the enhanced multi-scale fusion feature map, through a series of convolutional layers, possibly fully connected layers, and finally a Softmax or Sigmoid classifier. These results can be classified into specific categories such as "normal", "falling rocks", "vehicle collision", "roadblock construction", and "cargo falling". The head can also output the bounding boxes of the target locations at the same time.

[0037] To further enhance the ability to understand the temporal sequence of continuous dynamic scenes, rather than simply processing isolated static images, the model is designed to handle continuous video frame sequences. Specifically, multiple consecutive frames, such as 5 or 10 frames, are extracted from real-time traffic videos at certain time intervals; these frames are then input as a sequence into the model. The model models the temporal relationships between these frames by increasing their receptive field and incorporating a temporal attention mechanism, thereby improving the accuracy of identifying high-speed moving targets. The temporal attention mechanism allows the model to dynamically focus on the time steps (i.e., keyframes) most relevant to the current recognition task within the sequence, and its attention weights are calculated using the scaled dot product attention formula.

[0038] In this formula, These are the query, key, and value matrices generated from a continuous frame feature sequence through a linear transformation. It is the dimension of the key vector, divided by It serves to stabilize the gradient. This mechanism allows the model to focus on the coherent changes of moving targets between different frames, just like "tracking," effectively assigning lower weights to blurred frames, thereby significantly improving the robustness of recognition in complex scenarios such as high-speed movement and image shakiness.

[0039] In one example, such as Figure 2The diagram illustrates the complete lightweight deep neural network model architecture. It integrates all the aforementioned core components and clearly shows the complete data flow path from left to right: the input image first enters the feature extraction network (containing an initial convolutional layer, multiple IRB layers, IRB-SE layers, and dilated convolutional-attention layers at key locations). The extracted multi-scale feature maps are fed into the feature fusion and enhancement network, fused from top to bottom through the FPN, then enhanced by the ASPP module, and finally output by the classification head, indicating the target category and location.

[0040] In this embodiment of the invention, after the lightweight deep neural network model outputs the traffic anomaly identification result, the reporting process can be initiated immediately. Specifically, reporting can be performed through an integrated 5G private network communication module. When the model detects anomalies such as rockfalls or collisions, it generates a structured alarm message. This message typically includes key data such as anomaly type, confidence level, geographic location coordinates, timestamp, and vehicle identifier. The 5G private network communication module can be pre-configured with specific access strategies and can access the network through a dedicated network slice and a customized data network name. In high-speed traffic scenarios, a non-standalone 5G private network can be used, where the 5G wireless access network and the 4G core network coexist. Existing operator 5G to B core network elements are utilized, reducing the cost of independent network construction through sharing.

[0041] In one example, such as Figure 8 As shown, a 5G network communication architecture is illustrated. Figure 8 In this scenario, a car and a mobile phone, acting as terminal devices, access the network through a 5G base station, and their emitted signals carry a private network identifier. This 5G site is shared by both the private and public networks, achieving infrastructure reuse. The signals received by the base station are transmitted to the user plane function equipment. The UPF (User Plane Function, an independent functional entity in the 5G core network responsible for user plane data processing) undertakes the critical tasks of packet forwarding and policy formulation. Starting from the UPF, the data flow is clearly divided into two paths: one path forwards public network data to the public internet; the other path, according to the policy, securely transmits data carrying the private network identifier through the private network data forwarding channel to the enterprise's internal service system, thereby achieving logical isolation and targeted transmission of public and private network data.

[0042] When an onboard ETC terminal device enters a highway and passes through a toll station, its private network service can be activated. When an alarm message needs to be sent, the terminal can send a data request through a 5G base station. The user plane function network elements in the core network can identify that the request is destined for the private network and forward it directly to the highway management company's internal private cloud or data center via security protocols such as GRE tunneling, rather than the public network. This effectively ensures the security and privacy of sensitive data and achieves logical isolation between public and private network data.

[0043] Upon receiving an alarm message from a vehicle, the cloud server can perform two operations: broadcasting a warning message and dispatching an inspection task. First, the cloud service can immediately broadcast the anomaly to other connected vehicles within a certain range behind the incident location via a dedicated 5G network. This broadcast information can be integrated into the vehicle's infotainment system or navigation system, using sound, images, or vibration to alert drivers behind to slow down or change lanes in advance, enabling proactive avoidance. Second, the cloud service will simultaneously push detailed information about the anomaly, including its precise location, on-site photos, or video clips, to the highway management center or the mobile terminals of the nearest inspection personnel. This allows inspection personnel to obtain information about the scene immediately, quickly respond, and prevent secondary accidents, thereby improving the overall safety and efficiency of highway traffic.

[0044] In one example, such as Figure 9 As shown, a panoramic application scenario diagram of intelligent traffic monitoring and emergency response based on 5G private network is presented. Figure 9 Along the mountain road, there are ordinary vehicles traveling, engineering vehicles on duty, and intelligent monitoring equipment installed on the roadside. These mobile and fixed sensing units can be closely connected to the network through the ubiquitous 5G private network signal to perceive various traffic conditions in real time, whether it is vehicle collisions, road construction, natural rockfalls, or emergencies requiring fire fighting. Figure 9 It can clearly display a three-dimensional, full-road-coverage perception network, thus completely breaking the geographical limitations of fixed cameras. This makes the detection of anomalies no longer dependent on accidental monitoring points, effectively improving the detection efficiency of traffic anomaly identification.

[0045] In one example, such as Figure 10 As shown, this illustrates the complete technology chain and architecture from perception, analysis, decision-making to response. Figure 10It can be seen that after a vehicle equipped with a smart terminal triggers the ETC service at a toll station, the video data it collects is transmitted directly to the cloud via a 5G private network. Simultaneously, it can be analyzed in real time by an edge computing unit integrating a lightweight target detection model and a data interaction module. The resulting abnormal alarm information and key data are then exchanged in real time with cloud services via the 5G network. After completing information aggregation and decision-making, the cloud can issue instructions in two directions simultaneously: first, by broadcasting warning information to relevant vehicles via the 5G network to achieve vehicle-road coordination and proactive safety; and second, by accurately assigning handling tasks to nearby inspection personnel, forming a closed-loop management system.

[0046] In this embodiment of the invention, real-time traffic video collected by a vehicle-mounted dashcam can be acquired first, and then input into a pre-trained lightweight deep neural network model for processing. This model sequentially includes a feature extraction network, a feature fusion and enhancement network, and a classification and recognition head. The feature extraction network consists of convolutional layers, inverted residual block layers, and dilated convolutional attention layers, used to extract multi-scale feature maps from the input image. The inverted residual block layers employ a depthwise separable convolutional structure and include two types of sublayers: one with an embedded attention mechanism module and one without. The feature fusion and enhancement network receives these multi-scale feature maps and performs top-down feature fusion through a feature pyramid network, effectively combining the rich semantic information of deep features with the precise location information of shallow features. Furthermore, it fuses multi-scale contextual information through dilated spatial pyramid pooling, ultimately outputting an enhanced multi-scale fused feature map. The classification and recognition head, based on the enhanced multi-scale fused feature map, outputs the recognition result for traffic anomalies.

[0047] This invention effectively solves the core problems of existing technologies that rely on fixed monitoring equipment, such as limited recognition range, numerous blind spots, high deployment costs, and insufficient ability to identify and warn of distant small targets and emergencies, by constructing an automated recognition link that uses video captured by a mobile vehicle dashcam as input and an end-to-end lightweight deep neural network as its core. This invention utilizes widely available vehicles as mobile sensing units to achieve continuous and dynamic monitoring of the entire highway, fundamentally solving the geographical limitations of fixed cameras. Simultaneously, by employing a lightweight network model, real-time processing capabilities on vehicle-mounted embedded devices are ensured, significantly reducing deployment costs. Furthermore, the model internally uses a carefully designed feature extraction network to acquire multi-scale features and utilizes a feature fusion and enhancement mechanism combining feature pyramid networks and hollow spatial pyramid pooling to significantly improve the perception and recognition accuracy of distant small-scale abnormal targets, thereby overcoming the technical shortcomings of existing solutions in complex scenarios, such as insufficient analysis and recognition capabilities and low accuracy.

[0048] Figure 11The traffic anomaly recognition device 1100 shown can achieve Figure 1 The method described in the embodiment achieves the same technical effect, and can be specifically referred to in the above description. Figure 1 The traffic anomaly identification method of the illustrated embodiment will not be described in detail here. The traffic anomaly identification device 1100 includes: The acquisition module 1101 is used to acquire real-time traffic video collected by the vehicle-mounted dashcam; The recognition module 1102 is used to input the real-time traffic video into a pre-trained lightweight deep neural network model to obtain the traffic anomaly recognition result output by the model. The lightweight deep neural network model includes a feature extraction network, a feature fusion and enhancement network, and a classification head connected in sequence. The feature extraction network consists of convolutional layers, inverted residual block layers, and dilated convolutional-attention layers, used to extract multi-scale feature maps from the input image. The inverted residual block layer adopts a depthwise separable convolutional structure, including a first inverted residual block sub-layer with an embedded attention mechanism module and a second inverted residual block sub-layer without an embedded attention mechanism module. The feature fusion and enhancement network receives the multi-scale feature maps output by the feature extraction network, performs top-down feature fusion through a feature pyramid network, and fuses multi-scale contextual information through dilated spatial pyramid pooling to output an enhanced multi-scale fused feature map. The classification head outputs the recognition result for the traffic anomaly based on the enhanced multi-scale fused feature map.

[0049] Optionally, the activation functions used in the lightweight deep neural network model include the HardMish activation function and the Hard-swish activation function; wherein the first inverted residual block sub-layer uses the Hard-swish activation function; and the second inverted residual block sub-layer uses the HardMish activation function.

[0050] Optionally, the dilated convolution-attention layer includes a dilated convolution module and a spatial attention module; The dilated convolution module is used to perform dilated convolution processing on the input feature map with a dilation rate greater than 1, so as to expand the receptive field of the input feature map while maintaining the spatial resolution of the input feature map, and output the feature map after dilated convolution processing. The spatial attention module is used to receive the feature map after the dilated convolution processing, generate a spatial attention weight mask based on the feature map after the dilated convolution processing, and perform weighted fusion of the spatial attention weight mask with the feature map after the dilated convolution processing to output an enhanced feature map.

[0051] Optionally, the identification module 1102 is used for: Extract consecutive multi-frame images from the real-time traffic video; The continuous multi-frame images are input into the model; wherein the model models the temporal correlation between the continuous multi-frame images by increasing the receptive field of the model and combining a temporal attention mechanism, so as to improve the recognition accuracy of high-speed moving targets.

[0052] Optionally, the identification module 602 is used for: In the top-down path, the deepest feature map from the feature extraction network is upsampled; The upsampled feature map is then fused with the corresponding shallow feature maps of the same resolution from the bottom up path. The fused feature map is upsampled again and fused with the next-level feature map of the corresponding resolution in the bottom-up path; the resolution of the next-level feature map is higher than the resolution of the adjacent shallow feature map. Repeat the upsampling and fusion steps, fusing the feature maps with progressively increasing resolution from bottom to top along the path, until fusion is completed with the lowest feature map.

[0053] Optionally, the device further includes ( Figure 11 (not shown in the image) The alarm module 1103 is used to upload alarm information containing the traffic anomaly identification result to the cloud server through an integrated 5G private network communication module after the traffic anomaly identification result is obtained from the model output; wherein, the 5G private network communication module accesses the 5G private network based on non-standalone networking mode through a contracted private network slice and a customized data network name, and establishes a secure connection with the internal network of the traffic management party.

[0054] In this embodiment of the invention, real-time traffic video captured by a vehicle-mounted dashcam can be acquired and input into a pre-trained lightweight deep neural network model for processing. This model sequentially includes a feature extraction network, a feature fusion network, and a target recognition head. The feature extraction network, through a structure of alternating convolutional and bottleneck layers, extracts multi-scale feature maps from a single frame of the video, containing high-resolution shallow feature maps and highly semantic deep feature maps. The feature fusion network receives these multi-scale feature maps, upsamples the deep feature maps to match their resolution with the corresponding shallow feature maps, and fuses them, thereby effectively combining the rich semantic information of deep features with the precise location information of shallow features, outputting a multi-scale fused feature map. The target recognition head receives the fused feature map and extracts contextual information at different scales based on spatial pyramid pooling operations, ultimately outputting the recognition result for traffic anomalies.

[0055] This invention effectively solves the core problems of existing technologies, such as limited recognition range, numerous blind spots, high deployment costs, and insufficient ability to identify small targets at a distance, caused by reliance on fixed monitoring equipment. It constructs an automated recognition link using video captured by a mobile vehicle-mounted dashcam as input and an end-to-end lightweight deep neural network as its core. This invention utilizes widely available vehicles as mobile sensing units, achieving continuous and dynamic monitoring of the entire highway, fundamentally overcoming the geographical limitations of fixed cameras. Simultaneously, the use of a lightweight network model ensures real-time processing capabilities on embedded vehicle devices, significantly reducing system deployment costs. More importantly, the model employs a carefully designed multi-scale feature extraction and fusion mechanism, combining high-semantic deep features with high-resolution shallow features, greatly enhancing the perception and recognition accuracy of small-scale anomalies at a distance. This overcomes the technical shortcomings of existing solutions, such as insufficient analysis and recognition capabilities and low accuracy in complex scenarios.

[0056] Figure 12 This is a schematic diagram of the structure of an electronic device provided in one embodiment of the present invention. Please refer to it. Figure 12 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and memory. The memory may include main memory, such as high-speed random-access memory (RAM), or non-volatile memory, such as at least one disk drive. Of course, the electronic device may also include other hardware required for other business operations.

[0057] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 12 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0058] Memory is used to store programs. Specifically, programs may include program code, which includes computer operation instructions. Memory may include main memory and non-volatile memory, and provides instructions and data to the processor.

[0059] The processor reads the corresponding computer program from non-volatile memory into main memory and then executes it, forming a non-contiguous transfer configuration at the logical level. The processor executes the program stored in memory and specifically performs the following operations: Acquire real-time traffic video captured by a vehicle dashcam; The real-time traffic video is input into a pre-trained lightweight deep neural network model to obtain the traffic anomaly recognition result output by the model. The lightweight deep neural network model includes a feature extraction network, a feature fusion and enhancement network, and a classification head connected in sequence. The feature extraction network consists of convolutional layers, inverted residual block layers, and dilated convolutional-attention layers, used to extract multi-scale feature maps from the input image. The inverted residual block layer adopts a depthwise separable convolutional structure, including a first inverted residual block sub-layer with an embedded attention mechanism module and a second inverted residual block sub-layer without an embedded attention mechanism module. The feature fusion and enhancement network receives the multi-scale feature maps output by the feature extraction network, performs top-down feature fusion through a feature pyramid network, and fuses multi-scale contextual information through dilated spatial pyramid pooling to output an enhanced multi-scale fused feature map. The classification head outputs the recognition result of the traffic anomaly based on the enhanced multi-scale fused feature map.

[0060] The above is as described in the present invention. Figure 1The traffic anomaly identification method disclosed in the embodiments described above can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in one or more embodiments of the present invention. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in one or more embodiments of the present invention can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0061] The electronic device can also perform Figure 1 The method for identifying traffic anomalies described herein will not be elaborated further here.

[0062] This invention also provides a computer-readable storage medium that stores one or more programs, the programs including instructions that, when executed by a portable electronic device including multiple applications, enable the portable electronic device to perform... Figure 1 The method of the illustrated embodiment will not be described in detail here.

[0063] This invention also provides a computer program product stored in a storage medium and executed by at least one processor to implement... Figure 1 The method of the illustrated embodiment will not be described in detail here.

[0064] Of course, in addition to the software implementation, the electronic device of the present invention does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc. That is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0065] In summary, the above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of one or more embodiments of the present invention should be included within the scope of protection of one or more embodiments of the present invention.

[0066] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0067] Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined in this embodiment of the invention, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0068] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0069] The various embodiments in this invention are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

Claims

1. A method for identifying traffic anomalies, characterized in that, include: Acquire real-time traffic video captured by a vehicle-mounted dashcam; The real-time traffic video is input into a pre-trained lightweight deep neural network model to obtain the traffic anomaly recognition result output by the model. The lightweight deep neural network model includes a feature extraction network, a feature fusion and enhancement network, and a classification head connected in sequence. The feature extraction network consists of convolutional layers, inverted residual block layers, and dilated convolutional-attention layers, used to extract multi-scale feature maps from the input image. The inverted residual block layer adopts a depthwise separable convolutional structure, including a first inverted residual block sub-layer with an embedded attention mechanism module and a second inverted residual block sub-layer without an embedded attention mechanism module. The feature fusion and enhancement network receives the multi-scale feature maps output by the feature extraction network, performs top-down feature fusion through a feature pyramid network, and fuses multi-scale contextual information through dilated spatial pyramid pooling to output an enhanced multi-scale fused feature map. The classification head outputs the recognition result of the traffic anomaly based on the enhanced multi-scale fused feature map.

2. The method according to claim 1, characterized in that, The activation functions used in the lightweight deep neural network model include the HardMish activation function and the Hard-swish activation function; wherein, the first inverted residual block sub-layer uses the Hard-swish activation function; and the second inverted residual block sub-layer uses the HardMish activation function.

3. The method according to claim 1, characterized in that, The dilated convolution-attention layer includes a dilated convolution module and a spatial attention module; The dilated convolution module is used to perform dilated convolution processing on the input feature map with a dilation rate greater than 1, so as to expand the receptive field of the input feature map while maintaining the spatial resolution of the input feature map, and output the feature map after dilated convolution processing. The spatial attention module is used to receive the feature map after the dilated convolution processing, generate a spatial attention weight mask based on the feature map after the dilated convolution processing, and perform weighted fusion of the spatial attention weight mask with the feature map after the dilated convolution processing to output an enhanced feature map.

4. The method according to claim 1, characterized in that, The step of inputting the real-time traffic video into a pre-trained lightweight deep neural network model includes: Extract consecutive multi-frame images from the real-time traffic video; The continuous multi-frame images are input into the model; wherein the model models the temporal correlation between the continuous multi-frame images by increasing the receptive field of the model and combining a temporal attention mechanism, so as to improve the recognition accuracy of high-speed moving targets.

5. The method according to claim 1, characterized in that, The top-down feature fusion via a feature pyramid network includes: In the top-down path, the deepest feature map from the feature extraction network is upsampled; The upsampled feature map is then fused with the corresponding shallow feature maps of the same resolution from the bottom up path. The fused feature map is upsampled again and fused with the next-level feature map of the corresponding resolution in the bottom-up path; the resolution of the next-level feature map is higher than the resolution of the adjacent shallow feature map. Repeat the upsampling and fusion steps, fusing the feature maps with progressively increasing resolution from bottom to top along the path, until fusion is completed with the lowest feature map.

6. The method according to claim 1, characterized in that, After obtaining the traffic anomaly identification result output by the model, the method further includes: The integrated 5G private network communication module uploads alarm information containing the traffic anomaly identification results to the cloud server. The 5G private network communication module accesses the 5G private network based on non-standalone networking mode through a contracted private network slice and a customized data network name, and establishes a secure connection with the internal network of the traffic management department.

7. A traffic anomaly identification device, characterized in that, include: The acquisition module is used to acquire real-time traffic video collected by the vehicle dashcam; The recognition module is used to input the real-time traffic video into a pre-trained lightweight deep neural network model to obtain the traffic anomaly recognition result output by the model. The lightweight deep neural network model includes a feature extraction network, a feature fusion and enhancement network, and a classification recognition head connected in sequence. The feature extraction network consists of convolutional layers, inverted residual block layers, and dilated convolutional-attention layers, used to extract multi-scale feature maps from the input image. The inverted residual block layer adopts a depthwise separable convolutional structure, including a first inverted residual block sub-layer with an embedded attention mechanism module and a second inverted residual block sub-layer without an embedded attention mechanism module. The feature fusion and enhancement network receives the multi-scale feature maps output by the feature extraction network, performs top-down feature fusion through a feature pyramid network, and fuses multi-scale contextual information through dilated spatial pyramid pooling to output an enhanced multi-scale fused feature map. The classification recognition head outputs the recognition result of the traffic anomaly based on the enhanced multi-scale fused feature map.

8. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store computer-executable instructions that, when executed by a processor, implement the steps of the method described in any one of claims 1 to 6.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the method described in any one of claims 1 to 6.

Images