Vehicle red light running monitoring method and system based on deep learning

Abstract

The present invention discloses a method and system for monitoring vehicles running red lights based on deep learning, the method comprising the following steps: collecting real-time monitoring videos of traffic intersections through high-definition cameras; dividing the real-time monitoring videos into detection areas; detecting the divided detection areas through deep neural networks; and judging whether the vehicle has run a red light based on the detection results of the divided areas. The system comprises: a vehicle real-time video acquisition module, a target intersection area division module, a vehicle detection module based on deep neural networks, a key frame acquisition module during red light display, a red light running behavior judgment module, an alarm module, an information storage module and an electronic equipment deployment device. The present invention can identify vehicles running red lights, and automatically extract license plate information based on license plate photos to complete data storage and alarm.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a vehicle red light running monitoring method and system based on deep learning. Background Art

[0002] With the acceleration of urbanization, the number of vehicles has increased dramatically, and traffic congestion and traffic violations have become one of the important issues of urban management. Red light running is a high-frequency and serious violation of the law. The high risk of collision, multi-vehicle chain reaction, high mortality rate and serious injury rate that it may bring pose a major threat to road safety and social public safety. Traditional methods of monitoring vehicles running red lights mainly rely on manual monitoring and electronic eye monitoring systems. Manual monitoring requires a lot of manpower and material resources, is costly and has low monitoring efficiency. It is difficult to achieve 24-hour uninterrupted monitoring due to the fatigue and concentration of monitoring personnel, resulting in misjudgment and missed detection. The electronic eye monitoring system relies on underground induction coils, which has problems such as complex construction and high maintenance costs.

[0003] At present, deep learning technology has been widely used in traffic management, and the most common one is the YOLO series of target detection algorithms. However, most of the existing target detection algorithms in red light running detection use their initial network structure and weights, which may not be suitable for specific traffic monitoring scenarios and have a high probability of false detection and missed detection. Summary of the invention

[0004] In order to overcome the shortcomings of the prior art, the present invention provides a vehicle red light running monitoring method and system based on deep learning, which uses an improved lightweight target detection algorithm to monitor vehicle red light running behavior, has high real-time and recognition accuracy, and is highly adaptable while ensuring computing efficiency and running speed, and can autonomously divide the area to be monitored according to different road conditions. When the red light is on, the stop line area detects the target vehicle to obtain a clear license plate photo. When the red light running condition setting is met, the present invention can automatically extract the license plate information based on the license plate photo to complete data storage and alarm. The specific technical solution is:

[0005] The vehicle red light running monitoring method based on deep learning specifically includes the following steps:

[0006] Collect real-time monitoring videos of traffic intersections through high-definition cameras;

[0007] Divide the real-time surveillance video into detection areas;

[0008] The divided detection area is detected through the improved Yolov5 model;

[0009] Judging whether the vehicle has run a red light based on the detection results of the divided areas;

[0010] The improved Yolov5 model specifically includes: ICA module and RepSA module; the ICA module is located between the Neck and Head parts of YOLOv5 to perform feature screening on multi-scale features; the RepSA module is located in the Neck part to fuse the multi-scale high-dimensional features extracted by Backbone;

[0011] The ICA module specifically includes:

[0012] X max Pool layer, used for maximum pooling in the X direction, i.e., width direction; Y max Pool layer, used for maximum pooling in the Y direction, i.e., height direction; W×1 Conv layer, 1×H Conv layer, and 1×1 Conv layer are used for convolutions with width W and height 1, width 1 and height H, and width and height 1 respectively; Batch Nom layer, used for batch normalization; permute function, used to rearrange the dimensions of the array; h-swish and Sigmoid are used as activation functions;

[0013] The ICA module operation process includes: the input of the ICA module is respectively input into the X max Pool layer and the W×1Conv layer in the X direction and arrayed through the permute function, and the input into the Y max Pool layer and the 1×H Conv layer in the Y direction, the maximum pooling and convolution results in the X and Y directions are spliced, and the results are passed through the 1×1 Conv layer, the Batch Nom layer and the h-swish activation function, and the results are respectively downsampled in the X and Y directions, and passed through the 1×1 Conv layer and the Sigmoid activation function to obtain the attention vectors in the X and Y directions; the attention vectors in the X and Y directions are multiplied with the input features of the ICA module to obtain the final feature map with weights, and the output of the ICA module is obtained;

[0014] The RepSA module specifically includes: Conv1 convolution kernel, Conv3 convolution kernel and Conv5 convolution kernel, representing 1×1 convolution kernel, 3×3 convolution kernel and 5×5 convolution kernel respectively; SelfAttention, representing the self-attention mechanism;

[0015] The specific operation process of the RepSA module includes: the input of the RepSA module adjusts the number of channels through the Conv1 convolution kernel, and the Conv1 convolution results are respectively passed through 3×3 convolution and 5×5 convolution to obtain the model's perception of targets of different sizes. The input features after feature reduction by 1×1 convolution are added to the feature maps after 3×3 convolution and 5×5 convolution through the residual branch, and finally the output of the RepSA module is obtained through the self-attention mechanism.

[0016] Furthermore, the detection area includes: a first area: a stop line range area of ​​the vehicle to be monitored; a second area: a middle range area of ​​the intersection; and a third area: an opposite intersection area where the target vehicle is about to enter.

[0017] Furthermore, a judgment is made as to whether the vehicle has run a red light. The judgment is based on the following criteria: if the first area, the second area, and the third area all recognize the same target vehicle during the red light display period, then the vehicle has run a red light; if the first area, the second area, or only the first area recognizes the same target vehicle, then it is not determined as a complete red light running behavior.

[0018] A vehicle red light running monitoring system based on deep learning, used to implement any one of the vehicle red light running monitoring methods based on deep learning, comprising:

[0019] Vehicle real-time video acquisition module, used to collect real-time monitoring videos of traffic intersections through high-definition cameras;

[0020] The target intersection area division module is used to divide the detection area according to the intersections with different road conditions;

[0021] A vehicle detection module based on a deep neural network is used to detect the divided detection areas in real time;

[0022] A key frame acquisition module during red light display, which is used to acquire the key frame of the target vehicle in the detection area through a deep learning neural network when the red light is displayed;

[0023] A red light running behavior judgment module is used to judge whether the target vehicle has run a red light behavior according to the key frame image results;

[0024] An alarm module is used to automatically alarm when a vehicle runs a red light;

[0025] An information storage module is used to record key frame image data and clear the images obtained without red light running within a certain period of time;

[0026] An electronic device deployment device is used to deploy electronic devices of the algorithm provided by the present invention.

[0027] Furthermore, the key frame acquisition module during the red light display period specifically includes: when the red light is displayed, if the vehicle is identified in the first area, then the motion recognition is performed to determine the static or moving state, if it is in the moving state, the key frame is initially extracted according to the lighting conditions and image quality, and the initially extracted key frame is filtered and optimized; after the vehicle is identified as being in the moving state in the first area, the key frame acquisition operation is also performed for the second area and the third area;

[0028] The electronic device deployment module specifically includes: an electronic device deployment device, a sensor, and an electronic device module, wherein the electronic device module includes: an interface, a processor, a memory, and a communication module.

[0029] According to one aspect of the present invention, a storage medium is provided, wherein instructions are stored in the storage medium. When a computer reads the instructions, the computer executes any one of the above-mentioned deep learning-based vehicle red light running monitoring methods.

[0030] According to another aspect of the present invention, there is provided an electronic device, comprising a processor and the above-mentioned storage medium, wherein the processor executes instructions in the storage medium.

[0031] Compared with the prior art, the present invention has the following beneficial effects:

[0032] 1. The improved target detection algorithm used in the present invention can effectively identify obscured vehicles and small target vehicles. Different from the generalized model used in the prior art, the algorithm of the present invention is more lightweight, improves operating efficiency and can ensure higher accuracy.

[0033] 2. The present invention provides an electronic device deployment device, and the algorithm can be deployed to different devices and is suitable for a variety of application scenarios.

[0034] 3. The RepSA module proposed in the present invention uses a multi-step feature processing method that combines the advantages of multi-scale convolution, residual connection, and self-attention mechanism. It not only effectively improves the computational efficiency and speed of the model, but also enhances the model's ability to detect small targets and vehicles running red lights in complex scenarios, ensuring the comprehensiveness and accuracy of feature extraction.

[0035] 4. The ICA module proposed in the present invention has improved the coordinated attention mechanism and used the maximum pooling operation instead of the average pooling operation to prevent the feature loss problem in high-complexity scenes and enhance the sensitivity to the texture features in the feature map. It can more accurately detect and identify vehicles running red lights in complex traffic environments, while effectively improving the computational efficiency and detection speed of the model, ensuring the comprehensiveness and accuracy of feature extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0037] Figure 1 A schematic diagram of a flow chart of an embodiment of the present invention;

[0038] Figure 2 A schematic diagram of the system structure of an embodiment of the present invention;

[0039] Figure 3 It is a structural diagram of the RepSA module of an embodiment of the present invention;

[0040] Figure 4 This is a structural diagram of an ICA module according to an embodiment of the present invention;

[0041] Figure 5 An improved Yolov5 model diagram of an embodiment of the present invention;

[0042] Figure 6 It is a schematic diagram of an electronic device deployment module according to an embodiment of the present invention. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0044] The present invention provides a vehicle red light running monitoring method and system based on deep learning, which uses an improved innovative lightweight target detection algorithm to monitor vehicles running red lights, has high real-time performance and recognition accuracy, and has strong adaptability while ensuring computing efficiency and running speed, and can autonomously divide the area to be monitored according to different road conditions. When the red light is on, the stop line area detects the target vehicle to obtain a clear license plate photo. When the red light running condition setting is met, the system can automatically extract the license plate information based on the license plate photo to complete data storage and alarm.

[0045] like Figure 1 As shown, the vehicle red light running monitoring method based on deep learning specifically includes the following steps:

[0046] Collect real-time monitoring videos of traffic intersections through high-definition cameras;

[0047] Divide the real-time surveillance video into detection areas;

[0048] Detect the divided detection area through a deep neural network;

[0049] Based on the detection results of the divided areas, it is judged whether the vehicle has run a red light.

[0050] like Figure 2As shown, the vehicle red light running monitoring system 00 based on deep learning mainly includes: a vehicle real-time video acquisition module 01, which can obtain real-time monitoring videos of traffic intersections collected by high-definition cameras with high frame rate, low latency and multiple angles; a target intersection area division module 02, which is used to divide the three vehicle detection areas according to the intersections with different road conditions; a vehicle detection module 03 based on a deep neural network, which can detect the three vehicle detection areas divided in 02 in real time; a key frame acquisition module 04 during red light display, which is used to obtain the key frame of the target vehicle in the detection area through a deep learning neural network when the red light is displayed; a red light running behavior judgment module 05, which is used to judge whether the target vehicle has a red light running behavior according to the key frame image results; an alarm module 06, which is used to automatically alarm when the result in the above judgment module is yes. An information storage module 07, which is used to record key frame image data and clear the images obtained without red light running within a certain period of time; an electronic device deployment device 08, which is used to deploy the electronic device of the algorithm provided by the present invention.

[0051] Specifically, in the vehicle real-time video acquisition module, what is obtained is the real-time video of the monitored intersection from multiple angles and in all directions shot by the monitoring camera. The monitored vehicles can be motor vehicles and non-motor vehicles with license plates, etc.

[0052] Taking into account the differences in different video angles of different sections of the road, and in order to reduce the interference of a large amount of irrelevant background information in the monitoring video, a target area division module is set up, which can manually divide the monitoring area when the camera captures the video for the first time, so as to improve the operating efficiency of the system. According to the existing red light running judgment method, when the vehicle passes the stop line, the vehicle drives to the middle of the intersection, and the vehicle reaches the opposite intersection, the three photos taken during the red light period are considered as red light running. Based on this, the target area is divided into three parts. The target area mainly includes: the first area, the second area, and the third area. The first area corresponds to the stop line range of the vehicle to be monitored, the second area corresponds to the area in the middle of the intersection, and the third area represents the opposite intersection area where the target vehicle is about to enter. The three areas can be divided according to different video angles and different road conditions, and the shapes include but are not limited to quadrilaterals. This module can be regarded as an initialization process. After determining the intersection to be monitored, the full-view video captured by the camera is obtained for the first time, and the area can be manually divided according to the needs. If you want to re-divide the area, repeat this step to complete the initialization process. This module can improve the recognition accuracy and operating speed of red light running monitoring to a certain extent by autonomously selecting monitoring areas and reducing the amount of calculation required by the system to process irrelevant data.

[0053] Specifically, the vehicle detection module based on the deep neural network is used to detect vehicles in the divided target area. The deep neural network model is an innovative lightweight target detection algorithm based on the Yolov5 algorithm. It accelerates reasoning and reduces delays by optimizing the network structure, etc., and can ensure that the detection accuracy is not greatly affected while improving the detection speed. In addition, the lightweight model described in the present invention also has the characteristics of low power consumption, flexibility and scalability, and can be easily deployed and applied in the electronic equipment deployment module.

[0054] Specifically, the two module improvements provided by the present invention include an ICA (Improved Attention) module and a RepSA (RepLight Spatial Attention) module.

[0055] like Figure 3As shown in the figure, the RepSA module can use fewer parameters and calculate more feature maps. In the figure, Input and Output represent the input feature map and the output feature map, Conv1, Conv3, and Conv5 represent 1×1 convolution kernels, 3×3 convolution kernels, and 5×5 convolution kernels, respectively, and the SelfAttention mechanism. The range of the input feature is RH×W×C, where H, W, and C represent the width, height, and number of channels of the feature map, respectively. First, the number of channels is adjusted by 1×1 convolution to reduce the number of model feature maps. This operation can not only effectively avoid feature redundancy and simplify the model structure, but also significantly improve the operation speed of the model because it reduces the number of parameters and calculations involved in the calculation process. Secondly, different sizes of convolution kernels (3×3 convolution and 5×5 convolution) are used to obtain the model's perception of targets of different sizes. This multi-scale feature extraction method greatly improves the model's sensitivity to small targets (such as distant or partially obscured vehicles) because convolution kernels of different sizes can capture information of different scales, making the model more accurate and comprehensive when processing targets of various sizes. Next, the input features after 1×1 convolution feature reduction are added to the feature maps processed with different receptive fields through the residual branch. This operation can effectively integrate multi-scale information, making the target features (such as the outline and motion trajectory of the vehicle) more prominent in the feature map, enhancing the richness and accuracy of the feature representation. At the same time, the residual connection helps to alleviate the gradient vanishing problem in the deep network, ensuring that the model training process is more stable and efficient. Finally, the self-attention mechanism is used to give the target (such as a vehicle running a red light) a greater weight and context information. The self-attention mechanism can dynamically adjust the importance of each feature, enhance the focus on key features, and combine the global context information to make the model's recognition of the target more accurate and robust. The introduction of the self-attention mechanism not only improves the accuracy of feature expression, but also enhances the model's performance in complex scenarios. This multi-step feature processing method combines the advantages of multi-scale convolution, residual connection and self-attention mechanism, which not only effectively improves the computational efficiency and speed of the model, but also enhances the model's ability to detect small targets and vehicles running red lights in complex scenarios, ensuring the comprehensiveness and accuracy of feature extraction.

[0056] like Figure 4As shown in the figure, C×H×W in the ICA module represents the number of channels, height and width of the feature image, and X max Pool and Y max Pool represent the maximum value pooling along the X and Y directions, that is, the width and height directions, respectively. W×1 Conv, 1×H Conv, and 1×1 Conv represent convolution kernels with a width of W and a height of 1, a width of 1 and a height of H, and a width and a height of 1, respectively. Batch Nom represents batch normalization, permute represents the rearrangement of the array dimensions, and h-swish and Sigmoid are both activation functions. The r in the figure represents the multiple of downsampling, and © and U represent data concatenation and multiplication operations.

[0057] like Figure 5 As shown in the figure, the ICA module is located between the Neck and Head parts of YOLOv5 to perform feature screening on multi-scale features; RepSA is located in the Neck part to fuse the multi-scale high-dimensional features extracted by Backbone to improve the accuracy of the model.

[0058] Among them, Conv represents the convolution layer, k represents the kernel size, s represents the stride, and p represents the edge pixel expansion (padding); in the C3 module, n represents the number of module stacking times, True and False represent whether the C3 module always has a shortcut; SPFF represents the efficient spatial pyramid pooling module (Spatial Pyramid Pooling-Fast), k represents the kernel size; RepSA represents the RepSA module, and ICA represents the ICA module.

[0059] For the target detection network used in some prior arts, simply splicing the shallow features extracted by the encoder with the deep features restored by the decoder will make it difficult to distinguish highly similar features and it is difficult to perfectly integrate the features of the two stages. The ICA module proposed in the present invention, i.e., the improved coordinated attention mechanism, is an attention mechanism that considers both spatial and channel dimensions and considers position-related information in the spatial direction. The mechanism assigns different weights to different channels in the channel dimension to enhance the recognition between the monitored vehicles and non-vehicle targets, which has a significant contribution to the accurate detection of vehicles running red lights. From the spatial dimension, the existing coordinated attention mechanism uses average pooling to encode input features along the height and width directions. However, due to the averaging effect of average pooling, this weakens the feature information of target vehicles that are partially obscured or far away in complex traffic scenes. In addition, due to the high similarity between vehicles of some colors and their backgrounds (such as zebra crossings and road signs), average pooling may cause the combination of feature information to make it difficult to distinguish highly similar categories. Therefore, the present invention improves the coordinated attention mechanism and uses the maximum pooling operation instead of the average pooling operation to prevent the problem of feature loss in highly complex scenes and enhance the sensitivity to texture features in feature maps. At the same time, W×1 Conv and 1×H Conv convolution branches are added to learn the texture features of the feature map to capture long-range spatial interactions with precise location information and learn texture features for each target vehicle area. Then, the attention vectors in the horizontal and vertical directions are obtained through a specific mutation method. Finally, these attention vectors are multiplied with the input features to obtain the final feature map with weights.

[0060] The ICA module provided by the present invention can more accurately detect and identify vehicles running red lights in complex traffic environments, while effectively improving the model's computational efficiency and detection speed, ensuring the comprehensiveness and accuracy of feature extraction. Finally, the vehicle detection model based on a deep neural network combined with the above two modules performed well in various traffic monitoring applications used in the test, with higher flexibility and adaptability.

[0061] Specifically, the key frame acquisition module during the red light display is used to obtain the key frames of the target vehicle in the detection area through the deep learning neural network when the red light is displayed. When the red light is displayed, if the vehicle is identified in the first area, the position and bounding box of the vehicle in the image are confirmed, and action recognition is performed to pay attention to its static or moving state. If it is in a moving state, the key frames are initially extracted according to the lighting conditions and image quality. The preliminary extracted key frames are filtered and optimized to ensure that its license plate and other information can be recognized, including but not limited to removing duplicate frames, smoothing animations, or reordering frames, etc., and different operations are performed according to different scenarios. After the vehicle is identified as being in motion in the first area, similar key frame acquisition operations are performed on the second and third areas. In addition to being saved in the information storage module, the acquired key frames are also input into the deep learning neural network for further feature extraction and recognition of its license plate information.

[0062] Furthermore, the red light running judgment module is used to judge whether there is a red light running behavior for the key frame after obtaining the key frame. The basis for judgment is that during the red light display period, the first area, the second area, and the third area all recognize the same target vehicle, then the vehicle has a red light running behavior; if the first area, 202 or only the first area recognizes it, it is not considered a complete red light running behavior, but there may still be uncivilized vehicle driving.

[0063] When it is determined that a red light run has occurred, the alarm module will automatically sound an alarm. The alarm module will send the license plate number information of the identified red light running vehicle to the police station closest to the monitored intersection so that the next step can be taken.

[0064] Specifically, the information storage module will store the extracted key frame data with license plate information, and will also record the video stream data of the divided target area corresponding to the surveillance video. For key frame data, the key frame data of vehicles that did not run red lights will be automatically cleared after a few working days; the key frame data of vehicles identified as running red lights and vehicles with uncivilized driving behavior will be retained until the event is processed and then manually cleared.

[0065] In addition, the present invention also provides an electronic equipment deployment module, the schematic diagram of which is shown in FIG. Figure 6As shown. It mainly includes an electronic device deployment device 500, a sensor 501, an electronic device module 502, and the electronic device includes an interface, a processor, a memory and a communication module. The sensor must include a high-definition camera, but is not limited to adding a laser radar, a camera sensor array and an inertial measurement unit, etc. The camera captures images and video data of the vehicle and its surroundings for target vehicle detection and tracking; the laser radar can use a laser beam to measure the distance and shape of the vehicle and its surroundings for high-precision distance measurement and map construction; the camera sensor array contains multiple camera sensors, which can provide a wider field of view and more data. The inertial measurement unit can be used to monitor the acceleration and angular velocity of the vehicle, and provide information about the motion state and direction of the device.

[0066] The electronic device module includes an interface, a processor, a memory, and a communication module. The interface part is used to communicate and exchange data with the sensor. The interface type can be an analog interface, a digital interface, a general purpose input / output (GPIO), a USB interface, an Ethernet interface, a wireless interface, a CAN bus interface, etc. There is no requirement for a specific type. It is selected according to the needs of the specific application and a suitable physical or logical connection is established with the sensor. In practical applications, it is necessary to consider factors such as the sensor's data transmission rate, transmission distance, power consumption, and cost to select a suitable interface type.

[0067] The processor part is used to execute the algorithm and other related software tasks proposed in the present invention, and can be a general-purpose processor (CPU), an embedded processor, a graphics processor (GPU), a dedicated accelerator, and an FPGA (field programmable gate array). A general-purpose processor (CPU) is usually used to execute various types of software tasks; an embedded processor is a processor specially designed for embedded systems, usually with low power consumption, small size and high performance; a graphics processor (GPU) is a processor specially designed for graphics processing and parallel computing, usually with large-scale parallel computing capabilities and high performance. In the vehicle detection algorithm hardware deployment module, a graphics processor can be used to accelerate the execution of the vehicle detection algorithm and improve the algorithm's computing performance and efficiency; a dedicated accelerator is a processor specially designed to accelerate specific types of computing tasks, such as a deep learning accelerator, a neural network processor, etc., which can improve the algorithm's computing performance and efficiency; an FPGA is a flexible programmable logic device that can be reconfigured as needed to implement different types of computing tasks.

[0068] The memory part is used to store information such as program code, data, and temporary variables. Common ones may be random access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD), and some external storage devices (hard drive, SD card, etc.). These memories should be selected according to the needs and performance requirements of the specific application, and used in conjunction with the processor and other hardware to implement the hardware deployment of the vehicle detection algorithm. The communication module is responsible for communicating with other devices or central servers, transmitting monitoring results, receiving instructions, etc. It can be Wi-Fi, Bluetooth, LoRa, Zigbee, CAN bus, Ethernet, mobile network, etc. In practical applications, the appropriate communication module is selected according to the specific needs and system design, and it can also be implemented in combination with multiple communication technologies.

[0069] The examples described in the present invention are merely descriptions of the preferred implementation modes of the present invention, and are not intended to limit the concept and scope of the present invention. Without departing from the design concept of the present invention, various modifications and improvements made to the technical solutions of the present invention by engineers and technicians in this field should all fall within the protection scope of the present invention.

Claims

1. A vehicle red light running monitoring method based on deep learning, characterized in that: The specific steps include: Collect real-time monitoring videos of traffic intersections through high-definition cameras; Divide the real-time surveillance video into detection areas; The divided detection area is detected through the improved Yolov5 model; Judging whether the vehicle has run a red light based on the detection results of the divided areas; The improved Yolov5 model specifically includes: ICA module and RepSA module; the ICA module is located between the Neck and Head parts of YOLOv5 to perform feature screening on multi-scale features; the RepSA module is located in the Neck part to fuse the multi-scale high-dimensional features extracted by Backbone; The ICA module specifically includes: X max Pool layer, used for maximum pooling in the X direction, i.e., width direction; Y max Pool layer, used for maximum pooling in the Y direction, i.e., height direction; W×1Conv layer, 1×H Conv layer, and 1×1Conv layer are used for convolutions with width W and height 1, width 1 and height H, and width and height 1 respectively; Batch Nom layer, used for batch normalization; permute function, used to rearrange the dimensions of the array; h-swish and Sigmoid are used as activation functions; The ICA module operation process includes: the input of the ICA module is respectively input into the X max Pool layer and the W×1Conv layer in the X direction and arrayed through the permute function, and the input into the Y max Pool layer and the 1×H Conv layer in the Y direction, the maximum pooling and convolution results in the X and Y directions are spliced, and the results are passed through the 1×1Conv layer, the Batch Nom layer and the h-swish activation function, and the results are respectively downsampled in the X and Y directions, and passed through the 1×1Conv layer and the Sigmoid activation function to obtain the attention vectors in the X and Y directions; the attention vectors in the X and Y directions are multiplied with the input features of the ICA module to obtain the final feature map with weights, and the output of the ICA module is obtained; The RepSA module specifically includes: Conv1 convolution kernel, Conv3 convolution kernel and Conv5 convolution kernel, representing 1×1 convolution kernel, 3×3 convolution kernel and 5×5 convolution kernel respectively; SelfAttention, representing the self-attention mechanism; The specific operation process of the RepSA module includes: the input of the RepSA module adjusts the number of channels through the Conv1 convolution kernel, and the Conv1 convolution results are respectively passed through 3×3 convolution and 5×5 convolution to obtain the model's perception of targets of different sizes. The input features after feature reduction by 1×1 convolution are added to the feature maps after 3×3 convolution and 5×5 convolution through the residual branch, and finally the output of the RepSA module is obtained through the self-attention mechanism.

2. The method according to claim 1, characterized in that The detection area includes: the first area: the stop line range area of ​​the vehicle to be monitored; the second area: the middle range area of ​​the intersection; the third area: the opposite intersection area where the target vehicle is about to enter.

3. The method according to claim 1, characterized in that Determine whether the vehicle has run a red light. The judgment basis is: if the first area, the second area, and the third area all identify the same target vehicle during the red light display period, then the vehicle has run a red light. If the first area, the second area, or only the first area recognizes the same target vehicle, it is not determined as a complete red light running behavior.

4. A vehicle red light running monitoring system based on deep learning, used to implement the vehicle red light running monitoring method based on deep learning according to any one of claims 1 to 3, characterized in that: include: Vehicle real-time video acquisition module, used to collect real-time monitoring videos of traffic intersections through high-definition cameras; The target intersection area division module is used to divide the detection area according to the intersections with different road conditions; A vehicle detection module based on a deep neural network is used to detect the divided detection areas in real time; A key frame acquisition module during red light display, which is used to acquire the key frame of the target vehicle in the detection area through a deep learning neural network when the red light is displayed; A red light running behavior judgment module is used to judge whether the target vehicle has run a red light behavior according to the key frame image results; An alarm module is used to automatically alarm when a vehicle runs a red light; An information storage module is used to record key frame image data and clear the images obtained without red light running within a certain period of time; An electronic device deployment module is used to deploy electronic devices of the algorithm.

5. The system according to claim 4, characterized in that The key frame acquisition module during the red light display period specifically includes: when the red light is displayed, if the vehicle is identified in the first area, the motion recognition is performed to determine the static or moving state. If it is in the moving state, the key frames are initially extracted according to the lighting conditions and image quality, and the initially extracted key frames are filtered and optimized; after the vehicle is identified as being in the moving state in the first area, the key frame acquisition operation is also performed for the second area and the third area; The electronic device deployment module specifically includes: an electronic device deployment device, a sensor, and an electronic device module, wherein the electronic device module includes: an interface, a processor, a memory, and a communication module.

6. A storage medium, characterized in that: The storage medium stores instructions, and when a computer reads the instructions, the computer executes the vehicle red light running monitoring method based on deep learning as described in any one of claims 1 to 3.

7. An electronic device, characterized in that: The invention comprises a processor and the storage medium as claimed in claim 6, wherein the processor executes instructions in the storage medium.

Images