Traffic Incident Detection Method and Device Based on Infrared and Visible Light Images

Abstract

This invention provides a traffic incident detection method and apparatus based on infrared and visible light images. The method involves acquiring infrared and visible light images of the road scene to be detected; obtaining pre-processed infrared and visible light images; when the brightness of the pre-processed visible light image is below a set threshold, fusing the pre-processed infrared and visible light images using a pre-trained deep learning image fusion model to obtain a fused image as the image to be detected; when the brightness of the pre-processed visible light image is not below the set threshold, using the pre-processed visible light image as the image to be detected; and inputting the image to be detected into a traffic incident detection model to obtain the detection result. This invention ensures high-precision detection under sufficient lighting conditions while effectively improving the accuracy and robustness of traffic incident detection under low-light conditions such as nighttime.

Description

Technical Field

[0001] This invention relates to a method and apparatus for detecting traffic incidents based on infrared and visible light images, belonging to the field of traffic incident detection technology. Background Technology

[0002] In recent years, the number of motor vehicles in Chinese cities has been increasing, placing a heavy burden on urban traffic. Traditionally, traffic incidents relied entirely on manual intervention and handling by traffic police, consuming a significant amount of manpower. However, with technological advancements, deep learning-based traffic incident detection methods have emerged. This system can detect traffic incidents in real time and push the detected incidents to the processing module, greatly saving manpower costs, improving the efficiency of traffic management departments, and enhancing the level of urban traffic governance.

[0003] However, most existing traffic incident detection methods are based on the analysis of visible light images. Visible light images contain a lot of noise, and traditional image denoising methods cannot effectively denoise the target images. Therefore, the accuracy of traffic incident detection using these images will be greatly reduced.

[0004] For example, a traffic incident detection method and device published in Chinese patent application publication number CN 114781479 A also suffers from the problem of low accuracy in traffic incident detection under low light conditions such as at night.

[0005] The above-mentioned issues are problems that should be considered and resolved in the process of traffic incident detection based on infrared and visible light images. Summary of the Invention

[0006] The purpose of this invention is to provide a traffic incident detection method and apparatus based on infrared and visible light images to solve the problem that the accuracy of traffic incident detection under low light conditions in the prior art needs to be improved.

[0007] The technical solution of this invention is:

[0008] A traffic incident detection method based on infrared and visible light images includes the following steps:

[0009] S1. Acquire infrared and visible light images of the road scene to be detected;

[0010] S2. Preprocess the acquired infrared and visible light images to obtain preprocessed infrared and visible light images;

[0011] S3. When the brightness of the preprocessed visible light image is lower than the set threshold, proceed to step S4; when the brightness of the preprocessed visible light image is not lower than the set threshold, use the preprocessed visible light image as the image to be detected, and proceed to step S5.

[0012] S4. Use a pre-trained deep learning image fusion model to fuse the pre-processed infrared image and the pre-processed visible light image to obtain the fused image as the image to be detected, and proceed to the next step S5.

[0013] S5. Input the image to be detected into the traffic incident detection model to obtain the detection results.

[0014] Further, in step S2, the acquired infrared and visible light images are preprocessed, specifically as follows:

[0015] S21. Perform noise reduction processing on the acquired infrared and visible light images;

[0016] S22. Adjust the contrast and brightness of the infrared and visible light images;

[0017] S23. Based on the infrared characteristics of the target object obtained from the infrared image, remove interference objects in the scene that do not have infrared characteristics.

[0018] Further, in step S22, the contrast and brightness of the infrared image and the visible light image are adjusted, specifically as follows:

[0019] S221. Use a convolutional neural network (CNN) model to extract features from infrared and visible light images respectively, and obtain the features of infrared and visible light images respectively.

[0020] S222, Standardize the brightness and contrast of the features of infrared and visible light images;

[0021] S223. Compare the brightness of the features of the visible light image with a preset threshold. When the brightness of the features of the visible light image is higher than the preset threshold, adjust the brightness and contrast of the features of the infrared image to be consistent with the visible light image. When the brightness of the features of the visible light image is lower than the preset threshold, adjust the brightness and contrast of the features of the visible light image to be consistent with the infrared image.

[0022] Further, in step S222, the brightness and contrast features of the infrared and visible light images are standardized, specifically as follows:

[0023] S2221. Perform brightness standardization on infrared and visible light images respectively: calculate the mean and standard deviation of the features, then subtract the mean from each value of the feature and divide by the standard deviation to ensure that the mean of the feature is zero and the standard deviation is one.

[0024] S2221. Perform contrast normalization on infrared and visible light images respectively: scale each value of the feature to a set range to enhance contrast.

[0025] Furthermore, in step S4, the pre-trained deep learning image fusion model adopts any one of the following: the general image fusion framework model IFCNN based on convolutional neural networks, the unified unsupervised image fusion network model U2Fusion, the semantically aware real-time infrared and visible light image fusion network model SeAFusion, or the fusion model of infrared and visible images, namely the DenseFuse model.

[0026] Furthermore, in step S5, the traffic incident detection model includes a configuration unit, a deep learning-based target detection model, a deep learning-based target tracking model, and a parameter matching unit.

[0027] Further, in step S5, the image to be detected is input into the traffic incident detection model to obtain the detection result, specifically,

[0028] S51. Use a deep learning-based target detection model to detect the image to be detected and output the detected target information, including vehicle target information, non-motorized vehicle target information and pedestrian target information.

[0029] S52. Use a deep learning-based target tracking model to track the target information detected in step S51 and output the target trajectory data.

[0030] S53. Based on the target information and target trajectory data output in steps S51 and S52, and combined with the traffic event detection parameters pre-configured by the configuration unit, the parameter matching unit makes a judgment. When the pre-configured traffic event detection parameters are met, the corresponding traffic event is determined to have occurred, and the detected target information and traffic event are taken as the detection result.

[0031] Furthermore, deep learning-based object detection models employ YOLO, SSD (Single-Step Multi-Box Detection), or faster region convolutional neural network models, such as Faster R-CNN.

[0032] Furthermore, the deep learning-based target tracking model employs the efficient convolutional neural network model ECO for target tracking, the interference-considering twin region proposal network DaSiamRPN, or the multi-domain target tracking and detection model, namely the MDNet model.

[0033] An apparatus for implementing the traffic incident detection method based on infrared and visible light images as described in any of the above claims includes an image acquisition module, an image preprocessing module, a detection image generation module, and a traffic incident detection module.

[0034] Image acquisition module: Acquires infrared and visible light images of the road scene to be detected;

[0035] Image preprocessing module: preprocesses the acquired infrared and visible light images to obtain preprocessed infrared and visible light images;

[0036] The detection image generation module: When the brightness of the preprocessed visible light image is lower than a set threshold, the pre-trained deep learning image fusion model is used to fuse the preprocessed infrared image and the preprocessed visible light image to obtain the fused image as the image to be detected; when the brightness of the preprocessed visible light image is not lower than the set threshold, the preprocessed visible light image is used as the image to be detected.

[0037] Traffic incident detection module: Input the image to be detected into the traffic incident detection model to obtain the detection results.

[0038] The beneficial effects of this invention are as follows: This traffic incident detection method and apparatus based on infrared and visible light images, by using visible light images for detection under sufficient lighting conditions and using fused infrared and visible light images for traffic incident detection under low lighting conditions, can ensure high-precision detection under sufficient lighting conditions while effectively improving the accuracy and robustness of traffic incident detection under low-light conditions such as nighttime. This invention can utilize existing monitoring equipment on the road, reducing hardware costs. Attached Figure Description

[0039] Figure 1 This is a schematic flowchart of a traffic incident detection method based on infrared and visible light images according to an embodiment of the present invention;

[0040] Figure 2 This is an illustrative diagram illustrating the traffic incident detection model in the embodiment;

[0041] Figure 3 This is an illustrative diagram illustrating a traffic incident detection device based on infrared and visible light images, as described in an embodiment. Detailed Implementation

[0042] The preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0043] Example

[0044] A traffic incident detection method based on infrared and visible light images, such as... Figure 1 This includes the following steps:

[0045] S1. Acquire infrared and visible light images of the road scene to be detected;

[0046] S2. Preprocess the acquired infrared and visible light images to obtain preprocessed infrared and visible light images;

[0047] In step S2, the acquired infrared and visible light images are preprocessed, specifically as follows:

[0048] S21. Perform noise reduction processing on the acquired infrared and visible light images to eliminate or reduce image noise;

[0049] S22. Adjust the contrast and brightness of the infrared and visible light images;

[0050] S221. Use a convolutional neural network (CNN) model to extract features from infrared and visible light images respectively, and obtain the features of infrared and visible light images respectively.

[0051] S222, Standardize the brightness and contrast of the features of infrared and visible light images;

[0052] In step S222, the brightness and contrast features of the infrared and visible light images are standardized, specifically as follows:

[0053] S2221. Perform brightness standardization on infrared and visible light images respectively: calculate the mean and standard deviation of the features, then subtract the mean from each value of the feature and divide by the standard deviation to ensure that the mean of the feature is zero and the standard deviation is one.

[0054] S2221. Perform contrast normalization on infrared and visible light images respectively: scale each value of the feature to a set range to enhance contrast.

[0055] S223. Compare the brightness of the features of the visible light image with a preset threshold. When the brightness of the features of the visible light image is higher than the preset threshold, adjust the brightness and contrast of the features of the infrared image to be consistent with the visible light image. When the brightness of the features of the visible light image is lower than the preset threshold, adjust the brightness and contrast of the features of the visible light image to be consistent with the infrared image.

[0056] S23. Based on the infrared characteristics of the target object obtained from the infrared image, remove interference objects in the scene that do not have infrared characteristics.

[0057] In step S23, based on the infrared features of the target object obtained from the infrared image, interference objects in the scene that do not have infrared features, such as trees, road signs, and streetlights, are removed, so that the image fusion model can focus more on feature extraction and fusion of the target to be detected.

[0058] S3. When the brightness of the preprocessed visible light image is lower than the set threshold, proceed to step S4; when the brightness of the preprocessed visible light image is not lower than the set threshold, use the preprocessed visible light image as the image to be detected, and proceed to step S5.

[0059] S4. Use a pre-trained deep learning image fusion model to fuse the pre-processed infrared image and the pre-processed visible light image to obtain the fused image as the image to be detected, and proceed to the next step S5.

[0060] In step S4, the pre-trained deep learning image fusion model adopts any one of the following: the general image fusion framework model IFCNN based on convolutional neural networks, the unified unsupervised image fusion network model U2Fusion, the semantically aware real-time infrared and visible light image fusion network model SeAFusion, or the fusion model of infrared and visible images, namely the DenseFuse model.

[0061] S5. Input the image to be detected into the traffic incident detection model to obtain the detection results.

[0062] In step S5, such as Figure 2 The traffic incident detection model includes a configuration unit, a deep learning-based object detection model, a deep learning-based object tracking model, and a parameter matching unit. The deep learning-based object detection model employs YOLO, SSD (Single-Step Multi-Box Detection), or a faster region convolutional neural network model, such as Faster R-CNN. The deep learning-based object tracking model employs ECO (Efficient Convolutional Neural Network for Object Tracking), DaSiamRPN (DaSiam Region Proposal Network Considering Interference), or MDNet (Multi-Domain Object Tracking and Detection Model).

[0063] In step S5, the image to be detected is input into the traffic incident detection model to obtain the detection result, specifically:

[0064] S51. Use a deep learning-based target detection model to detect the image to be detected and output the detected target information, including vehicle target information, non-motorized vehicle target information and pedestrian target information.

[0065] In step S51, the vehicle target information includes snapshot image, vehicle type, model, license plate, body color, etc.; the non-motorized vehicle target information includes snapshot image, target color, target type, etc.; and the pedestrian target information includes snapshot image.

[0066] S52. Use a deep learning-based target tracking model to track the target information detected in step S51 and output the target trajectory data.

[0067] S53. Based on the target information and target trajectory data output in steps S51 and S52, and combined with the traffic event detection parameters pre-configured by the configuration unit, the parameter matching unit makes a judgment. When the pre-configured traffic event detection parameters are met, the corresponding traffic event is determined to have occurred, and the detected target information and traffic event are taken as the detection result.

[0068] In step S53, the pre-configured traffic incident detection parameters include the area to be detected, the lane to be detected, and the detection line.

[0069] In step S5, both the target detection model and the target tracking model need to be trained a set number of times on the constructed dataset. The training dataset is constructed using preprocessed visible light images and fused images, including visible light image datasets and infrared and visible light fused image datasets.

[0070] This traffic incident detection method, based on infrared and visible light images, uses visible light images for detection under sufficient lighting conditions and a fused infrared and visible light image for detection under low lighting conditions. This ensures high-precision detection under sufficient lighting conditions while effectively improving the accuracy and robustness of traffic incident detection under low-light conditions such as nighttime. This invention can utilize existing road monitoring equipment, reducing hardware costs.

[0071] like Figure 3 The embodiment also provides an apparatus for implementing the traffic incident detection method based on infrared and visible light images as described in any of the above embodiments, including an image acquisition module, an image preprocessing module, a detection image generation module, and a traffic incident detection module.

[0072] Image acquisition module: Acquires infrared and visible light images of the road scene to be detected;

[0073] Image preprocessing module: preprocesses the acquired infrared and visible light images to obtain preprocessed infrared and visible light images;

[0074] The detection image generation module: When the brightness of the preprocessed visible light image is lower than a set threshold, the pre-trained deep learning image fusion model is used to fuse the preprocessed infrared image and the preprocessed visible light image to obtain the fused image as the image to be detected; when the brightness of the preprocessed visible light image is not lower than the set threshold, the preprocessed visible light image is used as the image to be detected.

[0075] Traffic incident detection module: Input the image to be detected into the traffic incident detection model to obtain the detection results.

[0076] In the traffic incident detection module, a deep learning-based traffic incident detection model is used to detect the image to be detected. When a traffic incident that is pre-configured according to the specific road scene is detected, the detected target information and the traffic incident are output as the detection result.

[0077] Pre-configured traffic events include one or more of the following: driving in the wrong direction, illegal lane changing, non-motorized vehicles not wearing helmets, pedestrians entering motorized vehicle lanes, non-motorized vehicles entering motorized vehicle lanes, trucks not keeping to the right, road congestion, traffic accidents, hazardous chemical vehicles, and construction obstruction. Specifically: For road congestion: lanes, lane speed limits, and trigger thresholds must be configured. Road congestion is defined when the lane speed falls below the set threshold and exceeds the set number of triggers. For traffic accidents: a detection zone must be configured. An accident is defined when one or more motorized or non-motorized vehicles are close to each other within the zone and remain there for more than 5 minutes, with pedestrians or other targets lingering nearby. For hazardous chemical vehicles: motorized vehicle lanes and travel time limits must be configured. A hazardous chemical vehicle is defined when it appears in the lane but is not within the designated travel time limit. For construction obstruction: a detection zone must be configured. Construction vehicles, construction workers, and traffic cones are defined when they appear within the zone and remain there for more than 5 minutes.

[0078] In this traffic incident detection method based on infrared and visible light images, existing surveillance cameras on the road can be used as visible light cameras to obtain visible light images; surveillance cameras with infrared capabilities can be used as infrared cameras to obtain infrared images, reducing the procurement cost of new equipment. The installation positions of the infrared and visible light cameras need to be adjusted to ensure consistency in imaging position, imaging angle, and imaging time.

[0079] This traffic incident detection method and apparatus based on infrared and visible light images acquires images of the target monitoring point using both visible light and infrared cameras. The acquired infrared and visible light images are then fused using an image fusion method. The fused image retains the color, texture, and other details of the visible light image while also possessing the infrared features of the target scene under low-light conditions such as nighttime, provided by the infrared image. Further detection using a deep learning-based traffic incident detection method on the fused image can effectively improve the detection accuracy of traffic incidents under low-light conditions such as nighttime.

[0080] This traffic incident detection method and apparatus based on infrared and visible light images can utilize existing monitoring facilities to acquire infrared and visible light images, thus saving equipment costs. Using fused infrared and visible light images for traffic incident detection under low-light conditions can effectively improve the accuracy and robustness of traffic incident detection at night and other low-light conditions. The computer program used in this method can be stored on a computer storage medium, which can be deployed in a computer room or miniaturized and deployed near the target road to be detected, thereby reducing deployment costs.

[0081] The above description is only a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. Any equivalent modifications or changes made by those skilled in the art based on the content disclosed in the present invention should be included within the scope of protection set forth in the claims.

Claims

1. A traffic incident detection method based on infrared and visible light images, characterized by: The method comprises the following steps, S1, acquiring an infrared image and a visible light image of a road scene to be detected; S2, preprocessing the acquired infrared image and visible light image to obtain a preprocessed infrared image and a preprocessed visible light image; In step S2, the acquired infrared image and visible light image are preprocessed, specifically, S21, performing noise reduction processing on the acquired infrared image and visible light image; S22, adjusting the contrast and brightness of the infrared image and visible light image; S221, using a convolutional neural network model CNN to perform feature extraction on the infrared image and visible light image respectively to obtain the features of the infrared image and visible light image respectively; S222, performing brightness and contrast normalization on the features of the infrared image and visible light image; S2221, performing brightness normalization on the infrared image and visible light image respectively: calculating the mean and standard deviation of the features, then subtracting the mean and dividing by the standard deviation of each value of the features to ensure that the mean of the features is zero and the standard deviation is one; S2221, performing contrast normalization on the infrared image and visible light image respectively: scaling each value of the features to a set range to enhance the contrast; S223, comparing the brightness of the features of the visible light image with a preset threshold, when the brightness of the features of the visible light image is higher than the preset threshold, adjusting the brightness and contrast of the features of the infrared image to be consistent with the visible light image; when the brightness of the features of the visible light image is lower than the preset threshold, adjusting the brightness and contrast of the features of the visible light image to be consistent with the infrared image; S23, according to the infrared features of the target object acquired by the infrared image, eliminating interference objects in the scene without infrared features; S3, when the brightness of the preprocessed visible light image is lower than a set threshold, entering step S4; when the brightness of the preprocessed visible light image is not lower than the set threshold, taking the preprocessed visible light image as a to-be-detected image and entering step S5; S4, using a pre-trained deep learning image fusion model to fuse the preprocessed infrared image and the preprocessed visible light image to obtain a fusion image as a to-be-detected image, and entering the next step S5; S5, inputting the to-be-detected image into a traffic event detection model to obtain a detection result; the traffic event detection model comprises a configuration unit, a target detection model based on deep learning, a target tracking model based on deep learning, and a parameter matching unit; S51, using the target detection model based on deep learning to detect the to-be-detected image, outputting detected target information including vehicle target information, non-motor vehicle target information, and pedestrian target information; S52, using the target tracking model based on deep learning to track the target information detected in step S51, outputting target trajectory data; S53, according to the target information and target trajectory data output by steps S51 and S52, combining the traffic event detection parameters preconfigured by the configuration unit, and determining by the parameter matching unit, when the preconfigured traffic event detection parameters are met, determining that a corresponding traffic event occurs, and taking the detected target information and traffic event as a detection result. 2.The traffic incident detection method based on infrared and visible light images according to claim 1, wherein: In step S4, the pre-trained deep learning image fusion model uses any one of a general image fusion framework model IFCNN based on a convolutional neural network, a unified unsupervised image fusion network model U2Fusion, a semantic perception real-time infrared and visible light image fusion network model SeAFusion, or an infrared image and visible image fusion model DenseFuse. 3.The traffic incident detection method based on infrared and visible light images of claim 1, wherein: The deep learning-based target detection model uses a target detection model YOLO, a single-step multi-frame target detection model SSD, or a Faster Region Convolutional Neural Network model FasterRCNN. 4.The traffic incident detection method based on infrared and visible light images according to claim 1, wherein: The deep learning-based target tracking model uses an efficient convolutional neural network model ECO for target tracking, a twin region proposal network DaSiamRPN considering interference, or a multi-domain target tracking detection model MDNet.

5. An apparatus for implementing the traffic incident detection method based on infrared and visible light images according to any one of claims 1 to 4, characterized in that: The system comprises an image acquisition module, an image preprocessing module, a detection image generation module, and a traffic event detection module, The image acquisition module acquires infrared images and visible light images of a road scene to be detected. The image preprocessing module preprocesses the acquired infrared images and visible light images to obtain preprocessed infrared images and preprocessed visible light images. The detection image generation module uses a pre-trained deep learning image fusion model to fuse the preprocessed infrared images and the preprocessed visible light images when the brightness of the preprocessed visible light images is lower than a set threshold, and obtains a fusion image as a detection image; or the detection image generation module uses the preprocessed visible light images as a detection image when the brightness of the preprocessed visible light images is not lower than the set threshold. The traffic event detection module inputs the detection image into a traffic event detection model to obtain a detection result.

Images