Traffic detection method and system based on visible light and infrared image fusion

Abstract

The invention provides a traffic detection method and system based on visible light and infrared image fusion, and belongs to the field of image processing. Comprising the following steps: acquiring an original visible light image and an infrared image in the same scene, and preprocessing; based on adaptive filtering, Laplacian pyramids are constructed for the preprocessed visible light image and infrared image, and multi-scale fine registration is carried out on the two Laplacian pyramids; performing gradient weighted fusion on the high-level images of the two Laplacian pyramids to obtain a high-level fusion image, and performing double-domain average fusion on the low-level images of the two Laplacian pyramids to obtain a low-level fusion image; performing layer-by-layer reconstruction based on the high-layer fusion image and the low-layer fusion image to obtain a reconstructed fusion image; and carrying out traffic detection based on the reconstructed fusion image. According to the method, edge details are reserved through adaptive filtering, the advantages of visible light and infrared images are integrated through different weighted fusion, and the target recognition accuracy and robustness in traffic detection are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a traffic detection method and system based on the fusion of visible light and infrared images. Background Art

[0002] With the acceleration of urbanization, traffic accidents occur frequently, especially at night and in bad weather conditions. The fusion of visible light images and infrared dual-band images makes up for the shortcomings of a single sensor. The two combine the rich details of visible light and the stability of infrared images to achieve information complementarity and generate stable and information-rich fusion images, thereby enhancing the system's perception of scene information and target recognition capabilities in complex environments such as night, bad weather, and poor lighting conditions.

[0003] However, the existing technology pays little attention to the fusion effect of images of different modalities. For example, the Chinese invention application with application number 202311218824.1, entitled Traffic incident detection method and device based on infrared image and visible light image, only directly uses the existing common model to fuse images without making adaptive adjustments, so that the fused image cannot accurately reflect the key information in the original image, thereby causing a decrease in target detection performance. At the same time, the existing fusion technology relies on traditional feature matching algorithms and rarely considers the combination of multi-scale transformation and registration. These algorithms often cannot achieve high-precision registration when dealing with illumination changes, perspective differences and motion blur, especially when dealing with image features of different resolutions and scales. Features of different scales often have significant differences in importance and performance. Failure to fully utilize this information will further affect the fusion quality. In addition, current technology usually uses Gaussian filtering for image smoothing. This method may cause the loss of image details while eliminating noise. In the case of more noise or poor image quality, the use of Gaussian filtering will make the final fused image appear blurred in detail presentation, thereby reducing the accuracy of subsequent processing (such as target detection). Summary of the invention

[0004] In order to solve the above problems, the present invention proposes a traffic detection method and system based on the fusion of visible light and infrared images. By using adaptive filtering, the smoothing intensity can be dynamically adjusted according to the local characteristics of the image, the edge details of the image can be effectively retained while removing noise, and a high-quality foundation is provided for subsequent image fusion; and different weighted fusion strategies are adopted, that is, gradient weighted fusion of high-level images is performed to enhance detail information, and average fusion of low-level images is performed to maintain overall brightness and smoothness. This processing method can combine the advantages of visible light images and infrared images, improve the clarity and contrast of the reconstructed fused image, thereby significantly improving the accuracy and reliability of traffic detection; at the same time, processing and alignment are performed at multi-scale levels to further improve the registration accuracy, optimize the fusion effect, enhance the applicability of the system in complex scenes, and provide a higher quality foundation for subsequent target detection.

[0005] In order to achieve the above object, the present invention adopts the following technical solution: In a first aspect, the present invention provides a traffic detection method based on visible light and infrared image fusion, comprising: Obtain the original visible light image and infrared image of the same scene and perform preprocessing; Based on adaptive filtering, Laplacian pyramids are constructed for the preprocessed visible light image and infrared image respectively, and multi-scale fine registration is performed on the visible light Laplacian pyramid and the infrared Laplacian pyramid. A high-level image of the visible light Laplace pyramid is fused with a high-level image in the infrared Laplace pyramid by using gradient weighted fusion to obtain a high-level fused image, and a low-level image of the visible light Laplace pyramid is fused with a low-level image in the infrared Laplace pyramid by using dual-domain average fusion to obtain a low-level fused image; based on the high-level fused image and the low-level fused image, layer-by-layer reconstruction is performed to obtain a reconstructed fused image; Traffic detection based on reconstructed fused images.

[0006] Preferably, the pretreatment comprises: After the infrared image is preprocessed by two-point correction, blind pixel compensation and median filtering, the contrast is enhanced by platform histogram equalization, the pixel is quantized by maximum and minimum linear mapping, and the image size is unified by bilinear interpolation algorithm. The visible light image is scaled to ensure that the sizes of the visible light image and the infrared image match.

[0007] Preferably, the adaptive filtering is based on respectively constructing a Laplacian pyramid for the preprocessed visible light image and infrared image, and the specific steps include: Adaptive mean filtering method is used to smooth the preprocessed visible light image and infrared image; The filtered image is repeatedly downsampled to construct a visible light Gaussian pyramid and an infrared Gaussian pyramid respectively; The two Gaussian pyramids are respectively differentiated layer by layer to obtain a visible light Laplace pyramid and an infrared Laplace pyramid.

[0008] Preferably, the adaptive mean filtering method specifically includes: dynamically adjusting the smoothing strength according to the local characteristics of the image, performing strong smoothing in the flat area of ​​the image, and weakening the smoothing strength in the edge area.

[0009] Preferably, the performing multi-scale fine registration on the visible light Laplace pyramid and the infrared Laplace pyramid specifically includes: Obtaining the images in the two Laplacian pyramids, and using the Sobel operator to respectively calculate the gradients of the images in the two Laplacian pyramids; Divide the image into multiple small units and calculate the gradient direction histogram in each unit; Normalize the gradient direction histogram in each small unit, reconstruct the normalized histogram into visible light reconstructed image and infrared reconstructed image respectively, and calculate the HOG features of the visible light reconstructed image and infrared reconstructed image respectively; The distance between the feature points of the visible light reconstructed image and the infrared reconstructed image is calculated, and the feature point with the smallest distance is selected as the matching pair for feature matching. The infrared image is transformed according to the estimated transformation matrix to align it with the visible light image.

[0010] Preferably, the low-level image of the visible light Laplacian pyramid and the low-level image of the infrared Laplacian pyramid are fused by dual-domain average to obtain a low-level fused image, wherein the dual-domain average fusion includes global average fusion and local average fusion, specifically including: The lower-level image of the visible light Laplacian pyramid and the lower-level image of the infrared Laplacian pyramid are fused by global average to obtain a preliminary lower-level fused image; Divide the local areas of the two low-level images based on the preset window, traverse the local areas to perform local average fusion, and obtain a local low-level fusion image; The preliminary low-level fusion image and the local low-level fusion image are weightedly summed to obtain a final low-level fusion image.

[0011] Preferably, the method further comprises performing pseudo color processing on the reconstructed fused image by using a color transformation method.

[0012] In a second aspect, the present invention provides a traffic detection system based on visible light and infrared image fusion, comprising: Image acquisition module, used to obtain the original visible light image and infrared image in the same scene and perform preprocessing; A registration module is used to construct Laplacian pyramids for the preprocessed visible light image and infrared image based on adaptive filtering, and to perform multi-scale fine registration on the visible light Laplacian pyramid and the infrared Laplacian pyramid; A fusion and reconstruction module is used to fuse the high-level image of the visible light Laplace pyramid with the high-level image in the infrared Laplace pyramid by using gradient weighting to obtain a high-level fused image, and fuse the low-level image of the visible light Laplace pyramid with the low-level image in the infrared Laplace pyramid by using dual-domain average to obtain a low-level fused image; reconstruct the high-level fused image and the low-level fused image layer by layer to obtain a reconstructed fused image; The detection module is used for traffic detection based on the reconstructed fused image.

[0013] In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of a traffic detection method based on visible light and infrared image fusion as described in the first aspect.

[0014] In a fourth aspect, the present invention provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of a traffic detection method based on the fusion of visible light and infrared images described in the first aspect are implemented.

[0015] Compared with the prior art, the present invention has the following beneficial effects: 1. The present invention combines visible light with infrared images. Infrared technology can identify objects by sensing their temperature under low light conditions, thus eliminating the dependence of visible light sensors on light sources. Visible light technology can capture clearer and higher-resolution images. The fusion of the two can provide a clearer and more accurate target recognition technology.

[0016] 2. The present invention introduces an adaptive filter to replace the traditional Gaussian filter for optimization in the Laplacian pyramid multi-layer decomposition process of the image, and dynamically adjusts the smoothing strength according to the local characteristics of the image. It can perform strong smoothing in the flat area of ​​the image and retain more details in the edge area, thereby better capturing the edge and texture information in the image and enhancing the detail retention of the structural information in the fused image.

[0017] 3. The present invention uses a combination rule of gradient weighted fusion and average fusion in the fusion process of the Laplacian pyramid, which optimizes the image fusion effect while taking into account both image details and global information.

[0018] 4. The present invention combines multi-scale transformation and multi-scale registration, which can process and align image features at different pyramid levels, improve the registration accuracy and adaptability to corresponding complex scenes.

[0019] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings constituting a part of the present invention are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their description are used to explain the present invention but do not constitute a limitation of the present invention.

[0021] Figure 1 A flow chart of a traffic detection method based on visible light and infrared image fusion provided by an embodiment of the present invention; Figure 2 A main flow chart of a traffic detection method based on visible light and infrared image fusion provided by an embodiment of the present invention; Figure 3 A flowchart of image fusion based on multi-scale transformation provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0022] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0023] Embodiment 1 like Figure 1 As shown, this embodiment discloses a traffic detection method based on visible light and infrared image fusion, comprising the following steps: S1: Obtain the original visible light image and infrared image of the same scene and perform preprocessing; S2: Based on adaptive filtering, Laplacian pyramids are constructed for the preprocessed visible light image and infrared image respectively, and multi-scale fine registration is performed on the visible light Laplacian pyramid and the infrared Laplacian pyramid; S3: A high-level image of the visible light Laplace pyramid is fused with a high-level image in the infrared Laplace pyramid by using gradient weighted fusion to obtain a high-level fused image, and a low-level image of the visible light Laplace pyramid is fused with a low-level image in the infrared Laplace pyramid by using dual-domain average fusion to obtain a low-level fused image; based on the high-level fused image and the low-level fused image, layer-by-layer reconstruction is performed to obtain a reconstructed fused image; S4: Traffic detection based on reconstructed fused images.

[0024] Next, combine Figure 2 , a traffic detection method based on the fusion of visible light and infrared images disclosed in this embodiment is described in detail.

[0025] In S1, visible light CCD sensors and infrared focal plane array detectors are configured in the traffic monitoring area. For the same target scene, the original visible light image and original infrared image of the traffic scene are collected in real time, and the ROI area is extracted from the visible light image and the infrared image. Specifically, it includes: Select the appropriate visible light CCD sensor and infrared focal plane array sensor and install them on the same platform, use the same optical elements (such as lenses) for imaging, and ensure that the imaging fields of the two overlap to the greatest extent. Next, select the appropriate lens and adjust the optical system to ensure that the optical paths of visible light and infrared light overlap as much as possible. During installation, make on-site adjustments by fine-tuning the bracket or adjusting the screws to ensure that the sensor position is accurately aligned. After the system is set up, perform imaging tests to confirm the alignment of the two images, and make fine adjustments to the position if necessary to verify the coarse alignment effect.

[0026] Use visible light sensor and infrared sensor to collect images of the same scene and target, capture the original 8-bit visible light image and the original 14-bit infrared image. According to the selected ROI coordinates, cut out the ROI area from the original image to form a new sub-image.

[0027] The infrared image and visible light image acquired under the same target scene are preprocessed separately.

[0028] (1) Preprocess the infrared image. Perform three steps of preprocessing on the infrared image: correction, compensation and denoising. Use the two-point correction method to improve the overall quality and contrast of the image. Use blind pixel compensation to fill in the missing pixel values ​​in the image to improve the integrity of the image. Finally, use median filtering to remove noise in the image. Specifically: S101: Perform three-step preprocessing of the infrared image: correction, compensation and denoising.

[0029] First, use the two-point correction method to convert the original signal output by the sensor into a more accurate grayscale value. Use a standard grayscale card to select the known grayscale values ​​in the two images and record the corresponding sensor output values ​​as and , set the target grayscale value to and , the parameters are obtained by linear regression calculation and , establish a linear equation based on the known input and output relationship:

[0030] Use the linear equation to transform the original image Each pixel value in is corrected to obtain the corrected image :

[0031] Check each pixel in the image, identify pixels with abnormal values ​​(completely 0 or minimum value, etc.) and mark them as blind pixels, and use neighborhood information to compensate for each blind pixel. The blind pixel at , , , The mean value of the pixels at is assigned to the blind pixel.

[0032] Then, median filtering is performed on the corrected and compensated image to denoise it and the filter window size is determined ( , etc.), traverse each pixel in the image, Indicates that the image is The pixel value at the position, extracting its surrounding window The pixel grayscale values ​​in the window are sorted and the pixel values ​​in the window are set , replace the pixel value at the center of the current window with the median value obtained after sorting. Repeat this process for each pixel in the image until the entire image is processed, thus achieving the denoising effect. The pixels are shown in formula (3):

[0033] S102: using a platform histogram equalization method to enhance the contrast of the infrared image, so that detailed information in the image is clearer.

[0034] Computing images Grayscale histogram of , when the number of pixels at a certain gray level exceeds the platform threshold To avoid over-enhancement, the image is clipped when the image is clipped, as shown in formula (4):

[0035] In the formula, Crop the histogram for the threshold.

[0036] For each gray level to be clipped, the number of gray value pixels to be removed Evenly distribute to other gray levels to ensure that the total number of pixels remains unchanged. Then calculate the cumulative distribution function of the clipped histogram:

[0037] Using the cumulative distribution function The gray value of each pixel in the preprocessed image Mapped to the equalized gray value, thus achieving image enhancement. The formula is as follows:

[0038] in, is the minimum value of the cumulative distribution function, The maximum grayscale value of the image (usually 256).

[0039] S103: Using the maximum and minimum linear mapping method, the infrared image pixel value is converted from 14 bits data to 8 bits data, which is consistent with the visible light image pixel value.

[0040] Traverse all pixels of the infrared image and find their minimum grayscale value and the maximum gray value , the pixel values ​​of the original 14-bit image are changed from The range is linearly mapped to the 8-bit grayscale range , the formula is as follows:

[0041] S104: unify the image size by using a bilinear interpolation algorithm.

[0042] Due to the difference in size between visible light images and infrared images, a bilinear interpolation algorithm is used to set their sizes to be consistent. In the actual processing process, the processing of visible light images is only performed on the Y component (i.e., the brightness component), and the chrominance information (U and V components) is not processed to avoid affecting the visual quality of the image due to chrominance changes.

[0043] The preprocessed infrared image is used as the target image, and the size of the visible light image is adjusted using the bilinear interpolation algorithm. Since the visible light image is processed only on the Y component, it needs to be converted from the RGB color space to YUV first:

[0044] Extract the Y component from the converted YUV image to form a separate brightness image, and calculate the width and height scaling factors based on the size of the target infrared image and the original Y component , . Create a new Y component blank matrix of the target image size, looping through each new pixel position :

[0045] Select four neighboring pixels at the corresponding position as interpolation points, which is , , , , perform bilinear interpolation calculation on each pixel position obtained, and assign the result to the corresponding position of the new Y component :

[0046] in, is the pixel value of the target position after interpolation, , , , is the pixel value of the four adjacent integer pixels.

[0047] Combine the processed Y component with the original U and V components and convert back to RGB space representation:

[0048] Compared with visible light images, infrared images often require more processing to improve image quality and usability due to their imaging characteristics and environmental influences. This embodiment uses the above method to preprocess infrared images, which can significantly improve the overall quality and integrity of the image, reduce image defects through correction, compensation and denoising, and then use platform histogram equalization to enhance contrast and make image details clearer. At the same time, the maximum and minimum linear mapping method is used to optimize pixel values ​​to reduce data complexity, and the image size is unified through a bilinear interpolation algorithm, providing more accurate, efficient and convenient image data for subsequent image analysis, processing and application.

[0049] (2) Scale the visible light image to ensure that the sizes of the visible light image and the infrared image match.

[0050] The process of S2-S3 is as follows Figure 3 In S2, based on adaptive filtering, Laplacian pyramids are constructed for the preprocessed visible light image and infrared image respectively.

[0051] Prior to this, first, ensure that the two images are completely aligned in space so that the imaging fields of the visible light CCD sensor and the infrared focal plane array sensor are nearly consistent to achieve coarse registration.

[0052] S201: constructing a Gaussian pyramid, using an adaptive filter to repeatedly perform adaptive smoothing filtering on the image, and downsampling each smoothed image until an image pyramid with multiple resolution levels is generated.

[0053] First, construct a Gaussian pyramid and input the preprocessed visible light image and infrared images , the initial layer of the image is smoothed using an adaptive mean filter, and the pixel value after the adaptive mean filter is obtained by dynamically adjusting the filter strength to adapt to different feature areas of the image:

[0054] in, For the original image at position The pixel value of So A local neighborhood centered on or window), Z represents the normalization factor, is a local weight function used to dynamically adjust the filter strength.

[0055] For flat areas, i.e. areas with low contrast, set a higher smoothing strength:

[0056] For edge areas, i.e. areas with higher contrast, set a lower smoothing strength:

[0057] in, The threshold for controlling edge sensitivity. Represents the local contrast, as shown in the formula:

[0058] in, is the number of pixels in the neighborhood, is the average value in the neighborhood.

[0059] Then, after filtering, 2:1 downsampling is used to generate the next layer of image, and this step is repeated until the Gaussian pyramid is completed.

[0060] This embodiment uses an adaptive mean filter method to replace the traditional Gaussian filter, which can dynamically adjust the smoothing strength according to the local characteristics of the image. In the flat area of ​​the image, adaptive filtering can perform strong smoothing and effectively remove noise; while in the edge area of ​​the image, adaptive filtering will weaken the smoothing strength, thereby retaining the image details. By performing adaptive filtering on visible light images and infrared images, the quality of the image can be significantly improved, making it more suitable for subsequent image fusion and traffic detection tasks. Adaptive filtering helps to enhance the contrast of the image, making the target objects in the image clearer and easier to detect.

[0061] S202: constructing a Laplacian pyramid by performing layer-by-layer differentiation on the Gaussian pyramid.

[0062] Upsample each layer of the image in the Gaussian pyramid and calculate the difference between the current layer and the upsampled image to obtain each layer of the Laplacian pyramid. The formula is as follows:

[0063] In the formula, Represents the visible light Laplace pyramid Layer images, Represents the visible light Gaussian pyramid Layer images, It is for The visible light image obtained by upsampling the layer image; Infrared light Laplace pyramid Layer images, Indicates the infrared light Gaussian pyramid Layer images, It is for The infrared image is obtained by upsampling the layer image.

[0064] Perform multi-scale fine registration of visible light Laplacian pyramid and infrared Laplacian pyramid, including: S211: aligning the visible light Laplace pyramid and the infrared Laplace pyramid to obtain a fused Laplace pyramid; specifically including: Use the Sobel operator to calculate the gradient to obtain the gradient magnitude and direction of visible and infrared images:

[0065] Calculate the gradient magnitude and direction :

[0066] The image is divided into small units of 8x8 pixels, and the gradient direction histogram is calculated in each unit. The gradient direction in each unit is divided into several bins (for example, 9 directions), and the gradient magnitude of each pixel is accumulated in the corresponding bin:

[0067] in, Pixel The gradient amplitude, is the indicator function, is the corresponding bin.

[0068] The histogram in each small unit is normalized to improve the robustness to illumination changes. The normalized histograms are combined and reconstructed into visible light reconstructed images and infrared reconstructed images respectively, and the final HOG features of the visible light reconstructed images and infrared reconstructed images are calculated respectively. The normalization formula is as follows:

[0069] Where H represents the unnormalized HOG feature vector, is a small constant to prevent division by zero.

[0070] The distance between the feature points of the visible light reconstructed image and the infrared reconstructed image is calculated, and the feature point with the smallest distance is selected as the matching pair for feature matching. The infrared image is transformed according to the estimated transformation matrix to align it with the visible light image. The transformation is gradually applied at each layer of the Laplacian pyramid, and the Laplacian pyramids of the visible light and infrared images are fused to generate the fused Laplacian pyramid for fine registration.

[0071] In S3, the registered images are fused using a multi-scale decomposition and fusion method based on Laplace pyramid decomposition. The image fusion process based on multi-scale transformation includes the following steps: For the layers with higher resolution, the gradient weighted fusion method is used to retain the details of the visible light image and the heat source information of the infrared image; while for the layers with lower resolution, the average fusion method is used to smooth the global features of the fused image and ensure the consistency of the image brightness and contour.

[0072] For the image layer with higher resolution at the bottom of the pyramid, the gradient weighted fusion method is used. The fusion weight is calculated for each image according to the gradient amplitude. The area with larger gradient has higher weight, and the area with smaller gradient has lower weight. The weight calculation formula is:

[0073] in, is the visible light image weight, is the infrared image weight; represents the gradient of the visible light image, Represents the gradient of the infrared image.

[0074] The high-resolution layers of visible light and infrared images are weightedly fused using the gradient weighted fusion rule. The formula is:

[0075] in, represents a high-level fused image, is a high-level image of the visible light Laplacian pyramid, It is a high-level image in the infrared Laplacian pyramid.

[0076] For the low-resolution layer of the Laplacian pyramid, dual-domain average fusion is used to obtain the final low-level fused image, where dual-domain average fusion includes global average fusion and local average fusion.

[0077] First, perform global average fusion of visible light and infrared images to obtain preliminary fusion results:

[0078] in, represents the initial low-level fusion image, is the low-level image of the visible light Laplacian pyramid, It is the lower layer image in the infrared Laplacian pyramid. Based on the original global average fusion, the average fusion technology in the local area is added to retain more detailed image features.

[0079] Based on the global fusion results, local areas of visible and infrared images are selected for further processing based on specific image features (such as edges or textures). The local area is defined by a window of different sizes, and the average calculation is performed again in each local area to retain more detailed image features:

[0080] in, represents the local low-level fusion image, are the pixel coordinates in the local area, is the local area in the lower layer image of the visible light Laplacian pyramid, It is a local area in the low-level image in the infrared Laplacian pyramid.

[0081] Combine the global average result with the local average result to get the final low-resolution layer fusion image:

[0082] in, To adjust the parameters, control the ratio of global and local fusion.

[0083] In this embodiment, for high-level images, the gradient weighted fusion method can make full use of the edge and texture information in the high-level image, and obtain a clearer and more accurate high-level fused image through weighted fusion; for low-level images, the dual-domain average fusion method can smooth the noise and detail fluctuations in the image, thereby obtaining a more stable low-level fused image. By combining the gradient weighted fusion and average fusion methods, the complementary information in the visible light image and the infrared image can be fully utilized to improve the quality and accuracy of the fused image. This helps to make subsequent traffic detection tasks more accurate and reliable.

[0084] Among them, the dual-domain average fusion method first performs a preliminary fusion of visible light and infrared images through global average fusion to obtain a relatively balanced preliminary fusion result, which helps to integrate the overall information of the image. Secondly, based on global fusion, local average fusion processes the local area of ​​the infrared image according to specific image features and uses a window of a specific size for average calculation, which can retain more detailed image features such as edges and textures. Finally, by adjusting the parameters to merge the global and local fusion results, flexible control of the fusion ratio is achieved, so that the fused image achieves better results in both overall and detailed aspects.

[0085] The fused high-level fusion image and the low-level fusion image are reconstructed layer by layer to obtain a fusion image containing features of two modalities.

[0086] After all layers are fused, the Laplacian pyramid is reconstructed layer by layer. Upsampling is performed layer by layer starting from the bottom layer of the pyramid, and the obtained image is added to the low-resolution Laplacian image of that layer to restore the reconstructed image of that layer. This process is repeated, layer by layer, until the size of the original image is restored. The formula is as follows:

[0087] in, It is The reconstructed fused image of the layer, It is The fused Laplacian image of the layer.

[0088] After obtaining the reconstructed fused image, pseudo color processing is performed. Pseudo color processing uses the grayscale-color transformation method. By establishing a mapping relationship between the grayscale level of the grayscale image and various colors in the color space, the grayscale image is converted into a color image, further enhancing its visual effect.

[0089] The grayscale-color transformation method is used to perform pseudo-color processing on the image after the visible light image and the infrared image are fused. First, the fused image is normalized and each pixel value is Standardized to The formula is as follows:

[0090] in, and are the minimum and maximum pixel values ​​in the image, respectively.

[0091] Use a color mapping table to map each pixel value of the grayscale image to a color value in the RGB space, convert it into a pseudo-color image, and save it. The color of each pixel can be expressed as:

[0092] in, is a function that maps grayscale values ​​to RGB colors, Represents the grayscale value of the pixel at position (x,y).

[0093] In S4, the fusion of visible light and infrared images is applied to traffic detection in various complex scenarios. It can perform real-time identification and abnormal behavior analysis of pedestrians and vehicles, and assist in the detection and early warning of traffic accidents, further improving traffic safety.

[0094] As an optional implementation, a target detection model based on deep learning is used to detect the image to be detected, and the detected target information is output, including vehicle target information, non-motor vehicle target information and pedestrian target information; Track the detected target information using a deep learning-based target tracking model and output target trajectory data; Based on the output target information and target trajectory data, combined with the traffic event detection parameters pre-configured by the configuration unit, the parameter matching unit makes a judgment. When the pre-configured traffic event detection parameters are met, it is determined that a corresponding traffic event has occurred, and the detected target information and traffic event are used as the detection result.

[0095] Furthermore, the target detection model based on deep learning adopts the target detection model YOLO, the single-step multi-frame target detection model SSD or the faster regional convolutional neural network model, namely the FasterRCNN model.

[0096] Furthermore, the target tracking model based on deep learning adopts the efficient convolutional neural network model ECO for target tracking, the interference-considering twin region proposal network DaSiamRPN, or the multi-domain target tracking detection model, namely the MDNet model.

[0097] The visible light and infrared fusion images that have been fused can be applied to various types of traffic detection, including vehicle speed and position changes, pedestrian violations, etc., and are especially suitable for applications in low-light complex environments such as night scenes and bad weather, which can effectively reduce the interference caused by light changes. By using a deep learning model to detect targets and conduct data training on the fused image, it is possible to accurately identify and detect traffic participants such as pedestrians and vehicles in real time. On this basis, it is also possible to analyze the target's motion trajectory, monitor speed, direction changes and relative position, and identify whether there are potential abnormal behaviors, such as sudden braking, lane deviation, pedestrians running into red lights, etc. At the same time, through real-time and continuous traffic detection and setting judgment thresholds for abnormal behaviors, possible accidents can be analyzed and predicted in a timely manner. Once an abnormal event is identified, the system will automatically issue an alarm and notify the relevant traffic management department in a timely manner to reduce the possibility of accidents and improve the accuracy and efficiency of traffic accident detection.

[0098] Embodiment 2 This embodiment provides a traffic detection system based on visible light and infrared image fusion, including: Image acquisition module, used to obtain the original visible light image and infrared image in the same scene and perform preprocessing; A registration module is used to construct Laplacian pyramids for the preprocessed visible light image and infrared image based on adaptive filtering, and to perform multi-scale fine registration on the visible light Laplacian pyramid and the infrared Laplacian pyramid; A fusion and reconstruction module is used to fuse the high-level image of the visible light Laplace pyramid with the high-level image in the infrared Laplace pyramid by using gradient weighting to obtain a high-level fused image, and fuse the low-level image of the visible light Laplace pyramid with the low-level image in the infrared Laplace pyramid by using dual-domain average to obtain a low-level fused image; reconstruct the high-level fused image and the low-level fused image layer by layer to obtain a reconstructed fused image; The detection module is used for traffic detection based on the reconstructed fused image.

[0099] Embodiment 3 This embodiment provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps in the traffic detection method based on visible light and infrared image fusion as described in the first embodiment above are implemented.

[0100] Embodiment 4 This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps of a traffic detection method based on visible light and infrared image fusion as described in the first embodiment are implemented.

[0101] The steps or modules involved in the above embodiments 2 to 4 correspond to those in embodiment 1. For the specific implementation, please refer to the relevant description of embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood to include any medium that can store, encode or carry an instruction set for execution by a processor and enable the processor to execute any method in the present invention.

[0102] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A traffic detection method based on visible light and infrared image fusion, characterized in that: include: Obtain the original visible light image and infrared image of the same scene and perform preprocessing; Based on adaptive filtering, Laplacian pyramids are constructed for the preprocessed visible light image and infrared image respectively, and multi-scale fine registration is performed on the visible light Laplacian pyramid and the infrared Laplacian pyramid. A high-level image of the visible light Laplace pyramid is fused with a high-level image in the infrared Laplace pyramid by using gradient weighted fusion to obtain a high-level fused image, and a low-level image of the visible light Laplace pyramid is fused with a low-level image in the infrared Laplace pyramid by using dual-domain average fusion to obtain a low-level fused image; Reconstruct layer by layer based on the high-level fusion image and the low-level fusion image to obtain a reconstructed fusion image; Traffic detection based on reconstructed fused images.

2. A traffic detection method based on visible light and infrared image fusion as claimed in claim 1, characterized in that: The pre-processing comprises: After the infrared image is preprocessed by two-point correction, blind pixel compensation and median filtering, the contrast is enhanced by platform histogram equalization, the pixel is quantized by maximum and minimum linear mapping, and the image size is unified by bilinear interpolation algorithm. The visible light image is scaled to ensure that the sizes of the visible light image and the infrared image match.

3. The traffic detection method based on visible light and infrared image fusion as claimed in claim 1, characterized in that: The method of constructing a Laplacian pyramid for the preprocessed visible light image and infrared image based on adaptive filtering comprises the following specific steps: Adaptive mean filtering method is used to smooth the preprocessed visible light image and infrared image; The filtered image is repeatedly downsampled to construct a visible light Gaussian pyramid and an infrared Gaussian pyramid respectively; The two Gaussian pyramids are respectively differentiated layer by layer to obtain a visible light Laplace pyramid and an infrared Laplace pyramid.

4. A traffic detection method based on visible light and infrared image fusion as claimed in claim 3, characterized in that: The adaptive mean filtering method specifically includes: dynamically adjusting the smoothing strength according to the local characteristics of the image, performing strong smoothing in the flat area of ​​the image, and weakening the smoothing strength in the edge area.

5. The traffic detection method based on visible light and infrared image fusion as claimed in claim 1, characterized in that: The multi-scale fine registration of the visible light Laplacian pyramid and the infrared Laplacian pyramid specifically includes: Obtaining the images in the two Laplacian pyramids, and using the Sobel operator to respectively calculate the gradients of the images in the two Laplacian pyramids; Divide the image into multiple small units and calculate the gradient direction histogram in each unit; Normalize the gradient direction histogram in each small unit, reconstruct the normalized histogram into visible light reconstructed image and infrared reconstructed image respectively, and calculate the HOG features of the visible light reconstructed image and infrared reconstructed image respectively; The distance between the feature points of the visible light reconstructed image and the infrared reconstructed image is calculated, and the feature point with the smallest distance is selected as the matching pair for feature matching. The infrared image is transformed according to the estimated transformation matrix to align it with the visible light image.

6. The traffic detection method based on visible light and infrared image fusion as claimed in claim 1, characterized in that: The low-level image of the visible light Laplace pyramid and the low-level image of the infrared Laplace pyramid are fused by dual-domain average fusion to obtain a low-level fused image, wherein the dual-domain average fusion includes global average fusion and local average fusion, specifically including: The lower-level image of the visible light Laplacian pyramid and the lower-level image of the infrared Laplacian pyramid are fused by global average to obtain a preliminary lower-level fused image; Divide the local areas of the two low-level images based on the preset window, traverse the local areas to perform local average fusion, and obtain a local low-level fusion image; The preliminary low-level fusion image and the local low-level fusion image are weightedly summed to obtain a final low-level fusion image.

7. The traffic detection method based on visible light and infrared image fusion as claimed in claim 1, characterized in that: It also includes using color transformation method to perform pseudo color processing on the reconstructed fused image.

8. A traffic detection system based on visible light and infrared image fusion, characterized in that: include: Image acquisition module, used to obtain the original visible light image and infrared image in the same scene and perform preprocessing; A registration module is used to construct Laplacian pyramids for the preprocessed visible light image and infrared image based on adaptive filtering, and to perform multi-scale fine registration on the visible light Laplacian pyramid and the infrared Laplacian pyramid; A fusion reconstruction module is used to fuse the high-level image of the visible light Laplace pyramid with the high-level image in the infrared Laplace pyramid by using gradient weighting to obtain a high-level fused image, and to fuse the low-level image of the visible light Laplace pyramid with the low-level image in the infrared Laplace pyramid by using dual-domain averaging to obtain a low-level fused image; Reconstruct layer by layer based on the high-level fusion image and the low-level fusion image to obtain a reconstructed fusion image; The detection module is used for traffic detection based on the reconstructed fused image.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of a traffic detection method based on visible light and infrared image fusion as described in any one of claims 1 to 7 are implemented.

10. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps in the traffic detection method based on visible light and infrared image fusion as described in any one of claims 1-7 are implemented.

Images