Multi-light measurement environment disturbance adaptive compensation method, equipment and medium

Abstract

The invention provides a multi-light measurement environment disturbance self-adaptive compensation method and device and a medium, and relates to the technical field of multi-light imaging images, and the method comprises the steps: obtaining to-be-processed images, and carrying out the grouping of the to-be-processed images, and obtaining an ultraviolet image, a visible light image, and an infrared image; processing the ultraviolet image by adopting a Bayesian adaptive filtering algorithm and histogram gray level transformation to obtain a first ultraviolet image; processing the visible light image through a defogging model and Gamma correction to obtain a first visible light image; processing the infrared image through a temperature compensation model and a histogram enhancement algorithm to obtain a first infrared image; overlapping and fusing the first ultraviolet image and the first visible light image to obtain a clear visible light image; and fusing the first visible light image and the first infrared light image to obtain a high-resolution infrared image. The method achieves the targeted compensation enhancement of an image in a large wavelength range in a complex environment, and can greatly reduce the errors caused by the measurement of environment factors.

Description

Technical Field

[0001] This application relates to the field of multi-light imaging image technology, and in particular to a multi-light measurement environmental disturbance adaptive compensation method, device and medium. Background Art

[0002] Ultraviolet light and infrared light signals are commonly present in fire point sources, power discharges, and natural light sources, etc. Multi-light fusion detection devices have gradually developed and are widely used in the fields of fire protection and power industries, and are favored by people.

[0003] However, the disturbances under complex environmental conditions (such as rain, snow, haze, and fire fog) will cause great interference to sensor imaging. At the same time, the imaging technologies of light waves in different bands often vary greatly in the changes caused by the disturbance of the same environmental factor. Therefore, there is an urgent need for a method that can process and optimize the images generated by all-band optical imaging devices under complex environmental conditions. Summary of the Invention

[0004] The purpose of the present invention is to provide a multi-light measurement environmental disturbance adaptive compensation method, device and medium in order to solve the problem that the existing image processing methods cannot perform adaptive compensation on all-band optical images.

[0005] The above object of this application is achieved by the following technical solutions: S1: Obtain the image to be processed and group it according to the band to obtain an ultraviolet image, a visible light image, and an infrared image; S2: Process the ultraviolet image by using the Bayesian adaptive filtering algorithm and histogram gray transformation to obtain the first ultraviolet image; S3: Process the visible light image through a defogging model and Gamma correction to obtain the first visible light image; S4: Process the infrared image through a temperature compensation model and histogram enhancement algorithm to obtain the first infrared image; S5: Superpose and fuse the first ultraviolet image and the first visible light image to obtain a clear visible light image; S6: Fuse the first visible light image and the first infrared light image to obtain a high-resolution infrared image, and complete the adaptive compensation of the image within a large wavelength range; the clear visible light image, the high-resolution infrared image, and the first ultraviolet image are the images to be processed after adaptive compensation.

[0006] Optionally, step S3 includes: S31: Use the U-Net model as the defogging model; Obtain a labeled visible light image data set; train the U-Net model through the labeled visible light image data set to obtain a trained defogging model; Input the visible light image into the trained defogging model to obtain a defogged image; S32: Perform Gamma correction on the defogged image to obtain the first visible light image.

[0007] Optionally, step S4 includes: S41: Construct a temperature compensation model using a two-layer regression network based on a generalized regression neural network; Obtain an infrared image dataset collected under different environmental conditions, and train the temperature compensation model with the infrared image dataset; Automatically optimize the smoothing parameter in the temperature compensation model through the Grey Wolf Optimization algorithm; Process the infrared image with the optimized temperature compensation model; S42: Process the temperature-compensated infrared image using the neighborhood conditional histogram enhancement method to obtain the first infrared image.

[0008] Optionally, step S5 includes: S51: Binarize the first ultraviolet image and retain the spot pixels; S52: Overlay and fuse the spot pixels onto the corresponding pixel positions of the first visible light image to obtain a clear visible light image.

[0009] Optionally, step S6 includes: Fuse the first visible light image and the first infrared light image using the SGnet network model to obtain a high-resolution infrared image.

[0010] Optionally, the training steps of the SGnet network model are as follows: S61: Obtain existing high-resolution infrared images; resize the existing high-resolution infrared images to obtain low-resolution infrared images; S62: Construct a dataset with the clear visible light image, the existing high-resolution infrared image, and the low-resolution infrared image; S63: Train the SGnet network model with the dataset to obtain an output image; S64: Adjust the parameters of the SGnet network model by calculating the loss value between the output image and the existing high-resolution infrared image and combining the backpropagation algorithm.

[0011] An electronic device includes a processor, a memory, a user interface, and a network interface. The memory is used to store instructions. The user interface and the network interface are used to communicate with other devices. The processor is used to execute the instructions stored in the memory so that the electronic device executes a multi-light measurement environment perturbation adaptive compensation method.

[0012] A computer-readable storage medium stores instructions. When the instructions are executed, a method for adaptively compensating for multi-light measurement environment disturbance is performed.

[0013] The beneficial effects of the technical solution provided by this application are: 1. Adaptive image processing operations are performed according to the wavelength groups of the images to be processed. Differentiated processing operations are performed for the characteristics of ultraviolet, visible light, and infrared images, and targeted compensation and enhancement are performed for images in a larger wavelength range in complex environments (rain, snow, fog, haze, and fire fog, etc.), which can greatly reduce the errors caused by measurement environmental factors: the image to be processed is first subjected to a wavelet filtering algorithm to remove noise (the threshold is selected by the Bayesian adaptive threshold algorithm), and then the AHE technology is used to restore the contrast and clarity that have decreased due to scattering, solving the problem that ultraviolet light will be scattered by tiny particles in haze or fog, resulting in a decrease in image contrast and clarity. The image to be processed is first input into the defogging model to remove the fog in the image and restore the lost details. After that, the processed image is subjected to Gamma correction to restore the brightness and contrast of the image, solving the problems of detail loss, darkening, and reduced contrast in visible light images. The image to be processed is first input into the temperature compensation model to compensate for the temperature error caused by attenuation. Then the processed image is enhanced using the neighborhood conditional histogram enhancement method to restore its resolution and brightness, solving the problem that the thermal radiation in the infrared light will attenuate due to scattering, resulting in errors in the temperature recognized by the sensor, and the infrared image will have reduced resolution and darkened images. Finally, through the processed ultraviolet image, visible light image and infrared image, a clear visible light image and a high-resolution infrared image are obtained, and combined with the trained neural network model, the image to be processed after adaptive compensation is obtained.

[0014] 2. The first visible light image and the first infrared light image are fused through the SGnet network model to obtain a high-resolution infrared image with clearer details. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The present application will be further described below with reference to the accompanying drawings and embodiments, in which: Figure 1 is a step diagram of a method for adaptively compensating for multi-light measurement environment disturbances in an embodiment of the present application; Figure 2 SGnet network structure diagram of the multi-light measurement environment disturbance adaptive compensation method in the embodiment of the present application; Figure 3 It is a schematic diagram of the electronic device structure of the multi-light measurement environment disturbance adaptive compensation method in the embodiment of the present application. DETAILED DESCRIPTION

[0016] In order to have a clearer understanding of the technical features, objectives, and effects of this application, the specific implementation manners of this application will now be described in detail with reference to the accompanying drawings.

[0017] An embodiment of this application provides a method for adaptively compensating for disturbances in a multi-light measurement environment.

[0018] Please refer to Figure 1 , Figure 1 which is a step diagram of a method for adaptively compensating for disturbances in a multi-light measurement environment in an embodiment of this application, including: S1: Obtain the image to be processed and group it according to the wavelength band to obtain an ultraviolet image, a visible light image, and an infrared image; An embodiment provided by this application is as follows. Before performing processing optimization, it is necessary to group the light wave range of the image to be processed to facilitate subsequent adaptive image processing operations.

[0019] An embodiment provided by this application is as follows. For the ultraviolet image (wavelength 10~390nm), the present invention uses the Bayesian adaptive filtering algorithm to solve the problem of noise often existing in the ultraviolet image, and then uses histogram gray transformation to enhance the image contrast; for the visible light image (wavelength 390~780nm), the present invention first uses a defogging model to solve the problem of image degradation caused by harsh conditions such as rain, snow, and fog, and then uses Gamma correction on the defogged image to change the image brightness and contrast for image enhancement; for the infrared image (wavelength above 780nm), the present invention first uses a temperature compensation model to solve the temperature error problem caused by environmental factors such as atmospheric absorption and ambient temperature interference, and then uses a histogram enhancement algorithm to enhance the compensated image.

[0020] S2: Process the ultraviolet image using the Bayesian adaptive filtering algorithm and histogram gray transformation to obtain a first ultraviolet image; An embodiment provided by this application is as follows. In terms of the filtering method, the present invention uses the wavelet filtering algorithm. Compared with the traditional frequency domain filtering, the multi-resolution analysis idea adopted by wavelet analysis can decompose the intertwined frequency components into different sub-bands. By processing different sub-bands, better filtering effects can be obtained. The process of wavelet filtering is similar to that of frequency domain low-pass filtering. First, the image is decomposed at multiple scales, and a certain algorithm is used to process the wavelet high-frequency coefficients, and then the image is reconstructed to achieve the filtering effect. The more crucial problem lies in how to select the threshold. A Bayesian adaptive threshold algorithm is adopted. Since the image is composed of many piecewise smooth functions with different characteristics, different local regions have different singular characteristics, so different thresholds need to be set for different sub-blocks of the image.

[0021] An embodiment of the present application is provided as follows, histogram gray-scale transformation: The present invention uses local adaptive histogram equalization (AHE) technology for histogram gray-scale transformation. Adaptive histogram equalization divides the image into many small regions or templates, and then performs histogram equalization processing on the pixels within each template. This process is adaptive and will adjust the equalization effect according to the local characteristics of the image. For example, if a certain area of the image is darker, the brightness of this area will be increased to enhance the contrast and visual effect of the image. On the contrary, if a certain area of the image is brighter, the brightness of this area will be reduced to prevent overexposure or distortion of the image.

[0022] S3: Process the visible light image through a defogging model and Gamma correction to obtain a first visible light image; S4: Process the infrared image through a temperature compensation model and a histogram enhancement algorithm to obtain a first infrared image; S5: Superimpose and fuse the first ultraviolet image and the first visible light image to obtain a clear visible light image; S6: Fuse the first visible light image and the first infrared light image to obtain a high-resolution infrared image, completing the adaptive compensation of the image within a large wavelength range; The clear visible light image, the high-resolution infrared image, and the first ultraviolet image are the images to be processed after adaptive compensation.

[0023] Step S3 includes: S31: Use the U-Net model as the defogging model; Obtain a labeled visible light image dataset; Train the U-Net model through the labeled visible light image dataset to obtain a trained defogging model; Input the visible light image into the trained defogging model to obtain a defogged image; It should be noted that the U-Net network structure includes: an encoding stage and a decoding stage. The encoding stage consists of a series of convolutional layers and downsampling layers, which are used to gradually reduce the spatial dimension while increasing the feature dimension, capturing abstract and global context information. The decoding stage consists of a series of convolutional layers and upsampling layers, gradually restoring the spatial dimension while merging the feature maps of the corresponding layers in the encoding stage to ensure that low-level details are retained.

[0024] It should be noted that through these operations, functions such as upsampling of feature maps, feature fusion, and detail reconstruction can be achieved. During the decoding process, skip connections are used to connect different layers between the encoding module and the decoding module to help better transmit information and restore details.

[0025] S32: Perform Gamma correction on the defogged image to obtain a first visible light image.

[0026] An embodiment of the present application is provided as follows. By adjusting the Gamma curve of an image, the brightness and contrast of the dehazed image can be changed in a non-linear manner, which is used to correct the image captured by the sensor under non-standard lighting conditions.

[0027] Step S4 includes: S41: Construct a temperature compensation model by using a two-layer regression network based on the Generalized Regression Neural Network (GRNN); An embodiment of the present application is provided as follows. The acquired infrared, ultraviolet, and visible light image datasets are all collected under different environmental conditions, including different weather conditions, temperatures, humidities, and lighting conditions.

[0028] An embodiment of the present application is provided as follows. The input of the first-layer GRNN network in the two-layer regression network is the six-dimensional data of x1, x2... Y of the thermal infrared monitoring temperature and the atmospheric influence, and the output is the first-layer pre-training result Ypred; the second-layer GRNN network replaces Y in the input data with Y - Ypred, and outputs the second-layer pre-training result Δpred, obtaining the two-dimensional pre-training data after being trained by the two-layer GRNN. Input the two-dimensional pre-training result into the Logistic regression function to obtain the temperature compensation model.

[0029] Acquire the infrared image dataset collected under different environmental conditions, and train the temperature compensation model through the infrared image dataset; Automatically optimize the smoothing parameter in the temperature compensation model through the Grey Wolf Optimization algorithm; Process the infrared image through the optimized temperature compensation model; It should be noted that since manually setting hyperparameters may cause the dynamic evaluation model to fall into a local optimal solution, the present invention uses the Grey Wolf Optimization (GWO) algorithm to automatically optimize the smoothing parameter in the two-layer regression network.

[0030] S42: Process the temperature-compensated infrared image by using the neighborhood condition histogram enhancement method to obtain the first infrared image.

[0031] An embodiment of the present application is provided as follows. The weight information of the difference between the central pixel and the neighborhood pixels is added in the calculation of the histogram of a certain gray-level pixel, so that the details with larger gray-level gradients such as the image edge obtain larger statistical weights, while the gray-level flat areas such as the image background obtain smaller statistical weights.

[0032] Step S5 includes: S51: Binarize the first ultraviolet image and retain the spot pixels; S52: Overlay the light spot pixels onto the pixel positions corresponding to the first visible light image for superposition and fusion to obtain a clear visible light image.

[0033] An embodiment provided by this application is as follows. Since the ultraviolet light spot image (the first ultraviolet image) usually consists of multiple light spots and does not contain the detailed features of the actual object, but in practical applications, it is necessary to judge the discharge position and discharge degree of the object according to the area of the light spot and the position where it appears on the object (generally speaking, the larger the area of the light spot, the higher the discharge degree). Therefore, it is necessary to simply superimpose and fuse the high-quality ultraviolet light spot image obtained after the first two steps of processing with the high-quality visible light image obtained through the same processing to obtain a clear visible light image. The clear visible light image is convenient for subsequent judgment and processing of the object's discharge position and discharge degree.

[0034] Step S6 includes: Using the SGnet network model to fuse the first visible light image and the first infrared light image to obtain a high-resolution infrared image.

[0035] The training steps of the SGnet network model are as follows: S61: Obtain the existing high-resolution infrared image; adjust the size of the existing high-resolution infrared image to obtain a low-resolution infrared image; An embodiment provided by this application is as follows. When the drone captures a set of available corresponding visible light image and infrared image, input the two images into the drone-borne model. The model will output a high-resolution infrared image. At this time, replace the low-resolution infrared image with the high-resolution infrared image and save it.

[0036] S62: Construct a data set through the clear visible light image, the existing high-resolution infrared image, and the low-resolution infrared image; S63: Train the SGnet network model through the data set to obtain an output image; S64: Calculate the loss value between the output image and the existing high-resolution infrared image, and combine the backpropagation algorithm to adjust the parameters of the SGnet network model.

[0037] An embodiment provided by this application is as follows. The network structure of the SGnet network model is as Figure 2As shown in the figure, it mainly includes a Gradient Calibration Module (GCM) and a Frequency Awareness Module (FAM). The GCM contains multiple Spectrum Difference Blocks (SDBs), which are designed to restore more accurate detailed structures in the gradient and frequency domains. The FAM refines the detailed features by recursively executing the SDBs. The SDB module upsamples the infrared image and fuses it with the visible light image through convolution. Then, the fused feature map and the visible light image are mapped to the frequency domain through Fourier transform. After that, the amplitude features and phase features of the two in the frequency domain are respectively fed into separate convolutions for fused feature learning. Finally, the feature map enhanced in the frequency domain is obtained through inverse Fourier transform. The SDB module aims to strengthen the connection between the spatial domain and the frequency domain and thereby enhance the restoration of details. The loss function of the SGnet network model includes gradient perception loss , frequency perception loss and spatial perception loss .

[0038] This application also discloses an electronic device. Referring to Figure 3 , Figure 3 is a schematic structural diagram of an electronic device disclosed in an embodiment of this application. The electronic device 500 may include: at least one processor 501, at least one network interface 504, a user interface 503, a memory 505, and at least one communication bus 502.

[0039] Among them, the communication bus 502 is used to realize the connection and communication between these components.

[0040] Among them, the user interface 503 may include a display screen. Optionally, the user interface 503 may further include a standard wired interface and a wireless interface.

[0041] Among them, the network interface 504 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface).

[0042] This application also discloses a computer-readable storage medium, which stores multiple instructions suitable for a processor to load and execute the above-mentioned multi-optical measurement environment disturbance adaptive compensation method.

[0043] The above are only exemplary embodiments of the present disclosure and should not be used to limit the scope of the present disclosure. That is, any equivalent changes and modifications made in accordance with the teachings of the present disclosure still fall within the scope covered by the present disclosure. Those skilled in the art will easily think of other implementation schemes of the present disclosure after considering the specification and the practice of the present disclosure.

[0044] This application is intended to cover any variations, uses, or adaptations of the present disclosure, which follow the general principles of the present disclosure and include well-known knowledge or conventional technical means in the technical field not recorded in the present disclosure. The description and examples are only regarded as exemplary, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. A multi-light measurement environmental disturbance adaptive compensation method, characterized in that The method includes the following steps: S1: Obtain the image to be processed and group it according to the wavelength band to obtain an ultraviolet image, a visible light image, and an infrared image; S2: Process the ultraviolet image using the Bayesian adaptive filtering algorithm and histogram gray-scale transformation to obtain a first ultraviolet image; S3: Process the visible light image through a defogging model and Gamma correction to obtain a first visible light image; S4: Process the infrared image through a temperature compensation model and a histogram enhancement algorithm to obtain a first infrared image; S5: Superimpose and fuse the first ultraviolet image and the first visible light image to obtain a clear visible light image; S6: Fuse the first visible light image and the first infrared light image to obtain a high-resolution infrared image, completing the adaptive compensation of the image within a large wavelength range; the clear visible light image, the high-resolution infrared image, and the first ultraviolet image are the images to be processed after adaptive compensation.

2. The multi-light measurement environment disturbance adaptive compensation method according to claim 1, characterized in that, Step S3 includes: S31: Use the U-Net model as the defogging model; Obtain a labeled visible light image dataset; train the U-Net model with the labeled visible light image dataset to obtain a trained defogging model; Input the visible light image into the trained defogging model to obtain a defogged image; S32: Perform Gamma correction on the defogged image to obtain a first visible light image.

3. The multi-light measurement environment disturbance adaptive compensation method according to claim 1, wherein, Step S4 includes: S41: Use a double-layer regression network based on a generalized regression neural network to construct a temperature compensation model; Obtain an infrared image dataset collected under different environmental conditions and train the temperature compensation model with the infrared image dataset; Automatically optimize the smoothing parameter in the temperature compensation model through the grey wolf optimization algorithm; Process the infrared image through the optimized temperature compensation model; S42: Process the temperature-compensated infrared image using the neighborhood condition histogram enhancement method to obtain a first infrared image.

4. The multi-light measurement environmental disturbance adaptive compensation method according to claim 1, characterized in that Step S5 includes: S51: Binarize the first ultraviolet image and retain the spot pixels; S52: Cover the spot pixels to the corresponding pixel positions of the first visible light image for superimposed fusion to obtain a clear visible light image.

5. The multi-light measurement environment disturbance adaptive compensation method according to claim 1, characterized in that Step S6 includes: Use the pre-trained SGnet network model to fuse the first visible light image and the first infrared light image to obtain a high-resolution infrared image.

6. The multi-light measurement environment disturbance adaptive compensation method according to claim 5, wherein The training steps of the SGnet network model are as follows: S61: Obtain the existing high-resolution infrared image; adjust the size of the existing high-resolution infrared image to obtain a low-resolution infrared image; S62: Construct a dataset through the clear visible light image, the existing high-resolution infrared image, and the low-resolution infrared image; S63: Train the SGnet network model with the dataset to obtain an output image; S64: Adjust the parameters of the SGnet network model by calculating the loss value between the output image and the existing high-resolution infrared image and combining the backpropagation algorithm.

7. An electronic device, characterized in that, It includes a processor (501), a memory (505), a user interface (503) and a network interface (504). The memory (505) is used to store instructions. The user interface (503) and the network interface (504) are used to communicate with other devices. The processor (501) is used to execute the instructions stored in the memory (505) so that the electronic device executes the method described in any one of claims 1-6.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions which, when executed by a computer, perform the method steps described in any one of claims 1-6.

Images