Image illumination anomaly processing method and device, electronic equipment and storage medium

Abstract

The invention discloses an image illumination anomaly processing method and device, electronic equipment and a storage medium, which are used for solving the technical problem that the processing effect and the real-time performance cannot be considered at the same time in the prior art. Comprising the following steps: acquiring a brightness component of an RGB image to construct a single-channel brightness matrix; marking an overexposure area and a low-illumination area of the single-channel brightness matrix and calculating a global illumination intensity index; adjusting bilateral filtering parameters according to the global illumination intensity index to adjust a single-channel brightness matrix, and generating a filtered brightness matrix; carrying out enhancement through a Retinex algorithm to obtain an enhanced brightness matrix; determining a key area according to the enhanced brightness matrix and generating a local processing strategy; performing two-layer wavelet transform on the RGB image to obtain a plurality of frequency band components; adjusting each frequency band component according to the global illumination intensity index and the local processing strategy; reconstructing and adjusting the frequency band component through wavelet inverse transformation to obtain a reconstructed RGB image; and converting the reconstructed RGB image into a target image in a preset format.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to an image light abnormality processing method and device, electronic equipment and a storage medium. BACKGROUND

[0002] In a complex industrial scene, multiple objects and complex lighting (such as overexposure, shadows, strong light reflection, etc.) result in low distinction between objects and background environment, which seriously affects the accuracy of subsequent target detection. Traditional light processing methods, such as fixed parameter histogram equalization and Gaussian filtering, have obvious defects when dealing with complex lighting: it is difficult to dynamically adjust the processing strategy according to real-time lighting conditions, and the processing effect on overexposed areas and low-light areas is not good, resulting in insufficient contrast enhancement of objects and background, and reduced target detection accuracy.

[0003] In addition, with the widespread application of edge computing devices in industrial scenes, the problem of limited computing power on the edge side is increasingly prominent. Traditional methods have large computational complexity when processing high-resolution images, making it difficult to meet real-time requirements and limiting their application in actual scenarios.

[0004] Existing similar technologies mainly focus on single spatial or frequency domain light optimization, lack of collaborative processing of spatial and frequency domains, and do not address the problem of limited computing power on the edge side, making it difficult to balance processing effect and real-time performance. SUMMARY

[0005] The present application provides an image light abnormality processing method, device, electronic equipment and storage medium, which solves the technical problem that existing technologies cannot balance processing effect and real-time performance.

[0006] The present application provides an image light abnormality processing method, comprising:

[0007] Collecting image data and converting the image data into an RGB image;

[0008] Obtaining a luminance component from the RGB image and constructing a single-channel luminance matrix using the luminance component;

[0009] Traversing the single-channel luminance matrix, marking overexposed areas and low-light areas, and calculating a global lighting intensity index;

[0010] Determining the spatial interference signal distribution according to the overexposed area;

[0011] Adjusting the bilateral filtering parameters according to the global lighting intensity index;

[0012] Adjusting the single-channel luminance matrix based on the bilateral filtering parameters to generate a filtered luminance matrix;

[0013] enhancing the filtered luminance matrix based on the spatial interference signal distribution by a Retinex algorithm to obtain an enhanced luminance matrix;

[0014] determining a key region according to the enhanced luminance matrix, and generating a local processing strategy of the key region;

[0015] performing two-layer wavelet transform on the RGB image to obtain a plurality of frequency band components;

[0016] adjusting the frequency band components according to the global illumination intensity index and the local processing strategy to obtain adjusted frequency band components;

[0017] reconstructing the adjusted frequency band components by inverse wavelet transform to obtain a reconstructed RGB image;

[0018] converting the reconstructed RGB image into a target image in a preset format.

[0019] Optionally, the step of determining the spatial interference signal distribution according to the overexposure region includes:

[0020] generating an overexposure sub-matrix of the overexposure region;

[0021] performing fast Fourier transform on the overexposure sub-matrix to obtain a frequency domain amplitude spectrum;

[0022] calculating distances from each frequency domain coordinate in the frequency domain amplitude spectrum to a center point of the frequency domain amplitude spectrum;

[0023] determining an amplitude threshold value according to all the frequency domain coordinates;

[0024] generating a noise frequency mask according to the distances, the frequency domain coordinates, a preset distance threshold value and the amplitude threshold value;

[0025] calculating a product of the noise frequency mask and the frequency domain coordinates to obtain a filtered frequency domain signal;

[0026] performing inverse fast Fourier transform on the filtered frequency domain information to obtain a spatial domain signal;

[0027] identifying a non-zero region in the spatial domain signal to obtain the spatial interference signal distribution.

[0028] Optionally, the step of determining the key region according to the enhanced luminance matrix and generating the local processing strategy of the key region includes:

[0029] extracting a luminance feature of the enhanced luminance matrix, and calculating a distribution probability of the luminance feature;

[0030] determining a key region according to the distribution probability, and generating a local processing strategy of the key region.

[0031] Optionally, the step of adjusting each frequency band component according to the global illumination intensity index and the local processing strategy comprises:

[0032] adjusting a weight of each frequency band component according to the global illumination intensity index;

[0033] optimizing the weight according to the global illumination intensity index and the local processing strategy to obtain an optimized weight;

[0034] adjusting the brightness of each frequency band component according to the optimized weight to obtain an adjusted frequency band component.

[0035] Optionally, the bilateral filtering parameters comprise a spatial domain standard deviation and a color domain standard deviation; and the step of adjusting the bilateral filtering parameters according to the global illumination intensity index comprises:

[0036] when the global illumination intensity index is less than a first preset threshold, increasing the spatial domain standard deviation;

[0037] when the global illumination intensity index is greater than a second preset threshold, decreasing the color domain standard deviation.

[0038] Optionally, the frequency band component comprises a low frequency component and a high frequency component; and the step of optimizing the weight according to the global illumination intensity index and the local processing strategy to obtain an optimized weight comprises:

[0039] increasing the weight of the low frequency component in the low illumination area and decreasing the weight of the high frequency component in the low illumination area to obtain low illumination area optimized weights;

[0040] decreasing the weight of the high frequency component in the overexposure area and increasing the weight of the low frequency component in the overexposure area to obtain overexposure area optimized weights.

[0041] Optionally, after the step of converting the reconstructed RGB image into a target image in a preset format, the method further comprises:

[0042] determining whether the fluctuation of the global illumination intensity index of five consecutive target images exceeds 20%;

[0043] if yes, returning to the step of collecting image data and converting the image data into an RGB image.

[0044] The present application also provides an image illumination abnormality processing device, comprising:

[0045] an image collection and conversion module configured to collect image data and convert the image data into an RGB image;

[0046] a single-channel brightness matrix construction module configured to obtain a brightness component from the RGB image and construct a single-channel brightness matrix using the brightness component;

[0047] a global illumination intensity index calculation module configured to traverse the single-channel brightness matrix, mark overexposed regions and low-light regions, and calculate a global illumination intensity index;

[0048] a spatial interference signal distribution determination module configured to determine a spatial interference signal distribution according to the overexposed regions;

[0049] a bilateral filtering parameter adjustment module configured to adjust bilateral filtering parameters according to the global illumination intensity index;

[0050] a filtered brightness matrix generation module configured to adjust the single-channel brightness matrix based on the bilateral filtering parameters to generate a filtered brightness matrix;

[0051] an enhanced brightness matrix generation module configured to enhance the filtered brightness matrix based on the spatial interference signal distribution by a Retinex algorithm to obtain an enhanced brightness matrix;

[0052] a local processing strategy generation module configured to determine key regions according to the enhanced brightness matrix and generate a local processing strategy for the key regions;

[0053] a frequency band component acquisition module configured to perform two-layer wavelet transform on the RGB image to obtain a plurality of frequency band components;

[0054] a frequency band component adjustment module configured to adjust each frequency band component according to the global illumination intensity index and the local processing strategy to obtain adjusted frequency band components;

[0055] a reconstruction module configured to reconstruct the adjusted frequency band components by inverse wavelet transform to obtain a reconstructed RGB image;

[0056] a target image generation module configured to convert the reconstructed RGB image into a target image in a preset format.

[0057] The application further provides an electronic device, which comprises a processor and a memory:

[0058] The memory is configured to store program code and transmit the program code to the processor;

[0059] The processor is configured to execute the image illumination abnormality processing method according to the instructions in the program code.

[0060] The application further provides a computer readable storage medium for storing program codes for executing the image light abnormality processing method according to any one of the above.

[0061] From the above technical solutions, the application has the following advantages: the application provides an image light abnormality processing method, and specifically discloses the following: collecting image data and converting the image data into an RGB image; obtaining a luminance component from the RGB image and constructing a single-channel luminance matrix using the luminance component; traversing the single-channel luminance matrix, marking an overexposure area and a low-light area, and calculating a global light intensity index; positioning a spatial interference signal distribution according to the overexposure area; adjusting a bilateral filtering parameter according to the global light intensity index; adjusting the single-channel luminance matrix based on the bilateral filtering parameter to generate a filtered luminance matrix; enhancing the filtered luminance matrix through a Retinex algorithm to obtain an enhanced luminance matrix; determining a key area according to the enhanced luminance matrix and generating a local processing strategy for the key area; performing two-layer wavelet transformation on the RGB image to obtain a plurality of frequency band components; adjusting the frequency band components according to the global light intensity index and the local processing strategy to obtain adjusted frequency band components; reconstructing the adjusted frequency band components through inverse wavelet transformation to obtain a reconstructed RGB image; and converting the reconstructed RGB image into a target image in a preset format. The application can automatically adjust processing parameters according to real-time light conditions through a dynamic parameter adaptive mechanism, without manual intervention, thereby improving the universality and robustness of the method and making it suitable for various complex industrial scenes. BRIEF DESCRIPTION OF DRAWINGS

[0062] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description only constitute some embodiments of the application, and for those skilled in the art, other drawings can also be obtained from these drawings without creative labor.

[0063] Figure 1 A schematic diagram of a dynamic spatial and frequency domain collaborative adaptive light optimization process;

[0064] Figure 2 A step flowchart of an image light abnormality processing method provided by the embodiment of the application;

[0065] Figure 3 A structural block diagram of an image light abnormality processing device provided by the embodiment of the application. DETAILED DESCRIPTION

[0066] The embodiment of the present application provides an image light abnormality processing method and device, electronic equipment and storage medium, which are used for solving the technical problem that the prior art cannot simultaneously consider processing effect and real-time performance.

[0067] In order to make the application purpose, features and advantages of the present application more obvious and easy to understand, the technical solutions in the embodiments of the present application will be clearly and completely described below in combination with the drawings in the embodiments of the present application. Obviously, the following described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0068] Please refer to Figure 1 , Figure 1 For a dynamic spatial and frequency domain collaborative adaptive light optimization process schematic diagram. The process specifically includes the following modules:

[0069] Module 1: Light intensity perception module: based on HSV space, the brightness component of the image is statistically analyzed, the global light intensity index and the brightness distribution characteristics of the local area are obtained in real time, and the light abnormality area such as overexposure and shadow is accurately located.

[0070] Module 2: bilateral filtering and Retinex fusion module: the sigma space / color parameters of bilateral filtering are dynamically adjusted according to the light intensity, smaller parameters are used in the uniform light area to maintain details, and the parameters are increased in the light mutation area to enhance edge protection. The scale factor of the Retinex algorithm is optimized according to the frequency domain analysis result, which realizes more accurate estimation of the light component and the reflection component. Through the real-time analysis of the image histogram distribution by the lightweight CNN, the CBAM attention map is combined as the fusion weight, and the dynamic switching of the local area processing strategy is realized.

[0071] Module 3: lightweight CBAM attention network: the channel attention of the standard CBMA is divided into 4 parallel subgroups, each subgroup processes 1 / 4 channel dimension, and the calculation complexity is reduced. The low-rank approximation method is used to compress the spatial attention matrix calculation amount, a cross-layer feature reuse mechanism is designed, the number of convolution layers is reduced while the feature expression ability is maintained. The effective features are strengthened through the brightness guided channel attention mechanism, and the high frequency noise in the overexposure area is suppressed.

[0072] Module 4: frequency domain collaborative optimization module: based on wavelet transform, the image is decomposed into low-frequency, medium-frequency and high-frequency components, the fusion weight of each frequency band is dynamically adjusted according to the global light intensity index, the low-frequency component is enhanced to improve the overall light, and the high-frequency noise is suppressed to reduce the overexposure influence. The space-frequency dual-path attention mechanism is constructed, the key area is focused through the improved CBAM module in the spatial domain, the abnormal frequency band is suppressed by using the Fourier mask in the frequency domain, and the collaborative optimization of the spatial domain and the frequency domain is realized.

[0073] Based on the above process, please refer to Figure 2 The embodiment of the application provides a step flow chart of an image light abnormality processing method.

[0074] The image light abnormality processing method provided by the application can specifically include the following steps:

[0075] Step 201, collecting image data and converting the image data into an RGB image;

[0076] In the embodiment of the application, an industrial scene camera real-time stream or a local image file can be accessed as image data, and then the image data resolution is uniformly set to a preset size (such as 1280*720), and the image data is converted into an RGB image in an RGB three-channel floating-point data format, thereby providing a unified input basis for subsequent processing.

[0077] Step 202, obtaining a luminance component according to the RGB image, and constructing a single-channel luminance matrix using the luminance component;

[0078] After obtaining the standardized RGB image, the image can be converted into an HSV color space, the luminance component (V channel) is separated out, and a single-channel luminance matrix is constructed according to the luminance component, thereby providing original data for light intensity analysis.

[0079] Step 203, traversing the single-channel luminance matrix, marking an overexposure area and a low-light area, and calculating a global light intensity index;

[0080] After the single-channel luminance matrix is generated, the single-channel luminance matrix can be traversed, an overexposure threshold and a shadow threshold can be set, the overexposure area and the low-light area are marked according to the overexposure threshold and the shadow threshold, and the global luminance mean μ and the luminance standard deviation The global light intensity index (GLI) is calculated, wherein the GLI is:

[0081]

[0082]

[0083]

[0084] wherein, is the coordinate pixel luminance, is the image size, normalized luminance mean (pixel luminance range 0-255), normalized standard deviation (theoretically maximum standard deviation is about 127.5), and finally the GLI is mapped to the interval [0, 1], thereby quantifying the scene light intensity and uniformity.

[0085] Extraction of overexposed region: traverse the single-channel luminance matrix, mark all overexposed pixels according to the overexposed threshold, then merge the continuous overexposed pixels, and crop the sub-matrix with the size of M*N as the overexposed region The height and width of the overexposed region are determined as the overexposed region.

[0086] The extraction process of the low-light region can refer to the extraction process of the overexposed region, which will not be repeated here.

[0087] Step 204, determining the spatial interference signal distribution according to the overexposed region;

[0088] In the embodiment of the application, after the overexposed region is determined, a two-dimensional fast Fourier transform (FFT) can be performed on the overexposed region to obtain a frequency domain amplitude spectrum, then a frequency band in the high-frequency noise is counted to generate a noise frequency mask, so as to locate the spatial interference signal distribution.

[0089] In one example, step 204 can include the following sub-steps:

[0090] S41, generating an overexposed sub-matrix of the overexposed region;

[0091] S42, performing a fast Fourier transform on the overexposed sub-matrix to obtain a frequency domain amplitude spectrum;

[0092] S43, calculating the distance from each frequency domain coordinate in the frequency domain amplitude spectrum to the center point of the frequency domain amplitude spectrum;

[0093] S44, determining an amplitude threshold according to all frequency domain coordinates;

[0094] S45, generating a noise frequency mask according to the distance, the frequency domain coordinate, the preset distance threshold, and the amplitude threshold;

[0095] S46, calculating the product of the noise frequency mask and the frequency domain coordinate to obtain a filtered frequency domain signal;

[0096] S47, performing an inverse fast Fourier transform on the filtered frequency domain information to obtain a spatial domain signal;

[0097] S48, identifying a non-zero region in the spatial domain signal to obtain the spatial interference signal distribution.

[0098] In the embodiment of the application, the process of performing FFT transform on the overexposed region is as follows:

[0099] The overexposed sub-matrix is subjected to a two-dimensional FFT to convert it from the spatial domain to the frequency domain, and the formula is as follows:

[0100]

[0101] ​​​where, is the spatial coordinate (pixel position) of the overexposed sub-matrix; is the transformed frequency coordinate.

[0102] Then the frequency magnitude spectrum (i.e. the modulus of the complex ) is calculated, formula:

[0103]

[0104] where Re is the real part of the complex number, Im is the imaginary part of the complex number.

[0105] In the frequency domain, the coordinate is farther away from the center, the corresponding spatial signal frequency is higher. Set the distance threshold (take 1 / 5 of the frequency domain size, i.e. , satisfy the frequency components, defined as high-frequency noise (such components correspond to the rapidly changing details / noise in the spatial domain, such as color bands, speckles in overexposed areas).

[0106] Based on the defined distance threshold, the noise frequency mask can be generated, the process is as follows:

[0107] (1) Calculate the distance of the frequency coordinate to the center :

[0108]

[0109] (2) If (belong to high-frequency noise), and (amplitude threshold, take 2 times the average of the full frequency domain amplitude), then (label as noise frequency), otherwise (non-noise frequency).

[0110]

[0111] Combined with the noise frequency mask , the spatial interference signal can be located reversely, the process is as follows:

[0112] (1) Frequency domain filtering: multiply with the frequency magnitude spectrum , keep the noise frequency components, get the filtered frequency domain signal:

[0113]

[0114] (2) Inverse FFT transformation: perform inverse FFT on the filtered frequency domain signal (keep the phase, only mask the amplitude) to return to the spatial domain:

[0115]

[0116] (3) Spatial domain positioning: The non-zero region in the overexposed region is the distribution position (the higher the amplitude, the more significant the interference) of the spatial domain interference signal (high-frequency noise).

[0117] Step 205, adjusting the bilateral filtering parameters according to the global illumination intensity index;

[0118] In the embodiment of the application, step 205 can include the following sub-steps:

[0119] S51, when the global illumination intensity index is less than a first preset threshold, increasing the spatial domain standard deviation;

[0120] S52, when the global illumination intensity index is greater than a second preset threshold, decreasing the color domain standard deviation.

[0121] In a specific implementation, the bilateral filtering core parameters are the spatial domain standard deviation (control the spatial domain smoothing range) and the color domain standard deviation (control the color similarity weight), and the adjustment logic is:

[0122] If GLI<0.5 (low light scene, image details are easy to be covered by noise): increasing the spatial domain standard deviation to 15 (default 10), expanding the spatial domain smoothing range, and strengthening the edge and detail retention; the color domain standard deviation remains the default 0.3, avoiding excessive color gradient blurring.

[0123] If GLI>0.8 (overexposed scene, color is easy to distort): decreasing the color domain standard deviation to 0.1 (default 0.3), tightening the color similarity judgment, and suppressing the color overflow caused by overexposure; the spatial domain standard deviation remains the default 10, balancing the edge retention and noise suppression.

[0124] Step 206, adjusting the single-channel brightness matrix based on the bilateral filtering parameters to generate a filtered brightness matrix;

[0125] In the embodiment of the application, the bilateral filtering realizes "edge-preserving denoising" through the joint weight of the spatial domain and the color domain, and the following processing is performed on the brightness matrix:

[0126] The bilateral filtering execution formula is: for the pixel at coordinates in the brightness matrix, the filtered brightness is:

[0127]

[0128] wherein, is normalized weight, is the neighborhood pixel coordinate.

[0129]

[0130] Step 207, based on the spatial interference signal distribution, the filtered brightness matrix is enhanced by Retinex algorithm to obtain the enhanced brightness matrix;

[0131] In the embodiment of the application, the multi-scale Retinex (MSR) can enhance the image dynamic range and details by simulating human eye vision, and the filtered brightness matrix after bilateral filtering is input, the large-scale factor is optimized and processed:

[0132] (1) Large-scale factor adjustment rule: multi-scale Retinex contains multiple scale factors (control filtering range), for the noise of overexposed area, adjust the large-scale factor : if the high-frequency noise energy proportion of the overexposed area is > 30%, the large-scale factor is reduced from the default 80 to 50, the large-scale filtering range is reduced, and the interference of background light on details is reduced.

[0133] (2) Multi-scale Retinex execution process: adopt "double-scale Retinex" to simplify calculation (considering efficiency and effect), formula:

[0134]

[0135] wherein, , is the Gaussian filtering kernel corresponding to the scale factor (small scale, default unchanged) and (large scale, dynamically adjusted); is the scale weight, which is 1; * represents convolution operation, which simulates different scale "background light" through Gaussian filtering, and enhances details through logarithmic operation.

[0136] (3) After processing, the Retinex enhanced brightness matrix is obtained, and the formula is de-normalized:

[0137]

[0138] As the enhanced brightness feature, it is transmitted to the subsequent steps.

[0139] Step 208, determine the key area according to the enhanced brightness matrix, and generate the local processing strategy of the key area;

[0140] After generating the enhanced brightness matrix, the brightness adjustment can be extracted therefrom by a lightweight CNN, and the distribution probability is calculated according to the brightness feature, so as to generate a local processing strategy in combination with CBAM focusing on a key region.

[0141] In one example, step 208 can include the following sub-steps:

[0142] S81, extracting the brightness feature of the enhanced brightness matrix, and calculating the distribution probability of the brightness feature;

[0143] S82, determining a key region according to the distribution probability, and generating a local processing strategy of the key region.

[0144] In a specific implementation, the process of calculating the distribution probability is as follows:

[0145] 1. Histogram equalization feature extraction (CNN pre-processing): for the input brightness matrix (from the enhanced after step 207), first perform adaptive histogram equalization (AHE) to enhance local contrast, to provide more rich features for CNN: AHE equalizes the histogram by sliding window statistics, and the formula core is:

[0146]

[0147] wherein, CDF is the cumulative distribution function in the window, is the sliding window with the center at .

[0148] 2. Lightweight CNN feature extraction and brightness probability calculation: construct a lightweight CNN network, input extract features and calculate pixel-level brightness distribution probability:

[0149] (1) CNN network structure:

[0150] Input layer: 1 channel (brightness matrix), size ;

[0151] Feature layer: 3 layers of depth separable convolution (kernel 3x3, channel number 8→16→32) + 2 layers of maximum pooling (2x2), output feature map (size );

[0152] Output layer: 1x1 convolution + Softmax, map the features back to size, output brightness distribution probability map .

[0153] 3. Brightness distribution probability calculation: for the feature map Each pixel position Softmax computes the probability:

[0154]

[0155] where, is the intensity level (0-255), denotes the coordinate pixel intensity belongs to the level c probability) The final output is a single-channel probability map .

[0156] In the embodiment of the present application, the calculation process of the local processing strategy is as follows:

[0157] 1. Lightweight CBAM attention map generation (pixel-level probability + CBAM fusion): taking the intensity distribution probability map as the input, a channel-spatial hybrid attention map is generated through a lightweight CBAM module, focusing on key areas:

[0158] (1) Channel grouping lightweight processing: the feature map channels are evenly divided into 4 groups (each group has channels), and channel attention is calculated independently for each group:

[0159] Global average pooling (GAP) and global maximum pooling (GMP) are performed on each group of feature maps;

[0160] Sub-channel weights are generated through a shared MLP (multi-layer perceptron, number of neurons ): ;

[0161] The global channel attention is obtained by merging the weights of the 4 groups. .

[0162] (2) Spatial attention low-rank approximation: SVD decomposition is performed on the 3x3 spatial convolution kernel:

[0163]

[0164] The first two singular values are retained, and the rank r=2.

[0165] (3) Cross-layer feature reuse mechanism: shallow features are upsampled through bilinear interpolation and fused with deep features:

[0166]

[0167] This achieves the preservation of the original image edge details, avoids the loss of deep network features, and reduces 20% of the repeated calculations.

[0168] 2. Key object region determination and local mask generation: through attention map Determine key object region (i.e. the region with the most urgent need for illumination optimization) and generate local processing strategy mask :

[0169] (1) Key region determination rule: set the attention value threshold to be 1.5 times the mean value of the attention map Regions that meet the condition are determined as key object regions (such regions have complex brightness distribution and need to be optimized first).

[0170] (2) Local processing strategy mask generation: for key regions, customize processing strategies (such as low-light area enhancement and overexposure area suppression) according to brightness distribution probability Encode as mask :

[0171] If the low-light probability is high (i.e. the low-light area is determined), the mask value is set to 1 (marked as "need to enhance"); If the high-light probability is high (i.e. the overexposure area is determined), the mask value is set to 2 (marked as "need to suppress");

[0172] Otherwise, set to 0 (default processing).

[0173]

[0174] 3. Region weight coefficient calculation: for each key region pixel, calculate the weighted coefficient according to the brightness probability and attention value :

[0175]

[0176] Wherein, is an indicator function used to distinguish low-light / overexposure regions.

[0177] Step 209, performing two-layer wavelet transform on the RGB image to obtain a plurality of frequency band components;

[0178] In the embodiment of the present application, the RGB image is subjected to two-layer wavelet transform to obtain a plurality of frequency band components, including low-frequency component LL and high-frequency component, wherein the high-frequency component includes horizontal high-frequency LH, vertical high-frequency HL and diagonal high-frequency HH.

[0179] ​​​In the embodiment of the present application, after generating the local processing strategy, spatial and frequency domain filtering can be performed before the two-layer wavelet transform of the RGB image. Spatial domain first needs to be guided by attention map filtering. The bilateral filtering parameters calculated in step 205 are used to apply adaptive bilateral filtering to the key area, and the mean filtering is used for the non-key area to reduce the calculation amount; the frequency domain is attenuated by superimposing a noise frequency mask on the frequency domain data of the overexposed area to attenuate the interference signal, while retaining the effective edge frequency component, realizing the cooperative denoising of spatial / frequency domain. This processing provides an optimized basis for the subsequent steps. After denoising, the image noise is reduced and the edge details are retained, so that the two-layer wavelet transform in step 209 can more accurately decompose the low-frequency (reflecting the overall illumination) and high-frequency (reflecting the local details) components; clean frequency band data makes the weight adjustment in step 210 (such as enhancing low frequency in low light area and suppressing high frequency in overexposed area) more effective, ultimately improving the quality of the reconstructed image in step 211, laying the foundation for target image generation. The specific process is as follows:

[0180] 1. Spatial attention guided filtering:

[0181] (1) Key area processing: based on local processing strategy mask For the area marked as "need to enhance" or "need to suppress", apply the dynamically adjusted bilateral filtering parameters:

[0182] Low light area ( ): (spatial domain standard deviation), color domain standard deviation), enhance edge details;

[0183] Overexposed area ( ): , , suppress color distortion.

[0184] (2) Non-key area processing: for the area , 3x3 mean filtering is used to reduce the calculation amount:

[0185]

[0186] (3) Spatial output fusion, combined with the mask to generate the spatial filtering result:

[0187]

[0188] 2. Frequency domain noise attenuation and edge preservation:

[0189] (1) Dynamic attenuation coefficient calculation: according to the noise energy proportion (high frequency noise amplitude and total amplitude), calculate the attenuation coefficient:

[0190]

[0191] (2) Frequency domain mask superposition and phase reservation: the frequency domain data of the overexposed region is superimposed with a mask Perform:

[0192] Amplitude attenuation: .

[0193] Phase reservation: , wherein is the original frequency domain phase.

[0194] Inverse FFT transform: .

[0195] (3) Overexposed region backfill: the overexposed sub-matrix after denoising is backfilled to the original brightness matrix to generate a global frequency domain denoising matrix .

[0196] 3, Space-frequency collaborative fusion: the space domain filtering result is fused with the frequency domain denoising matrix according to the mask weight:

[0197]

[0198] Step 210, adjusting each frequency band component according to the global illumination intensity index and the local processing strategy to obtain an adjusted frequency band component;

[0199] In the embodiments of the present application, each frequency band component can be adjusted according to the global illumination intensity index and the local processing strategy to obtain an adjusted frequency band component.

[0200] In one example, step 210 can include the following sub-steps:

[0201] S101, adjusting the weight of each frequency band component according to the global illumination intensity index;

[0202] In specific implementation, if GLI<0.5, the weight of high frequency component is increased to 0.6 (default 0.4) to enhance the details; if GLI>0.8, the high frequency weight is reduced to 0.2 to suppress noise.

[0203] S102, optimizing the weight according to the global illumination intensity index and the local processing strategy to obtain an optimized weight;

[0204] After the initialization of the weight is completed, the weight can be optimized according to the global illumination intensity index and the local processing strategy to obtain an optimized weight.

[0205] In one example, step S102 can include the following sub-steps:

[0206] ​S1021, increase the weight of the low-frequency component in the low-light area, reduce the weight of the high-frequency component in the low-light area, and obtain a low-light area optimization weight;

[0207] S1022, reduce the weight of the high-frequency component in the overexposure area, and increase the weight of the low-frequency component in the overexposure area, to obtain an overexposure area optimization weight.

[0208] In a specific implementation, the optimization process is as follows:

[0209] 1. Secondary adjustment of the frequency band weight based on the local strategy:

[0210] (1) Weight optimization of the low-light area (low-light region):

[0211] Low-frequency component (LL) weight enhancement: For the region marked as low-light by the mask, the low-frequency LL weight of the corresponding wavelet decomposition block is increased by 0.2, and the formula is:

[0212]

[0213] The GLI dynamically corrects the weight adjustment amplitude to avoid excessive enhancement of noise in the low-light scene.

[0214] High-frequency component (LH / HL / HH) weight fine-tuning: To avoid amplification of dark noise, the high-frequency component weight is reduced by 0.1:

[0215]

[0216] Weight normalization:

[0217] (2) Weight optimization of the overexposure area (overexposure region):

[0218] High-frequency component weight suppression: For the region marked as overexposure by the mask, the high-frequency component weight of the corresponding wavelet decomposition block is reduced by 0.3, and the formula is:

[0219]

[0220] Low-frequency component weight fine-tuning: The low-frequency LL weight is increased by 0.1 to balance the brightness distribution of the overexposure area:

[0221]

[0222] Weight normalization:

[0223] (3) Weight maintenance of the normal area (normal region):

[0224] Maintain the initial weight without adjustment to reduce the calculation amount.​​​

[0225] 2. Regional dynamic allocation of wavelet band weights:

[0226] (1) Mask-based weight mapping:

[0227] For each wavelet decomposition block (such as LL, LH, etc.), according to the proportion of the original image region covered by the block , the weighted average adjustment coefficient is calculated:

[0228]

[0229] where is the attention weight of the pixels in the block (from the attention map ).

[0230] (2) Adaptive weight adjustment formula:

[0231] For each wavelet block, the final optimized weight is:

[0232] where is the initial weight of each wavelet band component; in low light areas, +0.2 (LL) / -0.1 (high frequency), in overexposed areas, -0.3 (high frequency) / +0.1 (LL); Based on local luminance variance calculation, the more uneven the illumination, the greater the adjustment.

[0233] S103, adjust the luminance of each band component according to the optimized weight to obtain the adjusted band component.

[0234] After obtaining the optimized weight of each band component, the luminance of each band component can be adjusted to obtain the adjusted luminance of each band component, thereby obtaining the adjusted band component.

[0235] Step 211, reconstruct the adjusted band component by wavelet inverse transform to obtain a reconstructed RGB image;

[0236] Step 212, convert the reconstructed RGB image into a target image in a predetermined format.

[0237] In the embodiments of the present application, the adjusted band components can be reconstructed into a reconstructed RGB image by wavelet inverse transform, and then gamma correction (gamma value is adjusted according to GLI, range 0.8~1.2) is performed on the reconstructed RGB image, and finally converted into an 8-bit unsigned integer (0~255) format target image, output to the cache queue.

[0238] Further, step 212 further includes:

[0239] determining whether the fluctuation of the global illumination intensity index of the target image of the continuous 5 frames exceeds 20%; if yes, returning to the step of collecting image data and converting the image data into an RGB image.

[0240] In a specific implementation, the target image obtained after the illumination optimization can be detected, and a processing log (containing a GLI index, parameter configuration, processing time consumption, etc.) is generated; if the GLI fluctuation of the continuous 5 frames exceeds 20%, the parameters are recalibrated to avoid the false triggering of the fixed threshold under different illumination scenes.

[0241] The application enables the system to automatically adjust the processing parameters according to the real-time illumination conditions through the dynamic parameter adaptive mechanism, without manual intervention, and improves the universality and robustness of the method, which is suitable for various complex industrial scenes.

[0242] Please refer to Figure 3 , Figure 3 A structural block diagram of an image illumination abnormality processing device provided by an embodiment of the application is provided.

[0243] An image illumination abnormality processing device is provided by an embodiment of the application, which comprises:

[0244] An image collection and conversion module 301 is configured to collect image data and convert the image data into an RGB image.

[0245] A single-channel brightness matrix construction module 302 is configured to acquire a brightness component from the RGB image and construct a single-channel brightness matrix by using the brightness component.

[0246] A global illumination intensity index calculation module 303 is configured to traverse the single-channel brightness matrix, mark an overexposure area and a low-illumination area, and calculate a global illumination intensity index.

[0247] A spatial domain interference signal distribution determination module 304 is configured to determine a spatial domain interference signal distribution according to the overexposure area.

[0248] A bilateral filtering parameter adjustment module 305 is configured to adjust a bilateral filtering parameter according to the global illumination intensity index.

[0249] A filtered brightness matrix generation module 306 is configured to adjust the single-channel brightness matrix based on the bilateral filtering parameter and generate a filtered brightness matrix.

[0250] An enhanced brightness matrix generation module 307 is configured to enhance the filtered brightness matrix by a Retinex algorithm based on the spatial domain interference signal distribution to obtain an enhanced brightness matrix.

[0251] A local processing strategy generation module 308 is configured to determine a key area according to the enhanced brightness matrix and generate a local processing strategy of the key area.

[0252] The frequency band component acquisition module 309 is configured to perform two-layer wavelet transform on the RGB image to obtain a plurality of frequency band components.

[0253] The frequency band component adjustment module 310 is configured to adjust the frequency band components according to the global illumination intensity index and the local processing strategy to obtain adjusted frequency band components.

[0254] The reconstruction module 311 is configured to reconstruct the adjusted frequency band components by inverse wavelet transform to obtain a reconstructed RGB image.

[0255] The target image generation module 312 is configured to convert the reconstructed RGB image into a target image in a preset format.

[0256] In the embodiment of the present application, the spatial domain interference signal distribution determination module 304 comprises:

[0257] The overexposure sub-matrix generation sub-module is configured to generate an overexposure sub-matrix of the overexposure region.

[0258] The frequency domain amplitude spectrum acquisition sub-module is configured to perform fast Fourier transform on the overexposure sub-matrix to obtain a frequency domain amplitude spectrum.

[0259] The distance calculation sub-module is configured to calculate the distance from each frequency domain coordinate in the frequency domain amplitude spectrum to the center point of the frequency domain amplitude spectrum.

[0260] The amplitude threshold value determination sub-module is configured to determine an amplitude threshold value according to all the frequency domain coordinates.

[0261] The noise frequency mask generation sub-module is configured to generate a noise frequency mask according to the distance, the frequency domain coordinate, the preset distance threshold value and the amplitude threshold value.

[0262] The filtered frequency domain signal generation sub-module is configured to calculate the product of the noise frequency mask and the frequency domain coordinate to obtain a filtered frequency domain signal.

[0263] The spatial domain signal generation sub-module is configured to perform inverse fast Fourier transform on the filtered frequency domain information to obtain a spatial domain signal.

[0264] The spatial domain interference signal distribution determination sub-module is configured to identify a non-zero region in the spatial domain signal to obtain a spatial domain interference signal distribution.

[0265] In the embodiment of the present application, the local processing strategy generation module 308 comprises:

[0266] The distribution probability calculation sub-module is configured to extract the brightness feature of the enhanced brightness matrix and calculate the distribution probability of the brightness feature.

[0267] The local processing strategy generation sub-module is configured to determine a key region according to the distribution probability and generate a local processing strategy for the key region.

[0268] In the embodiment of the present application, the frequency band component adjusting module 310 comprises:

[0269] The weight adjusting sub-module is configured to adjust the weight of each frequency band component according to the global illumination intensity index.

[0270] The optimization sub-module is configured to optimize the weight according to the global illumination intensity index and the local processing strategy to obtain an optimized weight.

[0271] The frequency band component adjusting sub-module is configured to adjust the brightness of each frequency band component according to the optimized weight to obtain an adjusted frequency band component.

[0272] In the embodiment of the present application, the bilateral filtering parameters comprise a spatial domain standard deviation and a color domain standard deviation; and the bilateral filtering parameter adjusting module 305 comprises:

[0273] The first adjusting sub-module is configured to increase the spatial domain standard deviation when the global illumination intensity index is less than a first preset threshold.

[0274] The second adjusting sub-module is configured to decrease the color domain standard deviation when the global illumination intensity index is greater than a second preset threshold.

[0275] In the embodiment of the present application, the frequency band component comprises a low frequency component and a high frequency component; and the optimization sub-module comprises:

[0276] The first optimization unit is configured to increase the weight of the low frequency component in the low illumination area and decrease the weight of the high frequency component in the low illumination area to obtain a low illumination area optimized weight.

[0277] The second optimization unit is configured to decrease the weight of the high frequency component in the overexposure area and increase the weight of the low frequency component in the overexposure area to obtain an overexposure area optimized weight.

[0278] In the embodiment of the present application, the method further comprises:

[0279] The judging module is configured to judge whether the fluctuation of the global illumination intensity index of the continuous 5 frames of target images exceeds 20%.

[0280] The returning module is configured to return to the step of collecting image data and converting the image data into an RGB image if the answer is yes.

[0281] The embodiment of the present application further provides an electronic device, which comprises a processor and a memory:

[0282] The memory is configured to store program code and transmit the program code to the processor.

[0283] The processor is configured to execute the image illumination abnormality processing method according to the instructions in the program code.

[0284] The embodiment of the present application further provides a computer readable storage medium for storing program codes, the program codes being used for executing the image light abnormality processing method of the embodiment of the present application.

[0285] Those skilled in the art can clearly understand the specific working process of the system, the device and the unit described above for the convenience and brevity of description, which can refer to the corresponding process in the foregoing method embodiments, and will not be described here.

[0286] Each of the embodiments in the specification is described in a progressive manner, and each embodiment focuses on the difference from other embodiments, and the same and similar parts between the embodiments can be referred to each other.

[0287] Those skilled in the art can understand that the embodiments of the embodiment of the present application can be provided as a method, a device or a computer program product. Therefore, the embodiments of the present application can adopt a completely hardware embodiment, a completely software embodiment or an embodiment combining software and hardware aspects. Moreover, the embodiments of the present application can adopt a computer program product in the form of being implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program codes.

[0288] The embodiments of the present application are described with reference to flowcharts and / or block diagrams according to the method, terminal device (system) and computer program product of the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be realized by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing terminal device to produce a machine, so that the instructions executed by the computer or other programmable data processing terminal device produce a device for realizing the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in one flow or multiple flows and / or blocks Figure 1 The device for realizing the functions specified in one block or multiple blocks.

[0289] These computer program instructions can also be stored in a computer readable storage medium which can guide the computer or other programmable data processing terminal device to work in a specific way, so that the instructions stored in the computer readable storage medium produce a product including instruction devices, which realize the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in one flow or multiple flows and / or blocks Figure 1 The device for realizing the functions specified in one block or multiple blocks.

[0290] These computer program instructions can also be loaded into a computer or other programmable data processing terminal device, so that a series of operational steps are performed on the computer or other programmable terminal device to generate a computer-implemented process, so that the instructions executed on the computer or other programmable terminal device provide a process for implementing the functions specified in the flowchart block(s) Figure 1 one or more flowcharts and / or blocks Figure 1 one or more flowcharts and / or blocks

[0291] Although the preferred embodiments of the embodiments of the present application have been described, those skilled in the art who have the basic inventive concept can make further changes and modifications to the embodiments. Therefore, the appended claims are intended to include the preferred embodiments and all changes and modifications falling within the scope of the embodiments of the present application.

[0292] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation portal for user to choose authorization or refusal.

[0293] Finally, it should be noted that in this document, relational terms such as first and second and the like can only be used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that these entities or operations exist in any such actual relationship or order. Moreover, the terms "comprising", "including" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed or inherent to such a process, method, article or terminal device. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or terminal device including the element.

[0294] The above-described and above-mentioned embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for processing image illumination anomalies, characterized in that, include: Acquire image data and convert the image data into an RGB image; The luminance components are obtained from the RGB image, and a single-channel luminance matrix is ​​constructed using the luminance components. Traverse the single-channel brightness matrix, mark overexposed and underexposed areas, and calculate the global illumination index; The spatial interference signal distribution is determined based on the overexposed area; Adjust the bilateral filter parameters according to the global illumination intensity index; The single-channel luminance matrix is ​​adjusted based on the bilateral filtering parameters to generate a filtered luminance matrix. Based on the spatial interference signal distribution, the filtered brightness matrix is ​​enhanced using the Retinex algorithm to obtain the enhanced brightness matrix; Based on the enhanced brightness matrix, key regions are determined, and local processing strategies for the key regions are generated. Perform a two-level wavelet transform on the RGB image to obtain multiple frequency band components; Adjust the frequency band components according to the global illumination intensity index and the local processing strategy to obtain the adjusted frequency band components; The adjusted frequency band components are reconstructed using inverse wavelet transform to obtain the reconstructed RGB image; The reconstructed RGB image is converted into a target image in a preset format.

2. The method according to claim 1, characterized in that, The step of locating the spatial interference signal distribution based on the overexposed area includes: Generate the overexposure submatrix of the overexposure region; Perform a fast Fourier transform on the overexposed submatrix to obtain the frequency domain amplitude spectrum; Calculate the distance from each frequency domain coordinate in the frequency domain amplitude spectrum to the center point of the frequency domain amplitude spectrum; Determine the amplitude threshold based on all frequency domain coordinates; A noise frequency mask is generated based on the distance, the frequency domain coordinates, the preset distance threshold, and the amplitude threshold. The filtered frequency domain signal is obtained by multiplying the noise frequency mask and the frequency domain coordinates. Perform an inverse fast Fourier transform on the filtered frequency domain information to obtain the spatial domain signal; The distribution of spatial interference signals is obtained by identifying non-zero regions in the spatial signal.

3. The method according to claim 1, characterized in that, The step of determining key regions based on the enhanced brightness matrix and generating local processing strategies for the key regions includes: Extract the brightness features of the enhanced brightness matrix and calculate the probability distribution of the brightness features; Based on the distribution probability, key regions are determined, and local processing strategies for the key regions are generated.

4. The method according to claim 1, characterized in that, The step of adjusting each frequency band component according to the global illumination intensity index and the local processing strategy to obtain the adjusted frequency band components includes: The weights of each frequency band component are adjusted according to the global illumination intensity index; The weights are optimized based on the global illumination intensity index and the local processing strategy to obtain optimized weights; The brightness of each frequency band component is adjusted according to the optimized weights to obtain the adjusted frequency band components.

5. The method according to claim 1, characterized in that, The bilateral filtering parameters include spatial domain standard deviation and color domain standard deviation; the step of adjusting the bilateral filtering parameters according to the global illumination index includes: When the global illumination intensity index is less than a first preset threshold, the spatial domain standard deviation is increased; When the global illumination intensity index is greater than the second preset threshold, the standard deviation of the color gamut is reduced.

6. The method according to claim 4, characterized in that, The frequency band components include low-frequency components and high-frequency components; The step of optimizing the weights based on the global illumination intensity index and the local processing strategy to obtain optimized weights includes: Increase the weight of the low-frequency component in the low-light region and decrease the weight of the high-frequency component in the low-light region to obtain the optimized weight of the low-light region. The weight of the high-frequency component in the overexposed region is reduced, and the weight of the low-frequency component in the overexposed region is increased to obtain the optimized weight of the overexposed region.

7. The method according to claim 1, characterized in that, After the step of converting the reconstructed RGB image into a target image of a preset format, the method further includes: Determine whether there are five consecutive frames of target images where the global illumination intensity index fluctuates by more than 20%. If so, return to the steps of acquiring image data and converting the image data into an RGB image.

8. An image illumination anomaly processing device, characterized in that, include: The image acquisition and conversion module is used to acquire image data and convert the image data into RGB images; A single-channel luminance matrix construction module is used to obtain luminance components from the RGB image and construct a single-channel luminance matrix using the luminance components. The global illumination index calculation module is used to traverse the single-channel brightness matrix, mark overexposed areas and underexposed areas, and calculate the global illumination index. A spatial interference signal distribution determination module is used to determine the spatial interference signal distribution based on the overexposed area; A bilateral filter parameter adjustment module is used to adjust the bilateral filter parameters according to the global illumination intensity index. The filtered luminance matrix generation module is used to adjust the single-channel luminance matrix based on the bilateral filtering parameters to generate the filtered luminance matrix. An enhanced brightness matrix generation module is used to enhance the filtered brightness matrix based on the spatial interference signal distribution using the Retinex algorithm to obtain an enhanced brightness matrix. A local processing strategy generation module is used to determine key regions based on the enhanced brightness matrix and generate local processing strategies for the key regions. The frequency band component acquisition module is used to perform a two-level wavelet transform on the RGB image to obtain multiple frequency band components; The frequency band component adjustment module is used to adjust each frequency band component according to the global illumination intensity index and the local processing strategy to obtain the adjusted frequency band component. The reconstruction module is used to reconstruct the adjusted frequency band components through inverse wavelet transform to obtain a reconstructed RGB image; The target image generation module is used to convert the reconstructed RGB image into a target image in a preset format.

9. An electronic device, characterized in that, The device includes a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is used to execute the image illumination anomaly processing method according to any one of claims 1-7 according to the instructions in the program code.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store program code for executing the image illumination anomaly processing method according to any one of claims 1-7.

Images