Image enhancement method, computer equipment and readable storage medium

Abstract

The invention discloses an image enhancement method, computer equipment and a readable storage medium, and belongs to the technical field of image signal processing. The method comprises the following steps: in response to the input of a YUV image, performing brightness enhancement on a first brightness image corresponding to a Y channel of the YUV image to obtain a second brightness image; performing global average pooling operation on the first brightness image and a first chromaticity image corresponding to a UV channel of the YUV image to obtain statistical characteristics; performing color cast correction on the first chromaticity diagram based on the statistical characteristics to obtain a second chromaticity diagram; and based on an inverse transformation formula from YUV to RGB, carrying out linear transformation and recombination calculation on the second brightness diagram and the second chromaticity diagram to obtain a nonlinear enhanced RGB image corresponding to the YUV image. According to the invention, color cast during image enhancement can be avoided, so that the classification precision of a downstream target detection network is improved.

Description

Technical Field

[0001] This application belongs to the field of image signal processing technology, specifically relating to an image enhancement method, a computer device, and a readable storage medium. Background Technology

[0002] In recent years, deep learning-based image signal processing (ISP) has been proposed, attempting to recover high-quality sRGB images directly from the dark and noisy sRGB images output by industrial cameras through end-to-end neural networks, in order to replace the traditional hardware ISP pipeline that relies on manual parameter tuning.

[0003] However, since the RGB channels simultaneously carry both luminance and chromaticity information, neural networks based on the deep learning ISP architecture often struggle to maintain a linear proportional relationship between the three channels when trying to maximize the brightness of dark areas. This results in non-physical color shifts in the enhanced image (such as the night sky appearing strangely green or purple), which seriously affects the classification accuracy of downstream object detection networks. Summary of the Invention

[0004] The purpose of this application is to provide an image enhancement method, computer device, and readable storage medium that can avoid color shift during image enhancement, thereby improving the classification accuracy of downstream target detection networks.

[0005] To solve the above-mentioned technical problems, this application is implemented as follows: In a first aspect, embodiments of this application provide an image enhancement method, the method comprising: In response to the input of a YUV image, the brightness of the first brightness map corresponding to the Y channel of the YUV image is enhanced to obtain a second brightness map; Global average pooling is performed on the first luminance map and the first chrominance map corresponding to the UV channels of the YUV image to obtain statistical features; Based on the statistical characteristics, the first chromaticity map is corrected for color shift to obtain the second chromaticity map; Based on the inverse YUV to RGB transformation formula, the second luminance image and the second chrominance image are linearly transformed and recombined to obtain the nonlinear enhanced RGB image corresponding to the YUV image.

[0006] In a second aspect, embodiments of this application provide a computer device including a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the method described in the first aspect.

[0007] Thirdly, embodiments of this application provide a computer-readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0008] Fourthly, embodiments of this application also provide a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in the first aspect.

[0009] In this embodiment, by using YUV images for luminance-color decoupling, luminance enhancement is applied only to the Y channel, and chrominance optimization is applied only to the U and V channels. This fundamentally avoids the color shift problem caused by channel ratio imbalance when enhancing all three RGB channels simultaneously, thereby preventing the decrease in classification accuracy of downstream object detection networks due to color shift, and ultimately improving the classification accuracy of downstream object detection networks. Attached Figure Description

[0010] Figure 1 This is one of the schematic flowcharts of an image enhancement method provided in some embodiments of this application; Figure 2 This is one of the schematic flowcharts of an image enhancement method provided in some embodiments of this application; Figure 3 These are structural diagrams of convolutional neural networks provided in some embodiments of this application; Figure 4 This is one of the schematic flowcharts of an image enhancement method provided in some embodiments of this application; Figure 5 This is one of the schematic flowcharts of an image enhancement method provided in some embodiments of this application; Figure 6 This is one of the schematic flowcharts of an image enhancement method provided in some embodiments of this application; Figure 7 These are comparison images of original low-light images, processed images, and normal-light images provided in some embodiments of this application; Figure 8 These are internal structural diagrams of a computer device provided in some embodiments of this application. Detailed Implementation

[0011] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0012] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0013] The embodiments of this application can be applied to a variety of application scenarios, such as intelligent fault diagnosis of rail transit vehicles.

[0014] Taking the intelligent fault diagnosis scenario under rail transit vehicles as an example, the core of this application's embodiments lies in providing an independent, lightweight, and efficient image enhancement preprocessing plugin (i.e., the LumaChromaISP module). This plugin can be seamlessly connected between various image acquisition devices and downstream advanced vision tasks (such as object detection, image segmentation, etc.). To make the objectives, technical solutions, and advantages of this application's embodiments clearer, the training, construction, deployment, and application process of this plugin will be described in detail below with reference to specific embodiments.

[0015] Example 1: Construction and offline training of a plug-and-play image enhancement plugin.

[0016] This section primarily aims to illustrate how to build and train this bright-color decoupled image enhancement plugin from scratch, enabling it to recover low-light conditions. In the data preparation and preprocessing stage, a paired image dataset for offline training is first constructed (such as using a publicly available Length of Day (LOD) dataset or a self-built low-light scene dataset).

[0017] The training data includes pairs of "low-light RAW images" and "normal-light RGB ground truth images" from the same scene. To adapt to the tensor operation interface of the neural network, the RAW images (such as Bayer arrays arranged in RGGB) are simply de-mosaiced and stacked, preprocessed into 4-channel tensors, and the data bit depth is normalized to the [0,1] interval.

[0018] During the plugin network initialization phase, according to Figure 5 The technical architecture shown is the LumaChromaISP architecture diagram. It sequentially instantiates a linear parameter predictor (5-layer convolution to extract global illumination), a zero-reference luminance enhancement stream (U-Net architecture with group normalization to prevent small batch statistical failures), a chromaticity optimization stream (including a multilayer perceptron (MLP) for calculating global mean and variance and an hourglass-shaped residual denoising convolutional network), and a spatial reconstruction and fusion module (including SE-Block and high-frequency extraction operators).

[0019] Specifically, for the chroma denoising residual network and high-frequency injection gating coefficients The "zero-initialization" strategy is enforced to ensure that the plugin is equivalent to a safe linear short wire when it is not trained, thus providing a good initial state for subsequent stable training.

[0020] During the supervised training and parameter optimization phase, the preprocessed 4-channel RAW tensor is input into the plugin. The plugin performs forward propagation and outputs a 3-channel enhanced RGB image. The combined loss function between the output image and the "normal lighting RGB ground truth image" is calculated: ,in, For the final loss, This is the L1 pixel-level reconstruction loss, used to constrain the consistency of overall color and brightness; Spatial gradient L1 loss is used to force the plugin to learn and preserve high-frequency physical textures of the image (such as edges and contours). A stochastic gradient descent (SGD) optimizer is used to update all learnable parameters within the plugin (such as the weights of the color correction matrix (CCM), the convolution kernel of Zero-DCE, the weights of SE-Block, etc.), with an initial learning rate set to 1e-4 and a cosine annealing strategy used for learning rate decay.

[0021] After training is complete, the network weights are fixed and exported as an independent pre-trained model file, thus completing the construction of this image enhancement plugin.

[0022] Example 2: Lightweight deployment and system-level application for under-vehicle fault diagnosis scenarios.

[0023] This embodiment mainly illustrates the plug-and-play deployment method of the image enhancement plugin in a real intelligent fault diagnosis system for rail transit vehicles, and its collaborative workflow with various downstream visual diagnostic tasks.

[0024] During the system integration and interface docking phase, the LumaChromaISP model trained in Example 1 is deployed as an independent pre-processing software plugin on the edge computing nodes (such as NVIDIA Jetson series, industrial control computers, etc.) of trackside monitoring equipment or automated undercarriage inspection robots. This plugin exposes a standardized input interface to upstream hardware (receiving real-time RAW format data streams of extreme low-light undercarriage data acquired by industrial linear / area array cameras) and a standardized output interface to downstream algorithms (outputting a 3-channel high-quality image tensor in standard sRGB color space, normalized to [0,1]. This decoupled interface design makes the plugin completely independent of subsequent diagnostic business logic.

[0025] During the real-time inference and under-vehicle image restoration stages, when the inspection equipment operates in a dimly lit environment with extremely uneven lighting and complex shadows under the vehicle, the weak RAW image stream acquired first enters the LumaChromaISP plugin in this embodiment. With extremely low latency (approximately a few milliseconds), the plugin sequentially and rapidly performs linear physical correction, luminance-chrominance orthogonal decoupling, Zero-DCE-based adaptive stretching of dark areas, chromatic affine denoising, and high-frequency texture reconstruction operations internally. Subsequently, the plugin outputs an enhanced RGB image to the system cache that is overall bright, reveals deep shadow details, is free of color artifacts, and has sharp edges.

[0026] In one exemplary embodiment, this application proposes an image enhancement method. The image enhancement method provided by this application will be described in detail below with reference to the accompanying drawings and specific embodiments and application scenarios.

[0027] Reference Figure 1 The method includes steps 102-108. Wherein: Step 102: In response to the input of the YUV image, the brightness of the first brightness map corresponding to the Y channel of the YUV image is enhanced to obtain the second brightness map.

[0028] In some embodiments, the pixel values ​​in the first brightness map can be directly subjected to linear transformations, such as addition or multiplication adjustments.

[0029] In some embodiments, enhancing the brightness of the first brightness map corresponding to the Y channel of the YUV image to obtain a second brightness map includes: Based on the first luminance map corresponding to the Y channel of the YUV image, the Zero-Reference Depth Curve Estimation (Zero-DCE) model is invoked to generate a set of pixel-level high-order curve parameter maps.

[0030] An iterative illumination enhancement model is constructed based on the aforementioned high-order curve parameter diagram.

[0031] Based on the iterative illumination enhancement model, a pixel-by-pixel nonlinear mapping is performed on the first luminance map to obtain the second luminance map.

[0032] In this embodiment, the Zero-DCE model uses a U-Net structure to capture multi-scale illumination distribution features and outputs a set of pixel-level high-order curve parameter maps. ,in, The preset total number of iterations (e.g.) =8, 9, etc.).

[0033] The iterative formula for the iterative illumination enhancement model is shown in Equation 1.

[0034] (1) in, Represents the spatial coordinates of a pixel; Indicates the current iteration number; Indicates the first The pixel brightness value after the next iteration, initial value This is the first brightness map input; The model predicts the first The parameter curve for each iteration has a value range limited to [-1, 1].

[0035] It should be noted that higher-order curves possess monotonicity and differentiability, enabling them to adaptively and significantly increase pixel values ​​in dark areas to the visible range while suppressing overexposure in highlight areas. Ultimately, the [number]th [curve] is taken. The output of the next iteration is used as the second brightness map, i.e. .

[0036] Step 104: Perform global average pooling on the first luminance map and the first chromaticity map corresponding to the UV channels of the YUV image to obtain statistical features.

[0037] It should be noted that the statistical features corresponding to the first luminance map are used to guide the correction of the first chromaticity map.

[0038] Step 106: Based on the statistical characteristics, perform color shift correction on the first chromaticity map to obtain the second chromaticity map.

[0039] In some embodiments, the statistical features include a first global mean and a first global standard deviation corresponding to the luminance map, and a second global mean and a second global standard deviation corresponding to the chrominance map; the step of correcting the color cast of the first chrominance map based on the statistical features to obtain a second chrominance map includes: The first global mean, the first global standard deviation, the second global mean, and the second global standard deviation are concatenated to obtain the global feature vector.

[0040] Based on the global feature vector, an MLP containing two fully connected layers is invoked to generate affine transformation parameters.

[0041] The first chromaticity map is subjected to a pixel-by-pixel linear transformation based on the affine transformation parameters to obtain the second chromaticity map.

[0042] In some embodiments, global feature vector It is a 4-dimensional vector, where, The first global mean, The first global standard deviation, The second global mean. This is the second standard deviation.

[0043] In some embodiments, the affine transformation parameters include a 2×2 scaling-rotation matrix. and a 2×1 translation vector .

[0044] In some embodiments, a pixel-by-pixel linear transformation is performed on the first chromaticity map to obtain a second chromaticity map, which is achieved by the following formula 2.

[0045] (2) in, It is the identity matrix. This is the first chromaticity diagram. This is the second chromaticity diagram.

[0046] This embodiment eliminates common global color shifts (such as bluish tint in dark areas) in low light by analyzing the brightness and color distribution patterns of the entire image. Furthermore, it constructs a very low-dimensional cascaded vector by extracting the global mean and standard deviation of the Y and UV channels. Global correction is performed by predicting the affine matrix using an MLP (Multi-Level Processing). In engineering experiments, this design completely solves the global color masking problem under extremely low light conditions with extremely low computational overhead.

[0047] Step 108: Based on the inverse YUV to RGB transformation formula, perform linear transformation and recombination calculations on the second luminance image and the second chrominance image to obtain the nonlinear enhanced RGB image corresponding to the YUV image.

[0048] This embodiment utilizes YUV images for luminance-color decoupling, ensuring that luminance enhancement applies only to the Y channel and chrominance optimization applies only to the U and V channels. This fundamentally avoids the color shift problem caused by channel imbalance when simultaneously enhancing all three RGB channels, thus preventing the decrease in classification accuracy of downstream object detection networks due to color shift, and ultimately improving the classification accuracy of downstream object detection networks.

[0049] Furthermore, such as Figure 2 As shown, before performing brightness enhancement on the first brightness map corresponding to the Y channel of the YUV image in response to the input of the YUV image to obtain the second brightness map, the following steps are included: In response to the input of a RAW image from the camera, the RAW image is downsampled.

[0050] Based on the downsampled RAW image, a parameter prediction model is invoked to generate a global illumination feature vector; wherein, the parameter prediction model includes 5 convolutional blocks, each of which includes a convolutional layer, a normalization layer, and a Gaussian Error Linear Unit (GELU) activation function.

[0051] The global illumination feature vector is decoded to obtain the dynamic color correction matrix increment and dynamic bias.

[0052] Based on the dynamic color correction matrix increment, the dynamic bias, the preset white balance parameters, and the preset basic color correction matrix, the RAW image is subjected to a pixel-by-pixel linear transformation to obtain the corresponding linear RGB image; wherein, the linear RGB image is used to convert to obtain the YUV image.

[0053] In some embodiments, such as Figure 3 As shown, the parameter prediction model is a lightweight convolutional neural network used to extract global illumination features of the image. The global illumination feature vector can be used to restore the physical color reference of the RAW image and correct dark current noise.

[0054] In some embodiments, the global illumination feature vector is a 12-dimensional feature vector, and the dynamic color correction matrix increment... Used for fine-tuning the overall color bias, i.e., restoring the physical color reference; dynamic bias. Used to correct black level drift, i.e., to correct dark current noise.

[0055] In some embodiments, the RAW image is subjected to a pixel-by-pixel linear transformation based on the dynamic color correction matrix increment, the dynamic bias, the preset white balance parameters, and the preset basic color correction matrix to obtain the corresponding linear RGB image, which can be achieved by the following formula 3.

[0056] (3) in, This represents the output linear RGB image, with values ​​normalized to a range of 1000. At this point, the image color is accurate but the brightness is extremely low; This represents the preset base color correction matrix, which has a dimension of 3×3 and is usually determined by the characteristics of the camera sensor. This represents the dynamic matrix offset (i.e., the dynamic color correction matrix increment), with a dimension of 3×3, used to adapt to the special lighting of the current scene (such as streetlights, moonlight). This represents the original RAW image input, typically in Bayer format (RGGB arrangement), which is then de-mosaiced and stacked into an RGB tensor. This represents the white balance gain vector (i.e., the preset white balance parameters), with a dimension of 1×3, corresponding to the gain coefficients of the R, G, and B channels respectively; This represents the dynamic bias, with a dimension of 3×1.

[0057] It should be noted that traditional techniques typically use sRGB images (such as JPEG / PNG format) processed by the camera's hardware ISP as input. However, sRGB images have already undergone gamma compression, non-linear color mapping, and other operations, completely destroying the original physical linear relationship of illumination and severely truncating the dynamic range (usually 8-bit). Performing low-light enhancement on this incomplete non-linear data makes it difficult to achieve true physical color reproduction.

[0058] RAW images are Bayer format images of the underside of vehicles captured by industrial cameras in extremely low light conditions, without any non-linear compression, and have a very high bit depth (typically 10-bit to 14-bit).

[0059] In summary, through the above operations, the corresponding linear RGB image obtained from the RAW image has accurate colors but lower brightness. Compared to the sRGB image, this ensures that the color reference and physical linearity of the image are strictly preserved before subsequent processing. Specifically, after converting the linear RGB image to a YUV image, using the YUV image as input for subsequent image enhancement preserves the photon distribution information and pure optical linear response of the scene; that is, it skips the non-linear compression of traditional ISP, avoids the destruction of the original physical illumination linearity in the RAW image, and thus enables better color reproduction.

[0060] In some embodiments, the conversion formula for converting a linear RGB image to a YUV image can follow the ITU-R BT.601 standard, and the conversion formula is as follows: Formula 4.

[0061] , , (4) in, , , These represent the red, green, and blue channel components of a linear RGB image, respectively. Represents the luminance component, carrying information about the image's structure, texture, and illumination intensity; its value range is... Subsequent brightness stretching operations only apply to this channel; , These represent the chromaticity components, carrying the hue and saturation information of the image. Due to the chromaticity components... , Depend on , , It is obtained through a linear transformation with positive and negative weights, therefore its values ​​extend beyond 0 into the positive and negative intervals, so the range of values ​​is approximately... Subsequent noise reduction and color correction operations only apply to this channel.

[0062] It is understandable that by decoupling light and color, even if the Y value is changed nonlinearly, the U and V ratio of the pixel can remain unchanged, thus ensuring that the hue does not shift at the mathematical level.

[0063] Further, the inverse YUV-to-RGB transformation formula is used to perform linear transformation and recombination calculations on the second luminance image and the second chrominance image to obtain the nonlinear enhanced RGB image corresponding to the YUV image, including: The second chromaticity map is downsampled by a factor of 2 to obtain the third chromaticity map.

[0064] The residual convolution model is invoked to remove color noise from the third chromaticity map, resulting in the fourth chromaticity map.

[0065] The fourth chromaticity image is upsampled by 2 times to obtain the fifth chromaticity image.

[0066] like Figure 4 As shown, based on the inverse YUV to RGB transformation formula, the second luminance image and the fifth chrominance image are linearly transformed and recombined to obtain the nonlinear enhanced RGB image corresponding to the YUV image.

[0067] In some embodiments, the residual convolutional model includes: a pooling layer, a downsampling layer, an activation layer, a linear layer, an upsampling layer, and a feature concatenation layer.

[0068] In some embodiments, the inverse transformation formula can be derived from the above formula 4, which will not be repeated in this embodiment.

[0069] This embodiment downsamples the data by 2x before feeding it into the residual convolution model to remove color noise. Leveraging the human eye's insensitivity to color space resolution, it reduces the computational cost of the color branch convolution by 75%, significantly saving computing power and making it suitable for deployment on edge devices. Furthermore, the residual convolution model allows the model to function as a safe, short, linear connection during the initial training phase, avoiding gradient explosion or mode collapse caused by the stacking of complex nonlinear modules, thus ensuring stable convergence of multi-task joint optimization.

[0070] Further, after performing linear transformation and recombination calculations on the second luminance image and the second chrominance image based on the inverse YUV-to-RGB transformation formula to obtain the nonlinear enhanced RGB image corresponding to the YUV image, the process includes: Based on the nonlinear enhanced RGB image and the linear RGB image, the channel attention module SE-Block is invoked to generate a weighted fused image.

[0071] The linear RGB image is subjected to mean filtering to obtain high-frequency residuals.

[0072] Based on learnable gating coefficients, the high-frequency residuals are injected into the weighted fused image to obtain an enhanced image.

[0073] It should be noted that, in order to recover the minute texture details that may be lost due to denoising operations (including correcting dark current noise and removing speckle noise), this embodiment further introduces attention fusion and high-frequency injection mechanisms.

[0074] In some embodiments, the recalled object may be a non-linear enhanced RGB image obtained by removing color noise, or a non-linear enhanced RGB image obtained without removing color noise.

[0075] SE-Block receives non-linear enhanced RGB images and linear RGB images as input and automatically learns the fusion weights. The two are then weighted and fused using the following formula 5.

[0076] (5) in, The image after weighted fusion. To fuse the weight map, the values ​​are set to [0,1], so that the model relies on enhanced features in extremely dark regions, while retaining the original linear features in high signal-to-noise ratio regions. The nonlinear enhanced RGB image obtained by removing color noise.

[0077] In some embodiments, the high-frequency residual is the edge and texture information.

[0078] Through a learnable gating coefficient High-frequency residuals can be forcibly injected into the weighted fused image, and this process can be achieved using the following formula 6.

[0079] (6) in, This represents the final output enhanced image, i.e., the enhanced image; This indicates a mean filtering operation; Represents the hyperbolic tangent activation function; This represents a numerical truncation (limiting) function.

[0080] It should be noted that Formula 6 is similar to an "unsharpening mask", which can significantly enhance the edge gradient of an object, making the enhanced image have high definition, high contrast and rich details, and can be directly used by subsequent object detection algorithms.

[0081] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0082] For ease of understanding, a specific embodiment will be used as an example: It should be noted that with the widespread application of deep learning technology in the field of computer vision, object detection algorithms based on convolutional neural networks (such as YOLO and DETR series) have achieved significant results in normal lighting scenarios. Especially in intelligent fault diagnosis scenarios under trains in the rail transit field (such as EMU undercarriage inspection robots and trackside TFDS image monitoring systems), the environment faced by image sensors is particularly extreme: the space under the train is narrow, with almost no natural light; the mechanical structure (such as bogies, motors, and brake discs) is intricate, making it easy to generate large areas of heavy shadows and uneven highlights when artificial lighting is used; in addition, the undercarriage components are often covered with oil and mud. In such extremely low-light and interference-prone scenarios, the images directly output by traditional industrial cameras are often pitch black and densely noisy, which seriously restricts the ability of the backend object detection network to automatically diagnose undercarriage faults (such as loose bolts, rusted metal parts, cracks in critical structures, and foreign object intrusion).

[0083] In recent years, deep learning-based image signal processing (ISP) has been proposed, attempting to recover high-quality sRGB images directly from image data through end-to-end neural networks, in order to replace the traditional hardware ISP pipeline that relies on manual parameter tuning.

[0084] Existing technologies still have many shortcomings in practical applications, mainly reflected in: ① It ignores the irreversible loss of physical linear information in the RAW domain: Most existing enhancement algorithms directly use sRGB images (such as JPEG / PNG) processed by the camera's hardware ISP as input. However, sRGB images have already undergone gamma compression, non-linear color mapping, and other operations, completely destroying the original physical linear relationship of illumination, and severely truncating the dynamic range (usually 8-bit). Performing low-light enhancement on this incomplete non-linear data is destined to fail to achieve true physical color restoration.

[0085] ② Severe color shift and artifacts: Most existing deep learning ISP architectures perform non-linear enhancement directly within the RGB color space. Since the three RGB channels simultaneously carry luminance and chromaticity information, the neural network often struggles to maintain a linear proportional relationship between the three channels while striving to enhance the brightness of dark areas. This results in non-physical color shifts in the enhanced image (such as the night sky appearing strangely green or purple), severely affecting the classification accuracy of downstream object detection networks.

[0086] ③ The contradiction between brightness enhancement and noise amplification: The noise distribution of low-light images is usually related to signal strength (Poisson-Gaussian distribution) and is mainly concentrated in the dark areas. Existing enhancement algorithms often use a uniform processing logic for the entire image. While stretching the brightness of dark areas, they inevitably amplify the underlying sensor noise, resulting in "noise explosion" in the image and masking the edge features of objects.

[0087] ④ Inefficient allocation of computational resources: The human visual system and deep learning models are far more sensitive to the spatial resolution of brightness than to chrominance. Existing methods typically perform convolution processing on the RGB three channels with equal computational power, which not only incurs high computational costs but also easily preserves chrominance noise that is not sensitive to the human eye, resulting in wasted computational power and poor performance.

[0088] ⑤ Training instability caused by multi-module coupling: In order to solve the problems of brightness, color and noise at the same time, existing research often attempts to directly and forcibly splice together multiple nonlinear processing modules (such as curve estimation, deep convolutional denoising, feature fusion, etc.). However, the superposition of multiple strong nonlinearities can easily lead to gradient explosion, color collapse or pattern collapse in the early stage of training, making it extremely difficult for end-to-end networks with multi-task joint optimization to converge, and lacking engineering feasibility.

[0089] Based on this, such as Figure 5 As shown, the image enhancement method follows the flow of data, and goes through four stages in sequence: linear physical correction based on the RAW domain, linear-preserving orthogonal decoupling of light and color spaces, dual-stream nonlinear enhancement based on statistical guidance and zero initialization, and spatial reconstruction and texture injection.

[0090] Phase 1: Linear Physical Correction This stage maps the raw Bayer RAW data into a linear RGB image with accurate colors but lower brightness. It skips the traditional camera hardware ISP process, directly receiving Bayer RAW images of the underside of vehicles in extremely low light, without any non-linear compression, captured by an industrial camera as input. This is denoted as... Compared to a truncated 8-bit sRGB image, It features extremely high bit depth (typically 10-bit to 14-bit), preserving the photon distribution information of the scene and a purely linear optical response to the greatest extent. Its structure diagram is shown below. Figure 2 shown. Specifically, 1. Parameter Prediction Network: To restore the physical color reference of the image and correct dark current noise, the network will... The image is downsampled and then input into a lightweight parameter prediction network (Matrix Predictor). This lightweight convolutional neural network takes the downsampled RAW image as input. The network contains five convolutional blocks, each consisting of a convolutional layer, an instance normalization layer, and a GELU activation function, used to extract global illumination features of the image. Its network structure is as follows. Figure 3 As shown.

[0091] 2. Dynamic Matrix Generation: The parameter prediction network outputs a 12-dimensional feature vector, which is decoded into two sets of parameters: ① Dynamic color correction matrix increment. It is used to fine-tune the overall color tendency; ② Dynamic bias amount It is used to correct black level drift.

[0092] 3. Linear transformation: Combined with preset white balance Parameters and basic color correction matrix For the original RAW image Perform pixel-level linear transformation, as shown in Formula 3.

[0093] Phase Two: Decoupling of Light Color Space Existing luminance-color decoupling techniques mostly employ the CIELab color space, but the nonlinear transformation of this space severely disrupts the physical linear reference established in the first stage. To overcome this technical bias, this embodiment introduces a YUV color space conversion module based on purely linear matrix operations. Convert from RGB space to YUV space. The conversion formula is shown in Formula 4.

[0094] Phase 3: Dual-stream Nonlinear Enhancement The two-stream nonlinear enhancement comprises two independent and parallel processing branches. Wherein: Branch A: Luminosity Enhancement Stream In the brightness enhancement stream, the Y component of a single channel is fed into a zero-reference depth curve estimation network. This network employs a U-Net structure to capture multi-scale illumination distribution features and outputs a set of pixel-level high-order curve parameter maps. This embodiment sets... =8. Using these parameters, an iterative illumination enhancement curve equation is constructed to perform a pixel-by-pixel nonlinear mapping of the Y component. The iterative formula is shown in Formula 1.

[0095] Branch B: Chroma Optimization Stream In the chroma optimization stream, the core logic is to use luminance information to guide chroma correction. The specific implementation consists of two steps: global statistical correction and local denoising. First, global affine correction is performed: the input single-channel luminance map is processed... (i.e., the first luminance map) and the two-channel chrominance map Global average pooling is performed on the (i.e., the first chroma image) to calculate their statistical characteristics. Specifically, the first global mean of the Y channel is calculated for each channel. First global standard deviation and the second global mean of the UV channel. Compared with the second standard deviation .

[0096] Subsequently, these four statistics are concatenated into a 4-dimensional global feature vector. This feature vector The input is fed into a multilayer perceptron containing two fully connected layers to predict the affine transformation parameters used to correct color cast: a 2×2 scaling-rotation matrix. and a 2×1 translation vector Using this affine transformation parameter to modify the original... Each pixel of the image (i.e., the first chromaticity image) undergoes a linear transformation, as shown in Formula 2.

[0097] After the correction is completed, the system proceeds to the downsampling and denoising stage: taking advantage of the fact that the human eye is not sensitive to color space resolution, the system will... The image (i.e., the second chroma map) is downsampled by a factor of 2, and then noise is removed using a lightweight residual convolutional network (i.e., a residual convolutional model). Finally, the denoised feature map is upsampled back to its original size to obtain a clean chroma map. (i.e., the fifth chroma map). Lightweight residual convolutional network structures include... Figure 6 As shown.

[0098] Phase Four: Spatial Reorganization and Integration Finally, the system processes the above dual-stream data. (Second brightness diagram) and Convergence. First, using the inverse YUV to RGB transformation formula, the enhanced luminance map is... and the denoised chroma map Linear transformation and recombination calculations generate a non-linear enhanced RGB image with suitable brightness and pure colors. The YUV-RGB module structure is as follows: Figure 4 As shown.

[0099] To recover subtle texture details that may be lost during denoising, attention fusion and high-frequency injection mechanisms were further introduced. The channel attention module (SE-Block) receives... And the original linear image output from the first stage As input, automatically learn fusion weights The two are then weighted and fused using Formula 5.

[0100] As a final refinement process, the original linear image is extracted using mean filtering. The high-frequency residuals (i.e., edge and texture information) are processed through a learnable gating coefficient. This is then forcibly injected into the fused image, as shown in Formula 6. This step is similar to an "unsharpening mask," significantly enhancing the edge gradients of objects and ultimately outputting an enhanced image with high definition, high contrast, and rich detail. This is directly used by subsequent object detection algorithms.

[0101] like Figure 7 As shown, "Low Light" refers to the image captured by the camera in a low-light scene; "After Low Light Processing" refers to the enhanced image obtained in this embodiment. "Normal lighting" refers to images captured by a camera under normal lighting conditions.

[0102] In summary, this embodiment addresses the aforementioned shortcomings and achieves the following beneficial effects: (1) Overcome technical bias and achieve full-link "physical-sensing" linear fidelity from the RAW domain to the sensing layer: To address the dual shortcomings of existing methods that suffer from physical information loss due to direct processing of truncated sRGB images and the destruction of original lighting patterns by using strong nonlinear mapping in Lab space, this paper innovatively proposes a combination of "first-stage physical linear matrix correction based on high-bit-depth RAW domain + second-stage pure linear orthogonal decoupling of YUV". This design not only directly utilizes the lossless wide dynamic range of RAW data, but also, while enhancing image brightness, strictly locks the hue and saturation benchmarks established by real physical lighting in a purely linear manner, fundamentally eliminating the strange color shifts in extreme low-light enhancement.

[0103] (2) It achieves the best balance between noise suppression and detail preservation: A dual-stream differential processing strategy is employed. In the chroma branch, downsampling convolution is used to efficiently remove color noise that is sensitive to the human eye, ensuring image purity. In the luminance branch, Zero-DCE preserves structural information. In the fusion stage, a high-frequency injection module is used to forcibly recover edge textures in the linear image. This "divide and conquer" strategy results in a final image that is both clean and smooth, while retaining key edge features required for object detection (such as lane lines and bolt outlines). Experiments show that it significantly improves the mAP (mApplied Per Target) score.

[0104] (3) Extremely high training stability and engineering feasibility: ① Zero-initialization residual design: The denoising network, fine-tuning network, and high-frequency injection module all adopt a "zero-initialization" strategy. This means that at the moment training begins, the entire network is equivalent to a simple linear mapping. This design avoids the gradient explosion or mode collapse problems common in complex nonlinear networks, enabling the model to converge quickly in the first few epochs.

[0105] ② Lightweight Deployment: It fully utilizes the low-frequency characteristics of the chroma channel, reducing the convolution computation of the chroma branch by 75% (through a 2x downsampling). Simultaneously, the matrix predictor processes downsampled thumbnails, and Zero-DCE is based on pixel-level curves rather than heavy convolutions. Compared to traditional U-Net-like augmented networks, the overall architecture has fewer parameters, making it more suitable for real-time operation on edge computing devices (such as Jetson NX and automotive chips).

[0106] (4) Deeply empower the intelligent fault diagnosis system under rail transit vehicles to improve the reliability of inspections under extreme environments: This application's embodiments demonstrate strong scene adaptability to the complex lighting and physical environment under vehicles. In under-vehicle fault diagnosis, because this application's embodiments achieve bright-color decoupling, it avoids distortion into a gray-black background or a blue-green artifact when brightening in low light, significantly reducing the missed detection rate of rust faults. At the same time, the high-frequency texture injection mechanism of this application's embodiments, while smoothing out the color noise of oil stains and mud, accurately preserves the edge contours and micro-crack features of metal parts, enabling downstream detection networks to easily capture fatal hidden dangers such as loose bolts (misaligned anti-loosening marks) or structural fractures. The plug-and-play nature of this application's embodiments means that existing under-vehicle inspection systems do not need to reconstruct large detection algorithms; only this plugin needs to be installed beforehand to achieve accurate fault diagnosis around the clock.

[0107] (5) Solved the color cast problem in low-light enhancement: The embodiments of this application innovatively introduce a brightness-color decoupling architecture, which forces the nonlinear brightness enhancement operation (Zero-DCE) to act only on the Y channel. According to the principle of color space, changing the Y value only affects the intensity of light, and never changes the hue and saturation determined by U and V. This mechanism effectively isolates brightness restoration from color distortion, and greatly suppresses non-physical color cast phenomena such as "local artifacts" that are very easy to occur in low-light scenes.

[0108] In one exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 8As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements an image enhancement method. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0109] Those skilled in the art will understand that Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0110] In one embodiment, a computer-readable storage medium is provided, on which a program or instructions are stored, which, when executed by a processor, implement the steps in the above-described method embodiments.

[0111] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0112] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0113] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0114] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. An image enhancement method, characterized in that, The image enhancement method includes: In response to the input of a YUV image, the brightness of the first brightness map corresponding to the Y channel of the YUV image is enhanced to obtain a second brightness map; Global average pooling is performed on the first luminance map and the first chrominance map corresponding to the UV channels of the YUV image to obtain statistical features; Based on the statistical characteristics, the first chromaticity map is corrected for color shift to obtain the second chromaticity map; Based on the inverse YUV to RGB transformation formula, the second luminance image and the second chrominance image are linearly transformed and recombined to obtain the nonlinear enhanced RGB image corresponding to the YUV image.

2. The image enhancement method according to claim 1, characterized in that, The step of enhancing the brightness of the first brightness map corresponding to the Y channel of the YUV image to obtain the second brightness map includes: Based on the first luminance map corresponding to the Y channel of the YUV image, the Zero-DCE model for zero-reference depth curve estimation is invoked to generate a set of pixel-level high-order curve parameter maps. An iterative illumination enhancement model is constructed based on the aforementioned high-order curve parameter diagram; Based on the iterative illumination enhancement model, a pixel-by-pixel nonlinear mapping is performed on the first luminance map to obtain the second luminance map.

3. The image enhancement method according to claim 1, characterized in that, The statistical features include the first global mean and the first global standard deviation corresponding to the luminance map, and the second global mean and the second global standard deviation corresponding to the chrominance map; The step of correcting the color cast of the first chromaticity image based on the statistical characteristics to obtain the second chromaticity image includes: The first global mean, the first global standard deviation, the second global mean, and the second global standard deviation are concatenated to obtain a global feature vector; Based on the global feature vector, a multilayer perceptron (MLP) containing two fully connected layers is invoked to generate affine transformation parameters. The first chromaticity map is subjected to a pixel-by-pixel linear transformation based on the affine transformation parameters to obtain the second chromaticity map.

4. The image enhancement method according to claim 3, characterized in that, The affine transformation parameters include a 2×2 scaling rotation matrix and a 2×1 translation vector.

5. The image enhancement method according to any one of claims 1-4, characterized in that, Before performing brightness enhancement on the first brightness map corresponding to the Y channel of the YUV image in response to the input of the YUV image to obtain the second brightness map, the following steps are included: In response to the input of a RAW image from the camera, the RAW image is downsampled; Based on the downsampled RAW image, a parameter prediction model is invoked to generate a global illumination feature vector; wherein, the parameter prediction model includes 5 convolutional blocks, each of which includes a convolutional layer, a normalization layer and a Gaussian error linear unit (GELU) activation function; The global illumination feature vector is decoded to obtain the dynamic color correction matrix increment and dynamic bias. Based on the dynamic color correction matrix increment, the dynamic bias, the preset white balance parameters, and the preset basic color correction matrix, the RAW image is subjected to a pixel-by-pixel linear transformation to obtain the corresponding linear RGB image; wherein, the linear RGB image is used to convert to obtain the YUV image.

6. The image enhancement method according to claim 5, characterized in that, The method based on the inverse YUV to RGB transformation formula, after performing linear transformation and recombination calculations on the second luminance image and the second chrominance image to obtain the nonlinear enhanced RGB image corresponding to the YUV image, includes: Based on the nonlinear enhanced RGB image and the linear RGB image, the channel attention module SE-Block is invoked to generate a weighted fused image; The linear RGB image is subjected to mean filtering to obtain high-frequency residuals; Based on learnable gating coefficients, the high-frequency residuals are injected into the weighted fused image to obtain an enhanced image.

7. The image enhancement method according to any one of claims 1-4, characterized in that, The inverse YUV-to-RGB transformation formula performs linear transformation and recombination calculations on the second luminance image and the second chrominance image to obtain a nonlinear enhanced RGB image corresponding to the YUV image, including: The second chromaticity map is downsampled by a factor of 2 to obtain the third chromaticity map; The residual convolution model is called to remove the color noise in the third chromaticity map, resulting in the fourth chromaticity map; The fourth chromaticity image is upsampled by a factor of 2 to obtain the fifth chromaticity image; Based on the inverse YUV to RGB transformation formula, the second luminance image and the fifth chrominance image are linearly transformed and recombined to obtain the nonlinear enhanced RGB image corresponding to the YUV image.

8. The image enhancement method according to claim 7, characterized in that, The residual convolutional model includes: pooling layer, downsampling layer, activation layer, linear layer, upsampling layer, and feature concatenation layer.

9. A computer device, characterized in that, It includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the image enhancement method as described in any one of claims 1-8.

10. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the image enhancement method as described in any one of claims 1-8.

Images