RAW image processing method for dark light target detection

Abstract

The invention discloses a RAW image processing method for dark light target detection, and relates to the technical field of dark light scene target detection, and the method comprises the steps: obtaining a RAW data set; constructing a linear mapping process component and a nonlinear transformation process component; performing regularization cooperative training on the linear mapping process component and the nonlinear transformation process component to obtain an image processing model; and performing image processing on the RAW data set according to the image processing model. According to the invention, the technical problem of poor target recognition effect caused by difficulty in fully utilizing rich information of an original image in a low-illumination related scene in the prior art is solved, and the technical effects of improving the accuracy of target recognition in the low-illumination scene and the adaptability of the model to different environments and reducing the difficulty of actual deployment are achieved.

Description

Technical Field

[0001] This invention relates to the field of target detection technology in low-light scenes, and particularly to a RAW image processing method for target detection in low-light scenes. Background Technology

[0002] Low-light scenes are prevalent in practical applications such as intelligent parking management. The results of image-based target detection directly impact the effectiveness of intelligent scene analysis. RAW images, containing rich physical information, are crucial data support for low-light target detection. Existing technologies either enhance RGB images using algorithms like generative adversarial networks, but suffer from poor recovery in low light and are prone to introducing noise; or when processing RAW images, either merging and quantization lead to information loss, or complex trainable ISP modules are used, making deployment difficult. These methods fail to fully utilize the advantages of RAW images, resulting in insufficient accuracy in low-light target detection and failing to meet the needs of precise scene analysis and intelligent management. Summary of the Invention

[0003] This application solves the technical problem that existing technologies struggle to fully utilize the rich information in original images in low-light scenarios, resulting in poor target recognition performance.

[0004] To address the aforementioned technical problems, this application proposes a RAW image processing method for low-light target detection. The method includes: acquiring a RAW dataset; constructing a linear mapping process component and a nonlinear transformation process component; performing regularized co-training on the linear mapping process component and the nonlinear transformation process component to obtain an image processing model; and performing image processing on the RAW dataset based on the image processing model.

[0005] This application proposes one or more technical solutions, which have at least the following technical effects:

[0006] This application acquires raw RAW image sets through video surveillance equipment and combines them with RGB images under different lighting conditions through inverse processing and noise simulation injection to generate a synthetic RAW dataset, thus constructing a complete RAW dataset. Then, it constructs a linear mapping component containing linear transformation channels and an attention mechanism, and a nonlinear transformation component containing non-convex polynomial basis functions. Through collaborative training combining a regularized loss function and an object detection loss function, an image processing model is obtained. This model then performs dynamic linear mapping and pixel-level nonlinear transformation on the RAW images sequentially. This fully leverages the rich physical information of the RAW images, improves the accuracy of object detection in low-light scenes, enhances the model's adaptability to different lighting conditions, and makes low-light object detection results more accurate and reliable. This achieves the technical effect of improving the accuracy of object recognition in low-light scenes and the model's adaptability to different environments, while reducing the difficulty of actual deployment. Attached Figure Description

[0007] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0008] Figure 1 This is a schematic flowchart of a RAW image processing method for low-light target detection provided in an embodiment of this application.

[0009] Figure 2 This is a schematic diagram of the process for obtaining an image processing model in a RAW image processing method for low-light target detection provided in an embodiment of this application. Detailed Implementation

[0010] This application provides a RAW image processing method for low-light target detection, which solves the technical problem that existing technologies cannot fully utilize the rich information of the original image in low-light related scenes, resulting in poor target recognition performance.

[0011] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0012] It should be noted that any variation of the terms "comprising" and "having" is intended to cover non-exclusive inclusion, for example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or modules that are not explicitly listed or that are inherent to such processes, methods, products, or devices.

[0013] like Figure 1 As shown, a RAW image processing method for low-light target detection includes:

[0014] Obtain the RAW dataset.

[0015] In this embodiment, RAW data is raw data collected by the camera sensor without image signal processing (ISP). It is usually 12-bit or 14-bit and retains physical information such as scene radiation and noise characteristics, as well as camera metadata. It contains richer details and color information than RGB images.

[0016] Specifically, the method for obtaining the RAW dataset is as follows: on the one hand, the original RAW image set is acquired through video surveillance equipment; on the other hand, RGB image sets under different lighting conditions are first acquired, and the RGB image sets are subjected to inverse data processing and injected with simulated noise to obtain a synthetic RAW dataset. Finally, the original RAW image set and the synthetic RAW dataset are combined to form a RAW dataset for low-light target detection.

[0017] Construct linear mapping process components and nonlinear transformation process components.

[0018] Optionally, the linear mapping process component consists of a linear transformation channel, a local attention mechanism, and a global attention mechanism. The linear transformation channel further includes a white balance channel, a merging operation channel, and a color space transformation channel, which work together to achieve basic linear processing and dynamic adaptation of RAW images. The nonlinear transformation process component uses non-convex polynomial basis functions as its core technology carrier to provide support for subsequent nonlinear enhancement of images.

[0019] The linear mapping process component and the nonlinear transformation process component are subjected to regularized co-training to obtain an image processing model.

[0020] In one embodiment of this application, a regularized loss function is first constructed, and then combined with the object detection loss function to generate an overall loss function. Finally, using the overall loss function as the optimization criterion, collaborative optimization training is performed on the linear mapping process component and the nonlinear transformation process component to generate an image processing model suitable for low-light object detection.

[0021] The RAW dataset is processed according to the image processing model.

[0022] Specifically, the RAW dataset is traversed to extract a single first RAW image. A dynamic linear mapping operation is then performed on this first RAW image using a linear mapping process component to obtain a first preliminary corrected image. Finally, a nonlinear transformation process component is used to perform a nonlinear transformation on the first preliminary corrected image. This results in a first enhanced image suitable for low-light target detection.

[0023] Furthermore, the method provided in this application embodiment includes:

[0024] The process involves acquiring a raw RAW image set using video surveillance equipment; acquiring an RGB image set under different lighting conditions; performing inverse data processing and noise simulation injection on the RGB image set to obtain a synthesized RAW dataset; and generating the RAW dataset based on the raw RAW image set and the synthesized RAW dataset.

[0025] Specifically, when acquiring the raw RAW image set, high-position video surveillance cameras or industrial-grade high-definition cameras used in roadside parking management scenarios can be selected. The camera is fixed on a stable tripod, and its height and angle are adjusted to ensure the shooting range covers the target parking scene, and that multiple images remain pixel-aligned. Through the camera's parameter settings interface, the exposure time is adjusted between 10ms and 100ms to simulate different lighting conditions, from low light to normal dim light. The unprocessed RAW format images output by the camera are directly captured and stored. Batch acquisition forms the raw RAW image set.

[0026] Next, the same camera equipment and shooting scenes were used as when acquiring the original RAW image set to avoid additional interference caused by equipment differences or scene changes. Shooting was conducted at different times, such as early morning, evening, and night, and under different weather conditions, such as rain, fog, and overcast skies. The camera was set to automatic exposure mode, directly outputting RGB format images that conform to human visual perception. Through multiple shots, it was ensured that the collected RGB images covered various lighting gradients, including low light, dim light, and normal lighting, forming a sufficiently large RGB image set.

[0027] Then, during the inverse data processing of the RGB image set, a reverse ISP workflow is employed. First, the gamma correction of the RGB images is removed using the inverse gamma function, with a gamma value of 2.2 chosen as commonly used in the industry, restoring the linear brightness information of the image. Next, based on the light source attributes of the image, the gain parameters of the RGB three channels are adjusted to reverse the white balance processing. Finally, through matrix inverse operations, the sRGB color space of the RGB image is converted back to the linear color space corresponding to the RAW image, completing the inverse data processing. In the noise simulation injection stage, Gaussian noise and Poisson noise are superimposed using the NumPy library to generate noise data with intensities between 0.01 and 0.05. This noise is then superimposed pixel-by-pixel onto the inverse-processed image to simulate the noise characteristics of RAW images in low-light environments, ultimately forming a RAW synthetic dataset.

[0028] Finally, the original RAW image set was integrated with the synthesized RAW dataset. All image files were read using the OpenCV image processing library, and the image resolution was uniformly adjusted to the standard size of 1920×1080. Blurry, damaged, or pixel-abnormal images were removed. Images were named according to the rule of "lighting condition-image type-serial number" to ensure clear data management. The final result was a complete RAW dataset for low-light target detection.

[0029] By combining real-world acquisition with synthetic supplementation, the method ensures data validity by directly capturing real RAW images, while expanding the sample size by generating synthetic data through reverse ISP and noise overlay. Ultimately, this provides a sufficient number of RAW datasets with diverse lighting scenarios that meet the characteristics of low light for model training.

[0030] Furthermore, the method provided in this application embodiment includes:

[0031] The linear mapping process components include a linear transformation channel, a local attention mechanism, and a global attention mechanism. The linear transformation channel includes a white balance channel, a merging operation channel, and a color space transformation channel.

[0032] Optionally, when constructing the linear mapping process component, the linear processing flow of a traditional camera ISP is used as the basis, with the input four-channel RAW image as the object of linear transformation. Specifically, the input RAW image can be represented at the pixel level as a combination of four channels, corresponding to the red channel r, the first green channel gr, the blue channel b, and the second green channel gb, forming raw data of size H x W x 4. The goal of the linear mapping process is to complete white balance correction, channel merging, and color space transformation using directly implementable matrix operations, thereby obtaining a three-channel correction result and providing a more stable input representation for subsequent object detection.

[0033] In the white balance channel, the channel gain output by the camera's automatic white balance algorithm is used as a parameter to construct a 4x4 diagonal gain matrix W, where the diagonal elements correspond to the gain coefficients of the four RAW channels. White balance correction is achieved by performing matrix multiplication on the four-dimensional RAW vector at each pixel location; that is, multiplying the pixel's r, gr, b, and gb values ​​by the corresponding gain coefficients, and outputting the white-balanced four-channel RAW data. This step only involves channel-by-channel multiplication and matrix multiplication operations, resulting in low computational overhead and consistency with the white balance implementation in existing ISPs, facilitating the reuse of existing camera parameters and processing chains in engineering projects.

[0034] In the merging operation, to convert a four-channel RAW to a three-channel representation, a green channel merging strategy is employed, averaging the two green channels. Specifically, a 3x4 sparse merging matrix B is constructed. The R in the output three channels is directly taken from the input r channel, the B in the output three channels is directly taken from the input b channel, and the G in the output three channels is obtained by averaging gr and gb with equal weights. In implementation, multiplying the four-dimensional vector at each pixel location by matrix B yields the three-dimensional vectors R, G, and B, resulting in a three-channel linearly merged output of size H x W x 3. This step involves only linear weighting and summation, making it easy to implement in parallel on a CPU or GPU.

[0035] In the color space transformation channel, a camera color correction workflow is employed, using a 3x3 color correction matrix C to map the three-channel data to the target perceived color space. Matrix C can be determined by camera calibration or the camera's built-in color profile. In actual calculations, matrix multiplication is performed on the 3D vector of each pixel to obtain the transformed three-channel result. Thus, the white balance matrix W, the merging matrix B, and the color correction matrix C can be concatenated sequentially into a unified linear mapping matrix P, allowing linear correction of the input RAW image to be completed in a single linear mapping. Specifically, the four-channel data after white balance are first merged to obtain three-channel data, and then a color space transformation is performed to obtain the corrected RGB output. From an engineering implementation perspective, this can be accomplished using matrix chaining multiplication and pixel-by-pixel matrix multiplication.

[0036] Since the aforementioned white balance and color correction parameters are typically kept fixed for the same camera, their adaptability to different scenes and noise variations under low-light conditions is insufficient. Therefore, a dynamic mapping process is further introduced into the linear mapping component to enhance robustness. Specifically, a lightweight convolutional neural network is used to extract multi-scale features from the input RAW image. The convolutional network can be composed of multiple layers of 3x3 convolutions, nonlinear activation, and downsampling operations. The output contains two types of features: one is a local feature sequence aligned with the spatial resolution of the original image. One type is used to characterize the brightness and color deviations in different regions; the other type is the global feature sequence obtained through global average pooling or stride downsampling. This is used to characterize the overall lighting and color trends of the entire image. Subsequently, based on a local attention mechanism... Local enhancement is performed to generate pixel-level dynamic parameters, which are used to adaptively adjust the linear mapping matrix at different pixel locations with fine granularity; simultaneously, a global attention mechanism is used to... Global enhancement is performed to generate image-level dynamic parameters for consistent linear mapping compensation across the entire image. In implementation, pixel-level dynamic parameters are organized as pixel-by-pixel increments isomorphic to the mapping matrix, while image-level dynamic parameters are organized as global increments to the mapping matrix. These two are then summed and fused with the static linear mapping matrix P to obtain the dynamic linear mapping matrix. This dynamic linear mapping is then applied to the input RAW image, outputting the dynamically corrected RGB result.

[0037] By unifying white balance, channel merging, and color space transformation into a linear matrix mapping and introducing dynamic parameters generated by local and global attention, adaptive correction of linear mapping to different regions and overall illumination changes in low-light scenes is achieved, thereby improving the stability and robustness of subsequent object detection inputs.

[0038] Furthermore, the method provided in this application embodiment includes:

[0039] The nonlinear transformation process component includes nonconvex polynomial basis functions.

[0040] Specifically, when constructing the nonlinear transformation process component, the corrected image output from the aforementioned steps is used as the representation. As input, The image data consists of three channels with dimensions H x W x 3, where H represents the image height, W represents the image width, and 3 represents the number of color channels. The purpose of this non-linear component is to perform pixel-level brightness and contrast remapping on the linearly corrected image, stretching and enhancing dark areas while compressing bright areas to suppress overexposure, thereby forming an enhanced image U suitable for subsequent object detection.

[0041] First, a convolutional neural network is used to predict the coefficients of each pixel. Specifically, a pixel-level coefficient prediction network is constructed for the input... Perform convolution to obtain a set of coefficients corresponding to each pixel position. The coefficient can be expressed as Where k represents the basis function index, i represents the vertical coordinate position of the pixel, and j represents the horizontal coordinate position of the pixel. This convolutional neural network can be implemented using multiple 3x3 convolutional layers connected in series. Non-linear activations can be added between the convolutional layers, and the final output is a coefficient tensor with a spatial resolution of [missing information]. The constant is H multiplied by W, and the number of channels is n plus 1, where n represents the highest degree of the polynomial basis functions used (excluding the constant term), and n plus 1 represents the total number of coefficients from k=0 to k=n. Therefore, for any pixel position i,j, a set of coefficient vectors can be obtained. arrive These coefficients are used to control the shape and intensity of the nonlinear mapping curve at that pixel.

[0042] Subsequently, a set of nonconvex polynomial basis functions is constructed to form a nonlinear transformation. Specifically, a set of polynomial basis functions is set. arrive ,in These are constant basis functions with a value of 1, used to provide bias terms; for basis functions with k greater than or equal to 1 , defined as a k-th order polynomial function of pixel value x, where x represents the value of the pixel to be transformed, which can take the following values: The normalized intensity value of a channel at pixel position (i,j) is obtained by normalization, which linearly maps 8-bit or floating-point pixel intensity to the 0-1 range. To meet the requirements of low-light enhancement, the polynomial basis function adopts a non-convex mapping form, making the curve slope steeper in the low-brightness range and thinner in the high-brightness range, thereby achieving dark area stretching and bright area compression. To facilitate implementation and ensure the consistency of mapping endpoints, each... All pass through the coordinates 0,0 and 1,1, meaning that for any k greater than or equal to 1, the following condition is met. equal to 0 and The value is set to 1 to ensure that the output remains 0 when the input is 0 and 1 when the input is 1, thus avoiding overall brightness drift. In actual implementation, a preset k-th order polynomial can be used, and its coefficients can be constrained to satisfy the above endpoint conditions and have a non-convex shape.

[0043] During pixel-level nonlinear transformation, for each pixel position (i,j), the input intensity value of that pixel is taken. Substituting it into each basis function yields arrive Then, a linear combination with the corresponding coefficients is performed to obtain the output value of the pixel. The specific expression is:

[0044] Where F represents the nonlinear transformation function, This represents the input pixel at pixel position (i,j). This represents the combination coefficient of the pixel on the k-th basis function. Let represent the k-th polynomial basis function, and n represent the highest order. By performing the above calculation point-by-point at each pixel location, the nonlinearly transformed image result U can be obtained, where U is related to... They have the same dimensions H x W x 3, and U is used as an augmentation input for subsequent object detection networks.

[0045] Furthermore, to represent the ability of this nonlinear transformation to approximate complex imaging responses, the above transformation can be regarded as an approximation of a certain complex function H, where H(x) can be expressed as... , This represents a small error term that tends to a higher order as the order increases, used to characterize the residual of the polynomial expansion approximation. Since F uses pixel-adaptive coefficients... The weighted set of polynomial basis functions can form different mapping curves at different pixel locations, thus enabling more fine-grained fitting of the brightness enhancement requirements of different areas in low-light scenes and suppression of overexposure in bright areas.

[0046] By predicting coefficients pixel by pixel using a convolutional neural network and forming a nonlinear mapping with a non-convex polynomial basis function that satisfies endpoint consistency, the system achieves stretching enhancement of dark regions and compression suppression of bright regions, thereby improving the image discriminability and robustness for target detection in low-light scenes.

[0047] Furthermore, such as Figure 2 As shown, the method provided in this application embodiment includes:

[0048] A regularized loss function is constructed; the regularized loss function is combined with the target detection loss function to generate an overall loss function; the linear mapping process component and the nonlinear transformation process component are collaboratively optimized and trained according to the overall loss function to generate the image processing model.

[0049] Specifically, low-light target detection models are typically trained solely based on the target detection loss function. This involves feeding the input image into the detection network and updating the model parameters using classification and location regression losses. However, due to the unstable image quality in low-light scenarios, relying solely on the detection loss can lead to a lack of constraints on the linear correction and nonlinear enhancement processes in the front-end image processing module. This results in inconsistent outputs between the two stages and unstable training. Therefore, a regularization process is introduced during model training to constrain the relationship between the linear mapping and nonlinear transformation components, promoting their collaborative optimization.

[0050] The input to the regularization process primarily comes from the outputs of both the linear and nonlinear components. The image after linear transformation is represented as follows: = Where I represents the input low-light RAW image, This represents the parameters of the linear mapping matrix obtained in the current training iteration of the linear mapping process component. This represents the image result after linear mapping of the RAW image I. Subsequently, Input the nonlinear transformation process component to obtain the enhanced image after nonlinear transformation. Where F represents a nonlinear mapping function composed of nonconvex polynomial basis functions, and U represents the enhanced image obtained by nonlinear transformation, which serves as the input to the object detection network.

[0051] During the regularization process, an RGB image under normal lighting conditions corresponding to the low-light RAW image is introduced as a reference, defined as follows: ,in, This represents the RGB image of a low-light RAW image under normal lighting conditions. H and W represent the height and width of the image, respectively, and 3 represents the three RGB channels. It can be obtained by collecting data under normal lighting conditions in the same scene, or directly provided by existing normal lighting RGB samples in the dataset.

[0052] Next, obtain and The best-matching linear mapping relationship is found by solving the optimal linear mapping matrix using the least squares method. Its mathematical expression is: ,in, Let represent the optimal linear mapping matrix obtained by the least squares method. Let I represent the linear mapping matrix variable to be optimized, and let I represent the low-light RAW image. This represents a reference RGB image under normal lighting conditions. The square of the L2 norm is used to measure... and The overall squared error between the two images is minimized. By minimizing this error, the linear mapping result is made as close as possible to the reference image under normal lighting in an overall sense.

[0053] In obtaining Then, construct the regularization loss function. Used to constrain the current parameters of the linear mapping process component. With reference optimal linear mapping The consistency between them is expressed as: ,in, This represents the regularization loss function. The reference linear mapping matrix is ​​obtained by the least squares method. The parameters of the linear mapping matrix are the linear mapping matrix parameters during the current iteration of the linear mapping process component. This is achieved by minimizing... ,make Gradually towards during training Alignment enhances compatibility between linear and nonlinear components.

[0054] Finally, the regularization loss function is combined with the object detection loss function to obtain the overall loss function of the model. This step will be explained in detail later. During model training, backpropagation is performed on L to update the parameters of both the linear mapping process component and the nonlinear transformation process component, ultimately resulting in the image processing model.

[0055] By introducing a regularized loss based on least-squares reference linear mapping and cosine distance constraints and jointly optimizing it with the target detection loss, collaborative training of the linear mapping process component and the nonlinear transformation process component is achieved, thereby improving the stability and robustness of target detection in low-light scenes.

[0056] Furthermore, the method provided in this application embodiment includes:

[0057] The overall loss function is: Where L represents the overall loss function, This represents the target detection loss function, which includes a classification loss function and a location regression loss function. The classification loss function is the cross-entropy loss function, and the location regression loss function is the IoU loss function. This represents the regularization loss function. This represents hyperparameters.

[0058] Specifically, in obtaining the regularization loss function Then, it is combined with the object detection loss function to construct an overall loss function, and collaborative training is completed based on the overall loss function, specifically as follows: Where L is the overall loss function, Let the target detection loss function be... The regularization loss function is... This is a hyperparameter used to adjust the weight of the regularization loss in the overall loss. The network employs two loss functions: a classification loss function (using cross-entropy) and a location regression loss function (using IoU). During training, the augmented image U is input into the object detection network to obtain classification predictions and bounding box predictions, which are then calculated. At the same time, by I and Seek And by and calculate Then, backpropagation is performed based on L, so that the linear mapping process component and the nonlinear transformation process component update the parameters simultaneously under the same objective, until convergence is obtained to obtain the image processing model.

[0059] Furthermore, the method provided in this application embodiment includes:

[0060] The first RAW image is extracted by traversing the RAW dataset; the first RAW image is dynamically linearly mapped according to the linear mapping process component to obtain a first preliminary corrected image; the first preliminary corrected image is nonlinearly transformed according to the nonlinear transformation process component to obtain a first enhanced image.

[0061] In one embodiment, target detection in low-light scenes typically involves directly processing and inferring from the RGB images output by the camera, or preprocessing with a fixed-parameter image enhancement algorithm before inference. Because exposure, noise, and color shift vary significantly across different scenes in low light, fixed-parameter processing methods are difficult to adapt to different lighting conditions, easily leading to insufficient detail in dark areas or overexposure in bright areas, thus affecting the stability and accuracy of subsequent target detection. Therefore, after training the image processing model, a two-stage processing approach—linear correction and nonlinear enhancement—is performed frame-by-frame on the RAW dataset during the inference or data processing stage. This allows the physical information in the RAW data to be fully utilized, outputting enhanced images more suitable for target detection.

[0062] In practice, the RAW dataset is first traversed to extract the first RAW image. The RAW dataset consists of multiple RAW images, each retaining the original data and noise characteristics acquired by the camera sensor. These images are typically multi-bit deep and contain richer detail. Traversal can be implemented using a sequential reading method, that is, reading the RAW files one by one according to the dataset index and decoding them into matrix-like pixel data. After reading, the currently extracted RAW image is defined as the first RAW image, used for subsequent two-stage processing.

[0063] Next, the first RAW image is transformed through a linear transformation channel to obtain a basic linear mapping. Then, pixel-level and image-level dynamic parameters are generated by combining local and global features extracted by the convolutional neural network. The linear transformation matrix is ​​adaptively adjusted to perform dynamic linear mapping on the first RAW image and obtain the first preliminary corrected image. This step will be explained in detail later.

[0064] Finally, corresponding pixel coefficients are assigned to each pixel in the first preliminary correction image, and the pixel coefficients are linearly combined with non-convex polynomial basis functions to form pixel-level nonlinear transformation characteristics, thereby performing pixel-by-pixel nonlinear mapping on the first preliminary correction image and obtaining the first enhanced image. This step will be explained in detail later.

[0065] Furthermore, the method provided in this application embodiment includes:

[0066] The first RAW image is input into a linear transformation channel to obtain a first linear transformation matrix; multi-scale feature extraction is performed on the first RAW image using a convolutional neural network to obtain a first local feature sequence and a first global feature sequence; local enhancement is performed on the first local feature sequence according to a local attention mechanism to obtain pixel-level dynamic parameters; global enhancement is performed on the first global feature sequence according to a global attention mechanism to obtain image-level dynamic parameters; the pixel-level dynamic parameters and the image-level dynamic parameters are combined with the first linear transformation matrix to obtain a first dynamic linear mapping matrix; the first RAW image is transformed using the first dynamic linear mapping matrix to obtain the first preliminary corrected image.

[0067] Optionally, the first RAW image is input into a linear transformation channel to obtain a first linear transformation matrix. This channel performs linear operations in the order of white balance, merging, and color space transformation. The white balance step uses a 4×4 diagonal gain matrix W, whose diagonal elements are the gain coefficients of each channel, to compensate for the color cast of the light source. The merging operation uses a 3×4 sparse matrix B to average the two green channels of the RAW image, achieving a four-channel to three-channel conversion. The color space transformation uses a 3×3 color correction matrix C to map the merged three-channel data to the perceptual color space. Calculations are performed row-by-row and column-by-column according to the matrix multiplication rule of "C×B×W," ultimately obtaining a first linear transformation matrix with a dimension of 3×4. This matrix integrates the core logic of the three types of linear processing.

[0068] A lightweight convolutional neural network is used to extract multi-scale features from the first RAW image to obtain the first local feature sequence and the first global feature sequence. The network first uses a 3×3 convolutional kernel with a stride of 1 and SAME padding to perform convolution operations on the four channels of the RAW image. After processing by the ReLU activation function, the output is a first local feature sequence with dimensions H×W×C, which is consistent with the input image size and is used to characterize the detailed information of the pixel neighborhood. Then, the local features are downsampled through a convolutional layer or a max pooling layer with a stride of 2 to expand the receptive field. After multiple downsampling, the first global feature sequence with dimensions H / 16×W / 16×C is obtained, which is used to capture the overall illumination and scene trend of the image.

[0069] The first local feature sequence is locally enhanced using a local attention mechanism to obtain pixel-level dynamic parameters. The first local feature sequence is used as the query vector Q, and the first linear transformation matrix P is used as the key vector K and value vector V. The similarity score between Q and K is calculated through matrix multiplication, and the local attention weights are obtained after normalization using the softmax function. These weights are then multiplied with V to generate H×W×3×4 dimension pixel-level dynamic parameters, the same size as the first RAW image. Each element corresponds to a fine-grained adjustment coefficient for a pixel in the three-channel linear transformation.

[0070] Global attention is used to enhance the first global feature sequence, resulting in image-level dynamic parameters. The first linear transformation matrix P is used as the query vector Q, and the first global feature sequence is used as the key vector K and value vector V. K and V are first adapted to match the channel dimensions of Q. A global similarity score is calculated via matrix multiplication, and after normalization, the global attention weight is obtained. Multiplying this weight by V outputs 3D image-level dynamic parameters, which correspond to the overall linear transformation compensation coefficients for the three channels, adapting to the illumination characteristics of the entire image.

[0071] The pixel-level dynamic parameters, image-level dynamic parameters, and the first linear transformation matrix are combined to obtain the first dynamic linear mapping matrix. First, the 3D image-level dynamic parameters are expanded to H×W×3×4 dimensions through a broadcast operation to match the dimensions of the pixel-level dynamic parameters. Then, the first linear transformation matrix P is copied along the spatial dimensions, similarly expanding to H×W×3×4 dimensions. Finally, the three expanded matrices are added element-wise to obtain the first dynamic linear mapping matrix of H×W×3×4 dimensions, achieving personalized linear transformation adaptation for each pixel.

[0072] The first RAW image is transformed using a first dynamic linear mapping matrix to obtain a first preliminary corrected image. The H×W×4 pixel data of the first RAW image is split according to pixel position, with each pixel forming a 4-dimensional vector. This vector is then multiplied with the corresponding 3×4 dynamic linear mapping matrix to obtain 3-dimensional corrected pixel values. After pixel-by-pixel processing of the entire image, the first preliminary corrected image (H×W×3 dimension) is output. This image preserves the physical information of the RAW image and, through dynamic parameters, adapts local details to global illumination.

[0073] Linear transformation is achieved through matrix operations. Multi-scale features are extracted by convolutional networks and attention mechanisms are used to generate dynamic parameters. Then, a personalized dynamic mapping matrix is ​​constructed by matrix superposition. Ultimately, the effect of dynamically adapting to the local and global characteristics of low-light scenes, accurately correcting RAW images, and fully preserving their physical information is achieved.

[0074] Furthermore, the method provided in this application embodiment includes:

[0075] Obtain the pixel coefficients corresponding to each pixel in the first preliminary corrected image; linearly combine the pixel coefficients with non-convex polynomial basis functions to obtain the nonlinear transformation characteristics of each pixel; perform pixel-level transformation on the first preliminary corrected image according to the nonlinear transformation characteristics of each pixel to obtain the first enhanced image.

[0076] Optionally, a lightweight convolutional neural network is used to obtain the pixel coefficients corresponding to each pixel in the first preliminary correction image. This network consists of three cascaded 3×3 convolutional layers, with ReLU activation functions inserted between the layers to introduce non-linear expressive power. The last convolutional layer has n+1 output channels, where n is the highest order of the non-convex polynomial basis function. The first preliminary correction image of size H×W×3 is input into this network. The network extracts local features of each pixel through sliding convolution kernels. After activation function processing, it outputs a coefficient tensor of size H×W×(n+1). The spatial resolution of this tensor is perfectly aligned with the first preliminary correction image. The n+1 channel values ​​corresponding to each pixel position are the combination coefficients of that pixel on different orders of polynomial basis functions, ensuring that each pixel obtains personalized non-linear transformation parameters.

[0077] Next, before linearly combining each pixel coefficient with the non-convex polynomial basis function, the pixel intensity values ​​of the first preliminary corrected image are normalized. Using a commonly used linear mapping method, 8-bit pixel values ​​(range 0-255) or floating-point pixel values ​​are uniformly mapped to the 0-1 interval, ensuring computational stability while meeting the input requirements of the polynomial basis function. The non-convex polynomial basis function is designed according to existing technology, wherein... The basis functions are fixed at a constant of 1 to provide bias terms, k≥1. Employing a k-th order polynomial, this type of polynomial naturally satisfies the requirement of passing through coordinate points (0,0) and (1,1), and exhibits a non-convex characteristic of a large slope in low-brightness regions and a small slope in high-brightness regions within the 0-1 interval. During the linear combination process, the normalized intensity value for each pixel position (i,j) is... Substituting the basis functions of each order in sequence, we obtain to Then, respectively, with the corresponding coefficients to Multiply the results and sum them to obtain the nonlinear transformation output value of the pixel, thus completing the nonlinear characteristic calculation of a single pixel.

[0078] Finally, when performing pixel-level transformation on the first preliminary corrected image based on the nonlinear transformation characteristics of each pixel, a row-first traversal method is used to process all pixels in the image. During the traversal, the above linear combination calculation is performed on the R, G, and B color channels of each pixel to avoid interference between channels. For example, for the normalized intensity value of the R channel of pixel (i,j) , obtained through linear combination This value is used as the intensity value of the R channel of the corresponding pixel in the enhanced image; similarly, the enhanced values ​​of the G and B channels are calculated. After all pixels are processed, a first enhanced image with a size of H×W×3 is generated. This image retains the spatial structure and color distribution of the first preliminary corrected image, while achieving shadow detail stretching and highlight overexposure suppression through nonlinear transformation.

[0079] By accurately predicting pixel-level coefficients using a lightweight convolutional network, combined with a personalized linear combination of non-convex polynomial basis functions, and then through pixel-by-pixel and channel-by-channel transformation operations, the effect of enhancing dark areas and compressing bright areas is finally achieved, significantly improving the discernibility of dark images and providing high-quality input images for subsequent object detection.

[0080] In summary, the RAW image processing method for low-light target detection provided in this application has the following technical effects:

[0081] This application generates a synthetic dataset by acquiring original RAW images and combining inverse RGB image processing with noise injection. It constructs a linear mapping component with an attention mechanism and a nonlinear transformation component with non-convex polynomial basis functions. After regularized co-training, a model is obtained, which performs dynamic linear mapping and pixel-level nonlinear transformation on RAW images. This fully explores the physical information of RAW images, improves the accuracy of target detection in low light, and makes the detection results more accurate and reliable. It achieves the technical effect of improving the accuracy of target recognition in low light scenes and the model's adaptability to different environments, while reducing the difficulty of actual deployment.

[0082] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0083] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of this application and its equivalents, this application also intends to include such modifications and variations.

Claims

1. A RAW image processing method for low-light target detection, characterized in that, The method includes: Obtain the RAW dataset; Construct components for linear mapping and nonlinear transformation processes; The linear mapping process component and the nonlinear transformation process component are regularly trained together to obtain an image processing model; The RAW dataset is processed according to the image processing model.

2. The RAW image processing method for low-light target detection as described in claim 1, characterized in that, Obtain the RAW dataset, including: Obtain the raw RAW image set through video surveillance equipment; Obtain RGB image sets under different lighting conditions; The RGB image set is subjected to inverse data processing and noise simulation injection to obtain a RAW synthetic dataset; The RAW dataset is generated based on the original RAW image set and the RAW composite dataset.

3. The RAW image processing method for low-light target detection as described in claim 1, characterized in that, The linear mapping process components include a linear transformation channel, a local attention mechanism, and a global attention mechanism. The linear transformation channel includes a white balance channel, a merging operation channel, and a color space transformation channel.

4. The RAW image processing method for low-light target detection as described in claim 1, characterized in that, The nonlinear transformation process component includes nonconvex polynomial basis functions.

5. The RAW image processing method for low-light target detection as described in claim 1, characterized in that, Regularized co-training is performed on the linear mapping process component and the nonlinear transformation process component to obtain an image processing model, including: Construct a regularized loss function; The regularization loss function is combined with the target detection loss function to generate the overall loss function; The image processing model is generated by co-optimizing and training the linear mapping process component and the nonlinear transformation process component based on the overall loss function.

6. The RAW image processing method for low-light target detection as described in claim 5, characterized in that, The overall loss function is: ； Where L represents the overall loss function, This represents the target detection loss function, which includes a classification loss function and a location regression loss function. The classification loss function is the cross-entropy loss function, and the location regression loss function is the IoU loss function. This represents the regularization loss function. This represents hyperparameters.

7. The RAW image processing method for low-light target detection as described in claim 1, characterized in that, Image processing of the RAW dataset is performed according to the image processing model, including: Iterate through the RAW dataset to extract the first RAW image; The first RAW image is dynamically linearly mapped according to the linear mapping process component to obtain a first preliminary corrected image; The first preliminary corrected image is nonlinearly transformed according to the nonlinear transformation process component to obtain the first enhanced image.

8. The RAW image processing method for low-light target detection as described in claim 7, characterized in that, The first RAW image is dynamically linearly mapped according to the linear mapping process component to obtain a first preliminary corrected image, including: Input the first RAW image into the linear transformation channel to obtain the first linear transformation matrix; Multi-scale feature extraction is performed on the first RAW image using a convolutional neural network to obtain a first local feature sequence and a first global feature sequence. The first local feature sequence is locally enhanced based on the local attention mechanism to obtain pixel-level dynamic parameters. The first global feature sequence is globally enhanced using a global attention mechanism to obtain image-level dynamic parameters. The pixel-level dynamic parameters and the image-level dynamic parameters are combined with the first linear transformation matrix to obtain the first dynamic linear mapping matrix; The first RAW image is transformed using the first dynamic linear mapping matrix to obtain the first preliminary corrected image.

9. The RAW image processing method for low-light target detection as described in claim 7, characterized in that, The first preliminary corrected image is nonlinearly transformed according to the nonlinear transformation process component to obtain a first enhanced image, including: Obtain the pixel coefficients corresponding to each pixel in the first preliminary corrected image; The nonlinear transformation characteristics of each pixel are obtained by linearly combining each pixel coefficient with a non-convex polynomial basis function. The first preliminary corrected image is transformed at the pixel level according to the nonlinear transformation characteristics of each pixel to obtain the first enhanced image.

Images