Deep network-based park intelligent behavior recognition image low-light real-time enhancement method

Abstract

The invention relates to a park intelligent behavior recognition image low-light real-time enhancement method based on a deep network, and aims to solve the problems of insufficient image brightness, detail missing and the like in a low-light environment and improve the behavior recognition accuracy of a park monitoring system. The method comprises the following steps: preprocessing and compressing an input video stream, and converting the video stream into a frame image; decomposing the frame image into an illumination component and a reflectivity component; through deep network stage-by-stage processing, transform domain enhancement is performed on the illumination component, feature optimization is performed on the reflectivity component, and the spatial resolution is recovered through down-sampling, feature transformation and up-sampling; and finally, combining the optimized illumination and reflectivity components to generate an enhanced RGB image. According to the method, the calculation complexity is reduced by adopting the depth separable convolution, the real-time performance and the light weight are achieved, the method is suitable for edge equipment, and the image quality and the monitoring behavior recognition performance are remarkably improved under the low-light condition.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a low-light real-time image enhancement method applied in a park monitoring system. Background Art

[0002] With the gradual promotion of smart parks and the increasing demand for security monitoring, the monitoring systems of campuses, factories and other parks have become an important means to ensure personnel safety, prevent emergencies and optimize management. Intelligent monitoring systems can not only provide video monitoring, but also use advanced algorithms such as behavior recognition to achieve real-time analysis and recognition of human behavior in the monitoring screen, such as posture analysis, action judgment and abnormal event detection. This technology is particularly important in scenarios such as campus security and factory production safety monitoring. It can detect falls, intrusions into restricted areas, dangerous operations and other situations in a timely manner, and trigger warnings to protect personnel and property safety. However, the monitoring system of the park is often affected by night or low-light environments. Under such conditions, the clarity and detail of the image are significantly reduced, which affects the accuracy of behavior recognition.

[0003] Images taken in low-light environments often have problems such as insufficient brightness, low contrast, blurred details, and increased noise. These defects will directly affect the image processing effect of the intelligent monitoring system and reduce the reliability and accuracy of behavior recognition. In order to improve the effect of the monitoring system in low-light environments, low-light enhancement technology has emerged. Low-light enhancement technology aims to improve the brightness, contrast and details of images in low-light environments, making the images clearer and easier to identify, thereby reducing the misjudgment rate of behavior recognition and ensuring the effectiveness of the monitoring system at night. Current low-light enhancement technologies can be roughly divided into model-driven methods and data-driven methods.

[0004] Model-driven low-light enhancement methods rely on predefined physical or mathematical models to model the physical properties of image illumination and reflectivity, simulating the human visual perception mechanism of images in low-light environments. By adjusting illumination and reflectivity, model-driven methods can effectively improve image brightness and contrast, and restore image details to a certain extent.

[0005] The advantages of this type of method are that it has a solid theoretical foundation, good interpretability, and a controllable model-driven enhancement process, which can usually show stable effects on specific image features. However, this type of method also has obvious limitations. On the one hand, model-driven methods usually need to rely on the parameters of the physical model. The selection and adjustment of these parameters require the preset of specific characteristics of the image and lack sufficient adaptability. On the other hand, due to the limitations of physical or mathematical models, the enhancement effect of model-driven methods is often not ideal when dealing with complex environments. In addition, the enhancement effect of model-driven methods in low-light scenes is usually limited and cannot meet the increasingly complex monitoring needs.

[0006] With the rapid development of deep learning and machine learning, data-driven low-light enhancement methods have gradually become a research hotspot. This type of method does not rely on physical models, but is trained through a large amount of low-light image data, so that the model automatically learns the mapping relationship between low-light images and normal brightness images. Typical deep learning enhancement networks use a large amount of paired data to train models to enhance the brightness, contrast, and details of images under low-light conditions. Technologies such as convolutional neural networks (CNNs) and generative adversarial networks (GANs) are widely used in data-driven low-light enhancement tasks.

[0007] Data-driven low-light enhancement methods can achieve good enhancement effects under data-rich conditions, especially in complex scenes, and can adaptively adjust the enhancement effect, thus making up for the shortcomings of model-driven methods to a certain extent. By training thousands of low-light images and normal images, deep learning models can capture the illumination and detail features in low-light scenes, and have high robustness in practical applications. However, data-driven methods also have some significant defects. First, such methods rely heavily on high-quality and rich training data. If there is insufficient data or there is a large difference between the training data set and the target application scenario, the enhancement effect of the model will be greatly reduced. Secondly, data-driven models are usually "black box" models, lack theoretical guidance, and it is difficult to explain their internal working mechanisms. In addition, such methods usually require a lot of computing resources and storage space, especially in resource-constrained environments such as edge devices, and cannot meet the needs of real-time processing well.

[0008] To address the limitations of model-driven and data-driven approaches, Deep Unfolding Network has gradually become a new solution. The design of Deep Unfolding Network is inspired by mapping traditional iterative optimization algorithms into a deep neural network structure. This architecture simulates the iterative optimization process, and each iterative step is designed as a layer in the network. Through deep unfolding, the theoretical advantages of model-driven and data-driven adaptive learning capabilities are combined.

[0009] The deep unfolding network has strong interpretability when dealing with low-light enhancement tasks. The operation of each layer corresponds to an iteration in the optimization process, allowing the network to gradually optimize the illumination and reflectance components of the image. Specifically, the deep unfolding network can gradually improve the brightness, contrast, and detail performance of the image by constructing a multi-stage network structure, with each stage performing independent iterative optimization on the illumination and reflectance components.

[0010] However, deep unfolding networks still face some challenges in application. First, deep unfolding networks usually contain multiple iterative stages, each of which requires a certain number of parameters, resulting in a large number of parameters for the overall model, which increases the burden of computation and storage. Second, due to the multi-stage structure of the network, the inference time is long, which makes it difficult to meet the real-time processing requirements of edge devices. Especially in campus monitoring systems, real-time performance is crucial, and deep unfolding networks are difficult to deploy directly on resource-constrained edge devices, limiting their application in actual low-light enhancement scenarios.

[0011] With the rapid development of the Internet of Things and edge computing technologies, more and more campus monitoring systems are deployed on edge devices to ensure real-time data processing and privacy protection. Edge devices usually have limited resources, so traditional low-light enhancement technology is difficult to apply directly. In order to achieve real-time behavior recognition in campus monitoring, low-light enhancement models need to be lightweight, low-latency, and efficient. Low-light enhancement models running on edge devices also need to consider the constraints of power, storage, and computing resources, so traditional models with high computational complexity are difficult to apply on edge devices.

[0012] In summary, existing low-light enhancement technologies face many challenges in campus monitoring systems. Model-driven methods lack adaptability and enhancement effects, data-driven methods require a high amount of data and are difficult to interpret, and although the deep unfolding network combines the advantages of both, it has a large number of parameters and poor real-time performance, making it difficult to deploy on resource-constrained edge devices. Therefore, how to design an efficient and lightweight low-light enhancement model to achieve real-time behavior recognition of campus monitoring systems on edge devices is an urgent problem to be solved in the current research on low-light enhancement technology. Summary of the invention

[0013] In order to solve the above problems in the prior art, the present invention proposes a low-light real-time enhancement method for park intelligent behavior recognition images based on a deep network, which is characterized by comprising the following steps:

[0014] Preprocess the input video stream to obtain compressed video frame images;

[0015] Decomposing the compressed video frame image into an illumination component and a reflectivity component;

[0016] The illumination component and the reflectance component are processed stage by stage using a deep network, and the stage by stage processing includes:

[0017] Perform gradient updates on the illumination component and reflectance component respectively, generate a gradient update module, and perform iterative updates through depthwise separable convolution;

[0018] performing transform domain enhancement on the illumination component;

[0019] The reflectance component is optimized and its spatial resolution is restored through downsampling, feature transformation and upsampling;

[0020] After the processing is completed, the illumination component and the reflectance component are combined to obtain an enhanced RGB image;

[0021] The enhanced RGB image is input into the behavior recognition model for subsequent behavior recognition. The gradient update module simulates the traditional gradient descent method to optimize the illumination component and the reflectivity component, and uses the residual connection to achieve continuous gradient update. The update formula is:

[0022]

[0023] in:

[0024] R new : Updated reflectivity component;

[0025] R old : Current reflectivity component;

[0026] γ R : learning rate of reflectivity component;

[0027] The gradient of the reflectivity component represents the partial derivative of the energy function E with respect to the reflectivity component R;

[0028] L new : Updated lighting component;

[0029] L old : Current lighting component;

[0030] γ L : learning rate of the illumination component;

[0031] The gradient of the illumination component represents the partial derivative of the energy function E with respect to the illumination component L;

[0032] The energy function E is a function calculated based on the reconstruction error of the reflectivity and illumination components.

[0033] The gradient update module adopts depth-separable convolution, including:

[0034] Deep convolution: Apply a separate convolution kernel to each input channel independently, so that when there are D channels in the input feature map, there are D convolution kernels, and each convolution kernel performs convolution operation only on the corresponding single channel;

[0035] Point-wise convolution: Use a 1×1 convolution kernel to perform convolution operations and combine the outputs of depth-wise convolution.

[0036] The illumination enhancement module enhances the illumination component L by the following steps:

[0037] L new =iFFT(FFT(L)·M)

[0038] Among them, firstly, the fast Fourier transform (FFT) is used to transform the illumination component from the spatial domain to the frequency domain; the amplitude transformation map M is applied in the frequency domain to adjust the amplitude of the illumination component; finally, the illumination component is transformed from the frequency domain back to the spatial domain through the inverse Fourier transform (iFFT).

[0039] The reflectivity optimization module uses the attention mechanism to optimize the reflectivity component through the following steps:

[0040] The reflectance component is converted into a deep-level feature by three consecutive downsamplings to extract key information in the reflectance component;

[0041] The attention module calculates the correlation between different features, weights the different features, and highlights the key image content. The specific calculation method is as follows:

[0042] Q=W q F+b q ,K=W k F+b k ,V=W v F+b v

[0043]

[0044] Attention(Q,K,V)=AV

[0045] in:

[0046] F: deep features of the input;

[0047] W q : The weight matrix of the query feature, used to map the input feature F to the query space;

[0048] W k : The weight matrix of key features, used to map the input features F to the key space;

[0049] W v: The weight matrix of the value feature, used to map the input feature F to the value space;

[0050] b q : Bias vector of query feature;

[0051] b k : bias vector of key features;

[0052] b v : bias vector of value feature;

[0053] Q: query feature matrix;

[0054] K: key feature matrix;

[0055] V: value feature matrix;

[0056] d k : Dimension of key features;

[0057] A: attention weight matrix;

[0058] Attention(Q,K,V): attention-weighted output;

[0059] The weighted deep features are upsampled and restored to the same spatial resolution as the original reflectance component.

[0060] The method uses images under low-light and normal-light conditions for model training. The data set selects public data sets LOL and SICE and performs preprocessing. The data set is divided into a training set and a test set in a ratio of 8:2.

[0061] The training loss function Loss includes the following parts:

[0062] Loss = λ1·Loss LR +λ2·Loss L +λ3·Loss R

[0063]

[0064]

[0065] in:

[0066] Loss LR : Overall loss of illumination and reflectivity;

[0067] L real : real lighting component;

[0068] L pred : Predict the illumination component;

[0069] Rreal : true reflectivity component;

[0070] R pred : Predicted reflectivity component;

[0071] Loss L : Light component loss at each stage;

[0072] K: the number of stages of the deep unfolding network;

[0073] L pred,k : The predicted illumination component of the kth stage;

[0074] Loss R : Reflectivity component loss stage by stage;

[0075] R pred,k : predicted reflectivity component of the kth stage;

[0076] λ1: Control Loss LR The weight of contribution to the overall loss;

[0077] λ2: Control Loss L The weight of contribution to the overall loss;

[0078] λ3: Control Loss R The weight of the contribution to the overall loss.

[0079] The optimizer used in the optimization adopts the Adam algorithm, and the learning rate is set to 0.001.

[0080] Beneficial effects:

[0081] The present invention optimizes the illumination component and the reflectance component separately by introducing a stage-by-stage processing mechanism of a deep network, thereby achieving brightness enhancement and detail restoration of low-light images. The computational complexity is reduced by using deep separable convolution, and the performance of the illumination component is improved by transform domain enhancement. The feature optimization module effectively restores the spatial resolution of the reflectance component through downsampling and upsampling, thereby generating a high-quality enhanced RGB image. This method is both real-time and lightweight, and is suitable for resource-constrained edge devices. It effectively improves the accuracy of behavior recognition and the operating efficiency of the campus monitoring system in low-light environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present application, but do not constitute an improper limitation of the present invention. In the drawings:

[0083] Figure 1 A flow chart of the method of the present invention is shown. DETAILED DESCRIPTION

[0084] The present invention will be described in detail below in conjunction with the accompanying drawings and specific embodiments, wherein the illustrative embodiments and descriptions are only used to explain the present invention but are not intended to limit the present invention.

[0085] The present invention discloses a method for real-time low-light enhancement of park intelligent behavior recognition images based on a deep network. Figure 1 As shown in the figure, it is mainly used for monitoring scenarios at night or in low-light environments in campuses, factories and other parks. This method uses an improved deep unfolding network architecture to perform gradient optimization and enhancement processing on the illumination component (L) and reflectivity component (R) of the low-light image, and combines the attention mechanism, transform domain filtering and deep separable convolution technology to achieve real-time image enhancement on edge devices with low computing resources, ensuring the real-time and reliability of the monitoring system. The specific implementation process is as follows.

[0086] First, the original video stream collected by the monitoring system is input into the low-light enhancement model of the present invention for processing. In order to adapt to the resource limitations of edge devices and improve the real-time response capability of the system, the present invention first performs video compression and format conversion before the video stream enters the deep expansion network for processing. Specifically, the present invention adopts an efficient video compression algorithm to convert a video stream with an original resolution of 720P into a 480P frame image with a lower resolution. This not only reduces the bandwidth and space required for data transmission and storage, but also significantly reduces the computational complexity of network processing, so that the system can run efficiently and stably on edge devices with limited hardware resources.

[0087] The compressed frame image is used as the input of the network and enters the preprocessing layer of the deep expansion network. In the preprocessing layer, the compressed image is further adjusted to be compatible with the subsequent network architecture module, thus laying the foundation for the next step of extracting the illumination and reflectivity components. This processing step can effectively optimize the overall performance of the system, greatly improving the real-time performance of image processing and the resource utilization of the system while ensuring the video clarity and behavior recognition accuracy.

[0088] In the preprocessing stage, the compressed video frame image is first processed through a linear transformation layer to decompose the image's illumination component (L) and reflectance component (R). The decomposition process is based on the Retinex theory, which is a visual model theory proposed by Edwin Land. It is believed that when human vision observes a scene, it can separate the surface characteristics of an object from the illumination conditions, thereby achieving color constancy. Based on this theory, the present invention decomposes the input image I into the illumination component L and the reflectance component R, satisfying the following relationship:

[0089] I=R·L

[0090] Among them, L represents the illumination intensity of the image, reflecting the illumination conditions in the scene; R represents the reflective characteristics of the object, which is the inherent visual feature of the object in the scene. Through this decomposition, the illumination information of the image can be effectively separated from the color and texture of the object itself.

[0091] In the deep expansion network architecture of the present invention, the decomposed illumination component (L) and reflectance component (R) will be used as independent input items to enter different network modules for subsequent optimization processing. The illumination component (L) focuses on enhancing brightness and contrast to optimize the overall visibility of the image; while the reflectance component (R) focuses on restoring details and colors to improve the clarity of image details. Through independent optimization processing of the two components, the present invention can achieve image enhancement in low-light environments, provide high-quality input images for subsequent behavior recognition, and ensure the accuracy and real-time nature of recognition.

[0092] The deep unfolding network architecture of the present invention is a multi-layer structure consisting of K stages, each of which consists of three key modules to gradually optimize the illumination component (L) and reflectivity component (R) of low-light images, thereby achieving efficient image enhancement. The processing of each stage follows a series of rigorous iterative steps, allowing the model to continuously improve the quality and detail of images under low-light conditions.

[0093] The design of the gradient update module is derived from the traditional gradient descent optimization algorithm, which aims to iteratively update the illumination component (L) and the reflectivity component (R). In each stage, the module performs an optimization iteration on L and R respectively. The update process can be expressed by the following formula:

[0094]

[0095] Among them, γ R and γ L are the learning rates for R and L, respectively, used to control the step size of each update; and Respectively represent the gradient of the energy function E with respect to R and L, reflecting the error direction and magnitude of the current component. By gradually reducing the error, the module can effectively optimize the estimated values ​​of illumination and reflectivity, thereby improving the quality of the image.

[0096] in:

[0097] R new : Updated reflectance component.

[0098] R old : Current reflectivity component.

[0099] γ R: The learning rate of the reflectivity component, which controls the step size of the update. It is usually a positive number and is used to adjust the convergence speed of the gradient descent method.

[0100] The gradient of the reflectivity component represents the partial derivative of the energy function E with respect to the reflectivity component R, and reflects the direction and degree of change of the current reflectivity component in the energy function.

[0101] L new : Updated lighting component.

[0102] L old : Current lighting component.

[0103] γ L : The learning rate of the illumination component, which controls the step size of the update. It is usually a positive number and is used to adjust the convergence speed of the gradient descent method.

[0104] The gradient of the illumination component represents the partial derivative of the energy function E with respect to the illumination component L, and reflects the direction and degree of change of the current illumination component in the energy function.

[0105] The energy function E is a function calculated based on the reconstruction error of the reflectivity and illumination components, and is usually used to optimize the image quality and reduce the error in image reconstruction. old and L old , to minimize the energy function and thus optimize the image quality.

[0106] In this network design, the gradient update module simulates the traditional gradient update process by combining residual connection and depthwise separable convolution, thereby significantly reducing the computational complexity while maintaining efficient optimization capabilities. Specifically:

[0107] Residual connection: By introducing residual connection in the update process, the network can better retain and transfer the feature information of the previous layer, thereby avoiding the gradient vanishing problem and accelerating the convergence process.

[0108] Depthwise separable convolution: Depthwise separable convolution consists of two parts, namely depthwise convolution and pointwise convolution. Depthwise convolution applies convolution operation to each input channel independently, thus reducing the computational requirements across channels; pointwise convolution uses 1×1 convolution kernel to integrate the output of depthwise convolution to achieve information fusion between channels. This design greatly reduces the computational complexity of convolution operation, enabling the model to run efficiently in environments with limited computing resources such as edge devices.

[0109] Through such a design, the gradient update module of the present invention can achieve rapid optimization of image illumination and reflectivity components with low computational overhead in resource-constrained scenarios, effectively improving the performance and efficiency of low-light enhancement.

[0110] The illumination enhancement module of the present invention is mainly used to enhance the illumination component L, and improve the brightness and contrast of the image through frequency domain transformation, so as to make the image clearer under low light conditions. The module first uses fast Fourier transform (FFT) to convert the illumination component L from the spatial domain to the frequency domain, so as to perform illumination enhancement operations more efficiently in the frequency domain. In the frequency domain, the pre-trained amplitude transformation map M is applied to perform corresponding enhancement processing on the illumination component, thereby improving the overall brightness. The mathematical expression of the entire processing flow is:

[0111] L new =iFFT(FFT(L)·M)

[0112] Specifically, the illumination component L is first converted to the frequency domain through the fast Fourier transform (FFT), and the amplitude transform map M is applied in the frequency domain to adjust the amplitude intensity of the illumination component. This enhancement operation can achieve a global brightness increase or adjustment, so that the image has higher visibility and detail presentation in low-light environments. After the enhancement is completed, the adjusted illumination component L is converted back to the spatial domain through the inverse fast Fourier transform (iFFT) to ensure that it is integrated with other components of the image, thereby achieving a natural illumination enhancement effect.

[0113] Through this frequency domain-based illumination enhancement method, the present invention can significantly improve the brightness and contrast of the image in a low-light environment while maintaining a natural visual effect, providing clearer and more accurate image input for subsequent behavior recognition processes.

[0114] The reflectivity optimization module in the present invention is designed based on the attention mechanism, and is mainly used to optimize the details and color information of the reflectivity component R to improve the detail performance and color reproduction ability of the image in low-light environments. The module first downsamples the R component multiple times, and extracts deep feature representations through three downsamplings, thereby capturing the key information in the reflectivity component.

[0115] After obtaining the deep features, the module calculates the attention weights of the features, which are calculated by the following steps:

[0116] Q=W q F+b q ,K=W k F+b k ,V=W v F+b v

[0117]

[0118] Attention(Q,K,V)=AV

[0119] Among them, W q , W k and W v are weight matrices used to generate query, key, and value features, respectively. q , b k and b v is the corresponding bias vector, and A is the attention weight matrix calculated by the SoftMax function. Finally, Attention(Q,K,V) represents the weighted output, which is used to enhance the important feature information in the image.

[0120] The description of each parameter is as follows:

[0121] F: The deep features of the input come from the feature representation of the reflectivity component after downsampling.

[0122] W q ,W k ,W v : They are the weight matrices of query, key, and value features, respectively, used to generate correlations between feature representations.

[0123] W q : The weight matrix of the query feature, used to map the input feature F to the query space.

[0124] W k : The weight matrix of key features, used to map the input features F to the key space.

[0125] Q v : The weight matrix of the value feature, used to map the input feature F to the value space.

[0126] b q ,b k ,b v : They are the bias vectors of query, key, and value, respectively, used to adjust the linear transformation results of the corresponding features.

[0127] b q : Bias vector of query features.

[0128] b k : bias vector of key features.

[0129] b v : Bias vector for value features.

[0130] Q: query feature matrix, calculated as Q = W q F+b q .

[0131] K: key feature matrix, calculated as K = W k F+b k .

[0132] V: value feature matrix, calculated as V = W v F+b v .

[0133] d k : The dimension of the key feature, used to scale the inner product calculation to prevent the value from being too large and affecting the gradient stability.

[0134] A: Attention weight matrix, calculated as Used to represent the correlation between the query and key features. The SoftMax function converts the correlation into a weight range [0,1], where the sum of the weights is 1.

[0135] Attention(Q,K,V): The weighted output of attention, calculated as Attention(Q,K,V) = AV. The value feature matrix V is weighted by weight A to obtain the weighted output features, highlighting the key image content in the reflectance component.

[0136] The weighted deep features are upsampled and restored to the same spatial resolution as the original reflectance components to ensure the integrity and clarity of the enhanced image details.

[0137] The calculation of the attention weight A enables the model to prioritize and enhance features with higher importance based on the correlation between features, thereby retaining and strengthening key details and color information. Through this weighted output, the key details and color performance of the image under low light conditions are enhanced, further improving the visual quality of the image.

[0138] Finally, the optimized reflectance component is restored to the original resolution through upsampling to ensure the detail richness and color accuracy of the final image, so that the image has both clear details and realistic color restoration effects, thereby providing higher quality input data for behavior recognition in low-light environments.

[0139] After K stages of iterative optimization, the network of the present invention combines the optimized illumination component L and reflectance component R to reconstruct an enhanced RGB image. In this process, the illumination component L provides the brightness information of the image, while the reflectance component R retains the details and color characteristics of the object. Through this combination, the generated RGB image can effectively improve the contrast and brightness in a low-light environment, making the image clearer and more detailed.

[0140] This reconstructed RGB image significantly enhances the brightness, contrast and detail, and can clearly display scene details in low-light environments, meeting the needs of the campus monitoring system and providing a high-quality image foundation for subsequent intelligent behavior recognition, thereby improving the reliability and applicability of the monitoring system in low-light conditions.

[0141] After the low-light enhancement model of the present invention is processed, the generated enhanced RGB image will be input into the behavior recognition model (such as the EdgeYOLO model) for intelligent behavior recognition analysis in the park monitoring scene. Through this behavior recognition model, the system can accurately identify the posture, movements and potential dangerous behaviors of people appearing in the image, thereby providing real-time and safe monitoring feedback. The model can efficiently detect and identify abnormal activities that may affect the safety of the park, such as falls, entering restricted areas or other unsafe behaviors, thereby ensuring the efficiency and real-time nature of park safety management.

[0142] The method of the present invention is trained based on the public low-light enhancement datasets LOL and SICE. The datasets are divided into training sets and test sets in a ratio of 8:2 to improve the generalization ability of the model so that it can adapt to a variety of low-light environments in practical applications. During the training process, the present invention uses the following loss function to guide the optimization process of the network:

[0143] Loss = λ1·Loss LR +λ2·Loss L +λ3·Loss R

[0144] Among them, the loss function of each part is specifically defined as follows:

[0145]

[0146] In the above formula, Loss LR Used to measure the real lighting and reflectivity component L real With the prediction component L pred The difference between L and Loss RThe stage-by-stage errors of the illumination component and the reflectivity component in each stage are respectively accumulated and calculated. Through this decomposition loss, the present invention can effectively optimize the segmented learning effect of the network on illumination and reflectivity.

[0147] Among them, Loss LR :The overall loss of illumination and reflectivity, calculates the overall error between the predicted illumination component and reflectivity component and the real illumination component and reflectivity component.

[0148] L real : Real lighting component, which represents the lighting information obtained from real data.

[0149] L pred : Predicted illumination component, which represents the illumination information output by the deep network.

[0150] R real : Real reflectance component, which represents the reflectance information obtained from real data.

[0151] R pred : Predicted reflectivity component, representing the reflectivity information output by the deep network.

[0152] Represents the squared Euclidean norm, which measures the difference between the predicted value and the true value.

[0153] Loss L : The illumination component is lost stage by stage, and the error of the illumination component of all stages is accumulated.

[0154] K: The number of stages of the deep unrolled network, each stage performs an independent optimization on the illumination component.

[0155] L pred,k : The predicted illumination component of the kth stage, which represents the illumination component output by the network at the current stage.

[0156] It means summing up the illumination component errors of all stages.

[0157] Loss R : Reflectivity component loss at each stage, and error accumulation of reflectivity components at all stages.

[0158] R pred,k : The predicted reflectivity component of the kth stage, which represents the reflectivity component output by the network at the current stage.

[0159] λ1, λ2, λ3: loss weight coefficients

[0160] λ1: Control Loss LR The weight of the contribution to the overall loss.

[0161] λ2: Control Loss L The weight of the contribution to the overall loss.

[0162] λ3: Control Loss R The weight of the contribution to the overall loss.

[0163] These weight coefficients are used to balance the impact of different loss terms on the optimization objective and are usually adjusted according to training requirements to achieve coordinated optimization of the illumination component and the reflectance component.

[0164] Through the design of this loss function, this method can simultaneously optimize the overall accuracy of illumination and reflectance components and the stage-by-stage optimization effect, thereby improving the performance and image quality of the low-light enhancement network.

[0165] The present invention uses the Adam optimizer for model training, and the learning rate is set to 0.001 to ensure that the network can converge quickly during the training process and avoid overfitting and gradient oscillation problems. The selection of this optimizer and the setting of the learning rate ensure the stability and efficiency of the training process, so that the model can efficiently complete the image enhancement task under low-light conditions, thereby providing clear and accurate image input for subsequent behavior recognition.

[0166] The above description is only a preferred embodiment of the present invention, so all equivalent changes or modifications made according to the structure, characteristics and principles described in the scope of the patent application of the present invention are included in the scope of the patent application of the present invention.

Claims

1. A method for real-time low-light enhancement of park intelligent behavior recognition images based on deep networks, characterized by: The following steps are involved: Preprocess the input video stream to obtain compressed video frame images; Decomposing the compressed video frame image into an illumination component and a reflectivity component; The illumination component and the reflectance component are processed stage by stage using a deep network, and the stage by stage processing includes: Perform gradient updates on the illumination component and reflectance component respectively, generate a gradient update module, and perform iterative updates through depthwise separable convolution; performing transform domain enhancement on the illumination component; The reflectance component is optimized and its spatial resolution is restored through downsampling, feature transformation and upsampling; After the processing is completed, the illumination component and the reflectance component are combined to obtain an enhanced RGB image; The enhanced RGB image is input into the behavior recognition model for subsequent behavior recognition.

2. The method for real-time low-light enhancement of park intelligent behavior recognition images based on deep network as claimed in claim 1, characterized in that: The gradient update module simulates the traditional gradient descent method to optimize the illumination component and the reflectivity component, and uses the residual connection to achieve continuous gradient update. The update formula is: in: R new : Updated reflectivity component; R old : Current reflectivity component; γ R : learning rate of reflectivity component; The gradient of the reflectivity component represents the partial derivative of the energy function E with respect to the reflectivity component R; L new : Updated lighting component; L old : Current lighting component; γ L : learning rate of the illumination component; The gradient of the illumination component represents the partial derivative of the energy function E with respect to the illumination component L; The energy function E is a function calculated based on the reconstruction error of the reflectivity and illumination components.

3. A method for real-time low-light enhancement of park intelligent behavior recognition images based on deep network as claimed in claim 1 or 2, characterized in that: The gradient update module adopts depth-separable convolution, including: Deep convolution: Apply a separate convolution kernel to each input channel independently, so that when there are D channels in the input feature map, there are D convolution kernels, and each convolution kernel performs convolution operation only on the corresponding single channel; Point-wise convolution: Use a 1×1 convolution kernel to perform convolution operations and combine the outputs of depth-wise convolution.

4. The method for real-time low-light enhancement of park intelligent behavior recognition images based on deep network as claimed in claim 1, characterized in that: The illumination enhancement module enhances the illumination component L by the following steps: L new =iFFT(FFT(L)·M) First, the illumination component is converted from the spatial domain to the frequency domain using the fast Fourier transform (FFT); the amplitude transform map M is applied in the frequency domain to adjust the amplitude of the illumination component; finally, the illumination component is converted from the frequency domain back to the spatial domain using the inverse Fourier transform (iFFT); new : Updated lighting component.

5. The method for real-time low-light enhancement of park intelligent behavior recognition images based on deep network as claimed in claim 1, characterized in that: The reflectivity optimization module uses the attention mechanism to optimize the reflectivity component through the following steps: The reflectance component is converted into a deep-level feature by three consecutive downsamplings to extract key information in the reflectance component; The attention module calculates the correlation between different features, weights the different features, and highlights the key image content. The specific calculation method is as follows: Q=W q f+b q ,K=W k F+b k ,V=W v F+b v Attention(Q,K,V)=AV in: F: deep features of the input; W q : The weight matrix of the query feature, used to map the input feature F to the query space; W k : The weight matrix of key features, used to map the input features F to the key space; W v : The weight matrix of the value feature, used to map the input feature F to the value space; b q : Bias vector of query feature; b k : bias vector of key features; b v : bias vector of value feature; Q: query feature matrix; K: key feature matrix; V: value feature matrix; d k : Dimension of key features; A: attention weight matrix; Attention(Q,K,V): attention-weighted output; The weighted deep features are upsampled and restored to the same spatial resolution as the original reflectance component.

6. The method for real-time low-light enhancement of park intelligent behavior recognition images based on deep network as claimed in claim 1, characterized in that: The method uses images under low-light and normal-light conditions for model training. The data set selects public data sets LOL and SICE and performs preprocessing. The data set is divided into a training set and a test set in a ratio of 8:

2.

7. The method for real-time low-light enhancement of park intelligent behavior recognition images based on deep network as claimed in claim 6, characterized in that: The training loss function Loss includes the following parts: Loss=λ1·Loss LR +λ2·Loss L +λ3·Loss R in: Loss LR : Overall loss of illumination and reflectivity; L real : real lighting component; L pred : Predict the illumination component; R real : true reflectivity component; R pred : Predicted reflectivity component; Loss L : Light component loss at each stage; K: the number of stages of the deep unfolding network; L pred,k : The predicted illumination component of the kth stage; Loss R : Reflectivity component loss stage by stage; R pred,k : predicted reflectivity component of the kth stage; λ1: Control Loss LR The weight of contribution to the overall loss; λ2: Control Loss L The weight of contribution to the overall loss; λ3: Control Loss R The weight of the contribution to the overall loss.

8. The method for real-time low-light enhancement of park intelligent behavior recognition images based on deep network as claimed in claim 1, characterized in that: The optimizer used in the optimization adopts the Adam algorithm, and the learning rate is set to 0.001.

Images