Real-time image denoising and enhancing method based on deep neural network

Abstract

The invention discloses an image real-time denoising and enhancing method based on a deep neural network, and the method comprises the following steps: collecting image data of different modes in parallel through a plurality of sensors, and capturing dynamic scene features through the cooperative work of the sensors; performing adaptive division on the acquired multi-modal image data, evaluating the characteristics of each sub-region according to environmental factors, and dynamically determining the processing priority of each sub-region; for different sub-regions, dynamically selecting the most suitable deep neural network architecture for parallel processing; adjusting the processing frequency through a self-adaptive time window in combination with the time sequence characteristics of the image; pixel-level dynamic fusion is carried out on the denoised and enhanced sub-regions; processing parameters are adjusted through a real-time feedback mechanism, incremental optimization and fine adjustment of the neural network model are achieved, and finally high-quality image data are output. The invention aims to improve the image quality in a dynamic environment through multi-modal data acquisition, adaptive division and real-time optimization.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a real-time image denoising and enhancement method based on a deep neural network. Background Art

[0002] With the development of image processing technology, achieving high-quality image denoising and enhancement in dynamic environments has gradually become an important research direction. Image denoising and enhancement technology is widely used in security monitoring, unmanned driving, medical imaging, industrial automation and other fields. Its goal is to obtain clear images in complex environments. However, existing technologies have obvious deficiencies in dealing with dynamically changing lighting conditions, high noise levels and the integration of multiple modal data in complex scenes.

[0003] Traditional image processing methods usually use a single sensor to collect image data. This method may be effective for specific scenarios or noise types, but it is difficult to meet the needs of multi-modal data fusion. For example, optical sensors may not be able to provide high-quality images in low-light environments, and although infrared or radar sensors can work in low-light conditions, they have difficulty capturing fine spatial features. Existing technologies lack a mechanism for multi-sensor collaboration, making it difficult to achieve real-time fusion of multi-modal image data in dynamic scenes. During the sensor data acquisition process, due to the working characteristics of different sensors and data synchronization issues, timing errors and spatial deviations are easily generated, resulting in a decrease in the final image quality.

[0004] In the image processing stage, existing technologies mostly use fixed-parameter denoising and enhancement strategies, which cannot be adaptively adjusted to the diversity of image regions. For example, for images with uneven noise distribution, fixed-parameter denoising methods are prone to over-processing or under-processing of certain areas, thereby losing image details. Existing methods mainly focus on global processing in the image enhancement process, ignoring the illumination changes, texture complexity and dynamic characteristics of different regions, which makes it difficult to balance the overall and local optimization effects of image processing results.

[0005] Traditional methods also face the problem of consistency in the time dimension. For example, in dynamic scenes, the speed of image change and characteristics vary greatly. Existing technologies make it difficult to dynamically adjust processing strategies based on the timing characteristics of image changes, resulting in discontinuity in processing results in the time dimension, affecting the visual effect of the image. At the same time, existing technologies lack a real-time feedback mechanism in the image processing process, making it difficult to dynamically optimize processing parameters based on environmental changes and image quality assessment results, making the system unable to adapt to complex and changing scene requirements.

[0006] Image processing technology based on deep neural networks has received widespread attention in recent years, but existing methods usually rely on a single network architecture and have limited adaptability to different noise types or image characteristics. For example, convolutional neural networks are suitable for dealing with local noise problems, but lack adaptability to lighting changes, while recursive neural networks are good at time series data processing, but may not perform well in scenes with high texture complexity. Existing technologies lack mechanisms for dynamic selection and multi-network collaborative optimization, which makes it difficult to achieve optimal processing effects in complex scenes.

[0007] Therefore, how to provide a real-time image denoising and enhancement method based on deep neural network is a problem that technicians in this field need to solve urgently. Summary of the invention

[0008] One object of the present invention is to propose a real-time image denoising and enhancement method based on deep neural network. The present invention makes full use of multi-sensor collaborative acquisition, multi-modal data fusion, deep neural network dynamic selection and adaptive optimization technology, and describes in detail the specific process of realizing image denoising and enhancement in a complex dynamic environment. It has the advantages of strong real-time performance, high image clarity, strong processing stability and wide adaptability.

[0009] According to an embodiment of the present invention, a method for real-time image denoising and enhancement based on a deep neural network comprises the following steps:

[0010] S1. In the image data acquisition stage, multiple heterogeneous sensors are used to collect image data of different modalities in parallel. During the acquisition process, image data of different modalities work together through sensors to automatically capture scene features with dynamic changes in the environment;

[0011] S2, dividing the acquired multimodal image data into multiple sub-regions in an adaptive manner, wherein the division criteria are based on environmental factors, and the processing priority and denoising strategy of different regions are determined by evaluating the dynamic characteristics of each sub-region;

[0012] S3. Through a dynamic selection mechanism, the corresponding deep neural network branches are automatically selected for parallel processing according to the processing priority of each sub-region. The neural network branches include multiple deep neural network architectures, and each architecture has an optimized processing effect on specific noise types and image features;

[0013] S4. During the processing, the timing characteristics of the image data and the environmental conditions are combined, and an adaptive time window is used to adjust the processing process according to the speed of image changes and real-time feedback;

[0014] S5, dynamically reconstruct the denoised and enhanced image sub-regions by association, and perform pixel-level step-by-step fusion according to the denoising strength and information integrity of different sub-regions, so that the image enhancement effect is continuous and seamless in spatial and temporal dimensions;

[0015] S6. Dynamically adjust processing parameters based on real-time feedback of environmental information and image quality assessment, monitor processing effects online through real-time feedback mechanism, and incrementally train and fine-tune the neural network model based on current conditions to maintain continuous optimization of image quality in a dynamic environment;

[0016] S7. Outputting real-time optimized image data, wherein the image data has image features after multimodal fusion and can ensure stable image clarity and low latency in a dynamic environment.

[0017] Optionally, the S1 specifically includes:

[0018] S11, collecting image data of different modalities in parallel through multiple heterogeneous sensors, wherein the sensors include optical sensors, infrared sensors, radar sensors, and sound sensors, wherein each sensor has different working characteristics, sampling accuracy, and adaptability;

[0019] S12, calculating the correlation between environmental conditions and sensor states in real time according to environmental feedback, and dynamically weighting and allocating image data of different modalities;

[0020] S13. According to the image change speed and spatial characteristics, the sampling timing and frequency of the sensor are optimized through time series modeling to achieve stable data acquisition and image quality;

[0021] S14, evaluating the spatiotemporal consistency of image data of different modalities, and performing preprocessing by fusion weights and feedback of sensor states;

[0022] S15. Estimate the spatiotemporal deviation of different modal data, optimize the fusion of multi-sensor data through adaptive calibration and synchronization mechanism, make data seamlessly connected, and eliminate the error caused by time series asynchrony;

[0023] S16. Monitor the sensor status in real time, warn of possible failures and abnormal conditions, and automatically adjust the collection strategy based on real-time feedback to select the optimal sensor for data collection.

[0024] Optionally, the S2 specifically includes:

[0025] S21, adaptively dividing the acquired multimodal image data according to environmental factors, dynamically adjusting the division criteria by real-time analysis of the characteristics of each sub-region to cope with the variability of environmental conditions;

[0026] S22, dividing the image data into a plurality of sub-regions according to an adaptive division standard, where the division is based on the spatial structure characteristics, texture complexity, time variation characteristics, and dynamic characteristics of each sub-region, so that the processing of each sub-region can adapt to different environmental conditions and image characteristics;

[0027] S23. By evaluating the dynamic characteristics of each sub-region in real time, taking into account the noise intensity, image change speed, illumination change degree and texture complexity, the processing priority of each sub-region is calculated and dynamically adjusted:

[0028]

[0029] Among them, P i is the processing priority of sub-region i, N i is the noise intensity of sub-region i, N max is the maximum noise intensity of all sub-regions in the image, ΔL i is the illumination change of sub-region i, L ref is the illumination reference value, V i is the image change speed of sub-region i, V max is the maximum change speed of all sub-regions in the image, T i is the texture complexity of sub-region i, T max is the maximum texture complexity of all sub-regions in the image, α, β, γ, δ are the weight coefficients of each factor;

[0030] S24, according to the processing priority P of the sub-region i , assign corresponding denoising strategies to each region, so that different characteristic regions can adopt appropriate denoising methods to avoid the simplification of processing strategies;

[0031] S25, performing dynamic denoising on each sub-region, giving priority to regions with greater noise intensity, and optimizing image quality by dynamically adjusting denoising intensity in other regions, while balancing processing speed and image clarity;

[0032] S26, real-time monitoring of the processing progress and effect of each sub-region, dynamically adjusting the division criteria, processing priority and denoising strategy according to the monitoring feedback, and optimizing the overall processing effect of the image;

[0033] S27. Based on environmental feedback and real-time evaluation of image quality, the region division and denoising strategies are further optimized. During the processing of each sub-region, multiple iterations and parameter fine-tuning are performed to achieve the optimal fusion effect of the image in time and space dimensions.

[0034] Optionally, the S3 specifically includes:

[0035] S31, according to the processing priority P of each sub-areai , through a dynamic selection mechanism, an adaptive deep neural network branch is selected to process each sub-region independently. The neural network branches include convolutional neural networks, recurrent neural networks, and network architectures based on attention mechanisms. Each network architecture is optimized for different noise types, lighting conditions, and texture complexity;

[0036] S32, in the processing process of each sub-area, adaptively adjust the parameters of the neural network branch based on the real-time feedback results to adapt to the changes in different environmental conditions;

[0037] S33, align the spatial and temporal features of the sub-region results after parallel processing, smooth the sub-region boundaries through an adaptive fusion algorithm, and calculate the fusion weight W of each sub-region at position (x, y) and time t. i The calculation formula for (x,y,t) is:

[0038]

[0039] Among them, W i (x, y, t) is the fusion weight of sub-region i, E i (x,y) is the edge strength, N i (x,y) is the noise level, ΔT i (x,y) is the time variation, T max is the maximum time variation, λ,μ,ν are the weight coefficients of each factor;

[0040] S34. Based on the multi-sub-region processing results, the feature weighted fusion method is used to integrate the processing outputs of all sub-regions into a complete image data output, optimize the clarity and noise suppression effect of the multimodal image, and maintain the consistency of the details and texture features of each region.

[0041] Optionally, the S4 specifically includes:

[0042] S41, based on the time series characteristics of the image data, define an adaptive time window T for each sub-region w (i)

[0043]

[0044] Among them, T w (i) is the time window of sub-region i, V i is the image change speed of sub-region i, L i is the current light intensity of sub-region i, L ref is the illumination reference value, N i is the noise level of sub-region i, α, β, γ are weight coefficients;

[0045] S42, based on the calculated time window T w (i) Dynamically adjust the processing frequency of each sub-region, assign a smaller time window to the sub-region with a faster change rate to optimize the processing frequency, and assign a larger time window to the sub-region with a slower change rate to optimize resource allocation;

[0046] S43, based on the real-time processing status of each sub-area, monitor the time window T w (i) Applicability: combining real-time feedback to adjust the time window size of each sub-region to adapt to the dynamic changes of regional characteristics, so that the time window always maintains the optimal configuration;

[0047] S44, based on the adaptive time window T w (i) The sub-region results after dynamic processing are synchronized, and the time series data of each sub-region are aligned through the time window alignment algorithm to eliminate the time difference between the sub-regions and ensure the consistency of the processing results in the time dimension;

[0048] S45. Combined with the real-time feedback of environmental changes, the time window parameters of all sub-areas are optimized, and the balance between image clarity and processing frequency is refined by adjusting the weight coefficients α, β, and γ.

[0049] Optionally, the S5 specifically includes:

[0050] S51, based on real-time feedback of environmental changes, monitor the image characteristics of each sub-area to obtain real-time image quality information including signal-to-noise ratio, contrast, illumination change amount and image change speed;

[0051] S52, using the image quality information, dynamically adjusting the denoising intensity, contrast and brightness enhancement parameters of each sub-region so that the processing of each sub-region matches its real-time characteristics;

[0052] S53, after each sub-region is subjected to denoising and enhancement processing, a pixel-level dynamic association reconstruction method is used to gradually merge the processing results of each sub-region, and the pixels of each sub-region are integrated to obtain a fused image pixel value P(x, y):

[0053]

[0054] Among them, W i (x, y) is the weight coefficient of sub-region i at position (x, y), P i (x, y) is the pixel value of sub-region i at position (x, y), Q i is the denoising strength factor of sub-region i, L i is the current light intensity of sub-region i, L ref is the reference illumination value, SNR iis the signal-to-noise ratio of sub-region i, C i is the contrast of sub-region i, N i is the noise level of sub-region i, ∈ is a small positive number to avoid the denominator being zero, and N is the total number of sub-regions;

[0055] S54. According to the environmental feedback data, the processing parameters are adaptively optimized, and the image clarity, denoising and enhancement effects are balanced by adjusting the weights of the parameters, so that the processing of each sub-area maintains consistency and stability in a changing environment.

[0056] Optionally, the S6 specifically includes:

[0057] S61. During the image processing process, each sub-region after denoising and enhancement is monitored in real time, and image characteristic information such as signal-to-noise ratio, contrast, illumination change, and image change speed of each sub-region is collected to generate real-time image quality assessment data;

[0058] S62, using the real-time image quality assessment data, dynamically optimizing the denoising intensity, contrast and brightness enhancement parameters of each sub-region, so that the processing parameters of each sub-region are adapted to its current image characteristics;

[0059] S63, after dynamic optimization, processing the image data of each sub-region based on the optimized denoising and enhancement parameters, integrating the processing results by a step-by-step fusion method, achieving spatial and temporal consistency of each sub-region through pixel-level integration, so that the image presents a seamless visual effect between regions;

[0060] S64, combining environmental feedback data, adaptively adjusting processing parameters, and dynamically balancing denoising intensity, contrast, and brightness enhancement parameters to keep the image clarity and quality of each sub-area stable in a changing environment;

[0061] S65. According to the feedback data of each sub-region, the parameter weight coefficients used are gradually adjusted to achieve multi-dimensional adaptive optimization.

[0062] The beneficial effects of the present invention are:

[0063] The present invention significantly improves the adaptability and processing effect of image denoising and enhancement in complex environments by introducing multimodal data acquisition, adaptive partitioning, dynamic selection of deep neural networks and real-time optimization mechanisms. It utilizes multiple heterogeneous sensors to collect image data of different modalities in parallel, overcomes the deficiency that a single sensor cannot fully capture the characteristics of dynamic scenes, and realizes efficient collaborative fusion of multimodal data. Through the adaptive partitioning method, the present invention can dynamically adjust the image partitioning criteria according to environmental factors, and assign processing priorities and adaptive denoising strategies to different sub-regions, so that each region achieves the best effect in noise suppression and detail retention.

[0064] The present invention adopts a dynamic selection mechanism to flexibly match the deep neural network architecture according to the image characteristics, and optimizes and adjusts the network parameters in combination with real-time feedback, thereby enhancing the pertinence and efficiency of processing, especially showing extremely high robustness and flexibility in complex dynamic scenes. By introducing an adaptive time window, accurate matching of the image change speed is achieved in the time dimension, effectively avoiding processing delays or detail loss problems, while enhancing the continuity and stability of the image.

[0065] The present invention combines a pixel-level dynamic fusion algorithm to gradually integrate the denoised and enhanced sub-region data, achieving consistency in spatial and temporal dimensions, significantly improving the visual coherence and naturalness of the image. Combined with a real-time feedback mechanism, the present invention further optimizes processing parameters, dynamically adjusts weight coefficients to adapt to environmental changes, and ensures high-quality output of images under changing conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0067] Figure 1 A flowchart of a method for real-time image denoising and enhancement based on a deep neural network proposed by the present invention;

[0068] Figure 2 This is a flow chart of adaptive time window adjustment and dynamic optimization in the present invention;

[0069] Figure 3 It is a structural schematic diagram of pixel-level dynamic fusion and spatial and temporal consistency optimization in the present invention. DETAILED DESCRIPTION

[0070] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, which only illustrate the basic structure of the present invention in a schematic manner, and therefore only show the components related to the present invention.

[0071] refer to Figure 1-3 , a real-time image denoising and enhancement method based on deep neural network, comprising the following steps:

[0072] S1. In the image data acquisition stage, multiple heterogeneous sensors are used to collect image data of different modalities in parallel. During the acquisition process, image data of different modalities work together through sensors to automatically capture scene features with dynamic changes in the environment;

[0073] S2, dividing the acquired multimodal image data into multiple sub-regions in an adaptive manner, wherein the division criteria are based on environmental factors, and the processing priority and denoising strategy of different regions are determined by evaluating the dynamic characteristics of each sub-region;

[0074] S3. Through a dynamic selection mechanism, the corresponding deep neural network branches are automatically selected for parallel processing according to the processing priority of each sub-region. The neural network branches include multiple deep neural network architectures, and each architecture has an optimized processing effect on specific noise types and image features;

[0075] S4. During the processing, the timing characteristics of the image data and the environmental conditions are combined, and an adaptive time window is used to adjust the processing process according to the speed of image changes and real-time feedback;

[0076] S5, dynamically reconstruct the denoised and enhanced image sub-regions by association, and perform pixel-level step-by-step fusion according to the denoising strength and information integrity of different sub-regions, so that the image enhancement effect is continuous and seamless in spatial and temporal dimensions;

[0077] S6. Dynamically adjust processing parameters based on real-time feedback of environmental information and image quality assessment, monitor processing effects online through real-time feedback mechanism, and incrementally train and fine-tune the neural network model based on current conditions to maintain continuous optimization of image quality in a dynamic environment;

[0078] S7. Outputting real-time optimized image data, wherein the image data has image features after multimodal fusion and can ensure stable image clarity and low latency in a dynamic environment.

[0079] In this implementation, S1 specifically includes:

[0080] S11, collecting image data of different modalities in parallel through multiple heterogeneous sensors, wherein the sensors include optical sensors, infrared sensors, radar sensors, and sound sensors, wherein each sensor has different working characteristics, sampling accuracy, and adaptability;

[0081] S12, calculating the correlation between environmental conditions and sensor states in real time according to environmental feedback, and dynamically weighting and allocating image data of different modalities;

[0082] S13. According to the image change speed and spatial characteristics, the sampling timing and frequency of the sensor are optimized through time series modeling to achieve stable data acquisition and image quality;

[0083] S14, evaluating the spatiotemporal consistency of image data of different modalities, and performing preprocessing by fusion weights and feedback of sensor states;

[0084] S15. Estimate the spatiotemporal deviation of different modal data, optimize the fusion of multi-sensor data through adaptive calibration and synchronization mechanism, make data seamlessly connected, and eliminate the error caused by time series asynchrony;

[0085] S16. Monitor the sensor status in real time, warn of possible failures and abnormal conditions, and automatically adjust the collection strategy based on real-time feedback to select the optimal sensor for data collection.

[0086] In this implementation, S2 specifically includes:

[0087] S21, adaptively dividing the acquired multimodal image data according to environmental factors, dynamically adjusting the division criteria by real-time analysis of the characteristics of each sub-region to cope with the variability of environmental conditions;

[0088] S22, dividing the image data into a plurality of sub-regions according to an adaptive division standard, where the division is based on the spatial structure characteristics, texture complexity, time variation characteristics, and dynamic characteristics of each sub-region, so that the processing of each sub-region can adapt to different environmental conditions and image characteristics;

[0089] S23. By evaluating the dynamic characteristics of each sub-region in real time, taking into account the noise intensity, image change speed, illumination change degree and texture complexity, the processing priority of each sub-region is calculated and dynamically adjusted:

[0090]

[0091] Among them, P i is the processing priority of sub-region i, N i is the noise intensity of sub-region i, N max is the maximum noise intensity of all sub-regions in the image, ΔL i is the illumination change of sub-region i, L ref is the illumination reference value, V i is the image change speed of sub-region i, V max is the maximum change speed of all sub-regions in the image, T i is the texture complexity of sub-region i, T max is the maximum texture complexity of all sub-regions in the image, α, β, γ, δ are the weight coefficients of each factor;

[0092] S24, according to the processing priority P of the sub-region i , assign corresponding denoising strategies to each region, so that different characteristic regions can adopt appropriate denoising methods to avoid the simplification of processing strategies;

[0093] S25, performing dynamic denoising on each sub-region, giving priority to regions with greater noise intensity, and optimizing image quality by dynamically adjusting denoising intensity in other regions, while balancing processing speed and image clarity;

[0094] S26, real-time monitoring of the processing progress and effect of each sub-region, dynamically adjusting the division criteria, processing priority and denoising strategy according to the monitoring feedback, and optimizing the overall processing effect of the image;

[0095] S27. Based on environmental feedback and real-time evaluation of image quality, the region division and denoising strategies are further optimized. During the processing of each sub-region, multiple iterations and parameter fine-tuning are performed to achieve the optimal fusion effect of the image in time and space dimensions.

[0096] In this implementation, S3 specifically includes:

[0097] S31, according to the processing priority P of each sub-area i , through a dynamic selection mechanism, an adaptive deep neural network branch is selected to process each sub-region independently. The neural network branches include convolutional neural networks, recurrent neural networks, and network architectures based on attention mechanisms. Each network architecture is optimized for different noise types, lighting conditions, and texture complexity;

[0098] S32, in the processing process of each sub-area, adaptively adjust the parameters of the neural network branch based on the real-time feedback results to adapt to the changes in different environmental conditions;

[0099] S33, align the spatial and temporal features of the sub-region results after parallel processing, smooth the sub-region boundaries through an adaptive fusion algorithm, and calculate the fusion weight W of each sub-region at position (x, y) and time t. i The calculation formula for (x,y,t) is:

[0100]

[0101] Among them, W i (x, y, t) is the fusion weight of sub-region i, E i (x,y) is the edge strength, N i (x,y) is the noise level, ΔT i (x,y) is the time variation, T max is the maximum time variation, λ,μ,ν are the weight coefficients of each factor;

[0102] S34. Based on the multi-sub-region processing results, the feature weighted fusion method is used to integrate the processing outputs of all sub-regions into a complete image data output, optimize the clarity and noise suppression effect of the multimodal image, and maintain the consistency of the details and texture features of each region.

[0103] In this implementation, S4 specifically includes:

[0104] S41, based on the time series characteristics of the image data, define an adaptive time window T for each sub-region w (i)

[0105]

[0106] Among them, T w (i) is the time window of sub-region i, V i is the image change speed of sub-region i, L i is the current light intensity of sub-region i, L ref is the illumination reference value, N i is the noise level of sub-region i, α, β, γ are weight coefficients;

[0107] S42, based on the calculated time window T w (i) Dynamically adjust the processing frequency of each sub-region, assign a smaller time window to the sub-region with a faster change rate to optimize the processing frequency, and assign a larger time window to the sub-region with a slower change rate to optimize resource allocation;

[0108] S43, based on the real-time processing status of each sub-area, monitor the time window T w (i) Applicability: combining real-time feedback to adjust the time window size of each sub-region to adapt to the dynamic changes of regional characteristics, so that the time window always maintains the optimal configuration;

[0109] S44, based on the adaptive time window T w (i) The sub-region results after dynamic processing are synchronized, and the time series data of each sub-region are aligned through the time window alignment algorithm to eliminate the time difference between the sub-regions and ensure the consistency of the processing results in the time dimension;

[0110] S45. Combined with the real-time feedback of environmental changes, the time window parameters of all sub-areas are optimized, and the balance between image clarity and processing frequency is refined by adjusting the weight coefficients α, β, and γ.

[0111] In this implementation manner, S5 specifically includes:

[0112] S51, based on real-time feedback of environmental changes, monitor the image characteristics of each sub-area to obtain real-time image quality information including signal-to-noise ratio, contrast, illumination change amount and image change speed;

[0113] S52, using the image quality information, dynamically adjusting the denoising intensity, contrast and brightness enhancement parameters of each sub-region so that the processing of each sub-region matches its real-time characteristics;

[0114] S53, after each sub-region is subjected to denoising and enhancement processing, a pixel-level dynamic association reconstruction method is used to gradually merge the processing results of each sub-region, and the pixels of each sub-region are integrated to obtain a fused image pixel value P(x, y):

[0115]

[0116] Among them, W i (x, y) is the weight coefficient of sub-region i at position (x, y), P i (x, y) is the pixel value of sub-region i at position (x, y), Q i is the denoising strength factor of sub-region i, L i is the current light intensity of sub-region i, L ref is the reference illumination value, SNR i is the signal-to-noise ratio of sub-region i, C i is the contrast of sub-region i, N i is the noise level of sub-region i, ∈ is a small positive number to avoid the denominator being zero, and N is the total number of sub-regions;

[0117] S54. According to the environmental feedback data, the processing parameters are adaptively optimized, and the image clarity, denoising and enhancement effects are balanced by adjusting the weights of the parameters, so that the processing of each sub-area maintains consistency and stability in a changing environment.

[0118] In this implementation manner, S6 specifically includes:

[0119] S61. During the image processing process, each sub-region after denoising and enhancement is monitored in real time, and image characteristic information such as signal-to-noise ratio, contrast, illumination change, and image change speed of each sub-region is collected to generate real-time image quality assessment data;

[0120] S62, using the real-time image quality assessment data, dynamically optimizing the denoising intensity, contrast and brightness enhancement parameters of each sub-region, so that the processing parameters of each sub-region are adapted to its current image characteristics;

[0121] S63, after dynamic optimization, processing the image data of each sub-region based on the optimized denoising and enhancement parameters, integrating the processing results by a step-by-step fusion method, achieving spatial and temporal consistency of each sub-region through pixel-level integration, so that the image presents a seamless visual effect between regions;

[0122] S64, combining environmental feedback data, adaptively adjusting processing parameters, and dynamically balancing denoising intensity, contrast, and brightness enhancement parameters to keep the image clarity and quality of each sub-area stable in a changing environment;

[0123] S65. According to the feedback data of each sub-region, the parameter weight coefficients used are gradually adjusted to achieve multi-dimensional adaptive optimization.

[0124] Embodiment 1:

[0125] In order to verify the feasibility of the present invention in implementation, the present invention is applied to the field of intelligent security monitoring. The monitoring system often faces complex dynamic environments, such as lighting changes, noise interference, and real-time analysis requirements of multi-target scenes. Taking urban traffic monitoring as an example, when dealing with low-light and high-noise scenes, the quality of the images collected by traditional monitoring systems is often poor, and it is difficult to meet the requirements of accurate recognition and analysis; for example, at night traffic intersections, due to the reflection of car lights, insufficient ambient light, and the influence of complex road surface texture, traditional single-sensor monitoring equipment is easy to capture blurred and noisy images, and cannot effectively identify vehicle driving trajectories and traffic violations.

[0126] In order to verify the effectiveness of the present invention, an image denoising and enhancement system based on the present invention was deployed at a traffic intersection in a large city. The system consists of optical sensors, infrared sensors and radar sensors, and uses multimodal data acquisition, adaptive partitioning and real-time optimized image processing methods for monitoring. The monitoring area includes a four-way intersection and two main roads, and night monitoring data collection and processing are carried out from 18:00 to 6:00 the next day.

[0127] In practical applications, the system first collects data of different modes through the collaborative work of multiple sensors. Optical sensors capture visible light images of vehicles and pedestrians, infrared sensors record thermal radiation images in low-light scenes, and radar sensors are used to detect vehicle motion trajectories and dynamic speeds. Since these sensors have different sampling accuracy and characteristics, the collected data have differences in spatial resolution and time synchronization. The system divides the collected multimodal data into three categories according to dynamic characteristics: high noise area, low light area, and static area through adaptive partitioning, and gives priority to denoising the high noise area.

[0128] During the image processing process, the system dynamically selects different deep neural network architectures to process each sub-region. For example, for high-noise areas, an optimized convolutional neural network is used to suppress noise; for dynamic motion areas, a recursive neural network is used to analyze target trajectories and enhance image details; for areas with drastic lighting changes, a network based on the attention mechanism is used to correct uneven lighting problems. Combined with a real-time feedback mechanism, the system continuously optimizes network parameters according to the changing characteristics of the sub-regions during operation.

[0129] To verify the beneficial effects of the present invention, we selected data from 15 consecutive days for comparative analysis. The statistical indicators include image signal-to-noise ratio, detail enhancement and target recognition accuracy. Under the same environment, the average signal-to-noise ratio of the monitoring system using the traditional image processing method is 27.5dB, while the average signal-to-noise ratio of the image processed by the present invention reaches 36.8dB, an increase of 33.8%. In terms of target recognition accuracy, the recognition rate of the traditional method in low-light scenes at night is 78.4%, while the recognition rate of the system of the present invention is 92.7%, an increase of 18.3%. In addition, the processing delay of the present invention in dynamic scenes is an average of 18 milliseconds, which is significantly lower than the 35 milliseconds of the traditional method, and the processing efficiency is increased by nearly 50%.

[0130] In practical applications, the present invention has shown significant advantages. At nighttime traffic intersections, the system can clearly identify vehicle license plates, pedestrian outlines, and traffic light status, ensuring that traffic violations such as running red lights and driving against traffic can be accurately captured. At the same time, the effect of image processing is extremely continuous, the boundaries of different sub-areas transition smoothly, and the vehicle movement trajectory is clear and coherent, effectively avoiding the visual abruptness caused by uneven processing in traditional methods.

[0131] Table 1 Image processing comparison data table

[0132]

[0133] Through the analysis of the table data, it can be seen that the present invention has significant advantages over traditional methods in multiple key indicators. In terms of image signal-to-noise ratio, the present invention method has significantly improved by 33.8%, from 27.5dB of the traditional method to 36.8dB, indicating that the present invention has excellent noise suppression capability and greatly enhances the clarity of the image. In terms of target recognition accuracy, the present invention method has improved by 18.3%, especially in dynamic environments and complex scenes, the target recognition capability is more stable and reliable. In terms of processing delay, the present invention shortens the average delay of the traditional method from 35 milliseconds to 18 milliseconds, with an improvement of 48.6%, which significantly improves the real-time processing capability of the system and meets the high-efficiency requirements in dynamic scenes.

[0134] In terms of nighttime recognition accuracy, the present invention also demonstrates strong adaptability, increasing the recognition accuracy from 65.2% of the traditional method to 84.5%, an increase of 29.6%, effectively solving the interference of insufficient nighttime light on image processing. The present invention also achieves significant progress in multi-target detection efficiency, from 12 targets / second of the traditional method to 20 targets / second, an efficiency increase of 66.7%, demonstrating a strong ability to simultaneously process multiple targets in high-density dynamic scenes.

[0135] The present invention significantly improves the effects of image denoising and enhancement through technologies such as multimodal data fusion, adaptive partitioning and dynamic optimization. It not only enhances the image quality, but also excels in target recognition and real-time processing efficiency, fully meeting the high requirements of complex scenes for image processing technology.

[0136] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A real-time image denoising and enhancement method based on deep neural network, characterized in that: The steps include: S1. In the image data acquisition stage, multiple heterogeneous sensors are used to collect image data of different modalities in parallel. During the acquisition process, image data of different modalities work together through sensors to automatically capture scene features with dynamic changes in the environment; S2, dividing the acquired multimodal image data into multiple sub-regions in an adaptive manner, wherein the division criteria are based on environmental factors, and the processing priority and denoising strategy of different regions are determined by evaluating the dynamic characteristics of each sub-region; S3. Through a dynamic selection mechanism, the corresponding deep neural network branches are automatically selected for parallel processing according to the processing priority of each sub-region. The neural network branches include multiple deep neural network architectures, and each architecture has an optimized processing effect on specific noise types and image features; S4. During the processing, the timing characteristics of the image data and the environmental conditions are combined, and an adaptive time window is used to adjust the processing process according to the speed of image changes and real-time feedback; S5, dynamically reconstruct the denoised and enhanced image sub-regions by association, and perform pixel-level step-by-step fusion according to the denoising strength and information integrity of different sub-regions, so that the image enhancement effect is continuous and seamless in spatial and temporal dimensions; S6. Dynamically adjust processing parameters based on real-time feedback of environmental information and image quality assessment, monitor processing effects online through real-time feedback mechanism, and incrementally train and fine-tune the neural network model based on current conditions to maintain continuous optimization of image quality in a dynamic environment; S7. Outputting real-time optimized image data, wherein the image data has image features after multimodal fusion and can ensure stable image clarity and low latency in a dynamic environment.

2. The method for real-time image denoising and enhancement based on deep neural network according to claim 1, characterized in that: The S1 specifically includes: S11, collecting image data of different modalities in parallel through multiple heterogeneous sensors, wherein the sensors include optical sensors, infrared sensors, radar sensors, and sound sensors, wherein each sensor has different working characteristics, sampling accuracy, and adaptability; S12, calculating the correlation between environmental conditions and sensor states in real time according to environmental feedback, and dynamically weighting and allocating image data of different modalities; S13. According to the image change speed and spatial characteristics, the sampling timing and frequency of the sensor are optimized through time series modeling to achieve stable data acquisition and image quality; S14, evaluating the spatiotemporal consistency of image data of different modalities, and performing preprocessing by fusion weights and feedback of sensor states; S15. Estimate the spatiotemporal deviation of different modal data, optimize the fusion of multi-sensor data through adaptive calibration and synchronization mechanism, make data seamlessly connected, and eliminate the error caused by time series asynchrony; S16. Monitor the sensor status in real time, warn of possible failures and abnormal conditions, and automatically adjust the collection strategy based on real-time feedback to select the optimal sensor for data collection.

3. The real-time image denoising and enhancement method based on deep neural network according to claim 1, characterized in that: The S2 specifically includes: S21, adaptively dividing the acquired multimodal image data according to environmental factors, dynamically adjusting the division criteria by real-time analysis of the characteristics of each sub-region to cope with the variability of environmental conditions; S22, dividing the image data into a plurality of sub-regions according to an adaptive division standard, where the division is based on the spatial structure characteristics, texture complexity, time variation characteristics, and dynamic characteristics of each sub-region, so that the processing of each sub-region can adapt to different environmental conditions and image characteristics; S23. By evaluating the dynamic characteristics of each sub-region in real time, taking into account the noise intensity, image change speed, illumination change degree and texture complexity, the processing priority of each sub-region is calculated and dynamically adjusted: Among them, P i is the processing priority of sub-region i, N i is the noise intensity of sub-region i, N max is the maximum noise intensity of all sub-regions in the image, ΔL i is the illumination change of sub-region i, L ref is the illumination reference value, V i is the image change speed of sub-region i, V max is the maximum change speed of all sub-regions in the image, T i is the texture complexity of sub-region i, T max is the maximum texture complexity of all sub-regions in the image, α, β, γ, δ are the weight coefficients of each factor; S24, according to the processing priority P of the sub-region i , assign corresponding denoising strategies to each region, so that different characteristic regions can adopt appropriate denoising methods to avoid the simplification of processing strategies; S25, performing dynamic denoising on each sub-region, giving priority to regions with greater noise intensity, and optimizing image quality by dynamically adjusting denoising intensity in other regions, while balancing processing speed and image clarity; S26, real-time monitoring of the processing progress and effect of each sub-region, dynamically adjusting the division criteria, processing priority and denoising strategy according to the monitoring feedback, and optimizing the overall processing effect of the image; S27. Based on environmental feedback and real-time evaluation of image quality, the region division and denoising strategies are further optimized. During the processing of each sub-region, multiple iterations and parameter fine-tuning are performed to achieve the optimal fusion effect of the image in time and space dimensions.

4. The real-time image denoising and enhancement method based on deep neural network according to claim 1, characterized in that: The S3 specifically includes: S31, according to the processing priority P of each sub-area i , through a dynamic selection mechanism, an adaptive deep neural network branch is selected to process each sub-region independently. The neural network branches include convolutional neural networks, recurrent neural networks, and network architectures based on attention mechanisms. Each network architecture is optimized for different noise types, lighting conditions, and texture complexity; S32, in the processing process of each sub-area, adaptively adjust the parameters of the neural network branch based on the real-time feedback results to adapt to the changes in different environmental conditions; S33, align the spatial and temporal features of the sub-region results after parallel processing, smooth the sub-region boundaries through an adaptive fusion algorithm, and calculate the fusion weight W of each sub-region at position (x, y) and time t. i The calculation formula for (x,y,t) is: Among them, W i (x, y, t) is the fusion weight of sub-region i, E i (x,y) is the edge strength, N i (x,y) is the noise level, ΔT i (x,y) is the time variation, T max is the maximum time variation, λ,μ,ν are the weight coefficients of each factor; S34. Based on the multi-sub-region processing results, the feature weighted fusion method is used to integrate the processing outputs of all sub-regions into a complete image data output, optimize the clarity and noise suppression effect of the multimodal image, and maintain the consistency of the details and texture features of each region.

5. The real-time image denoising and enhancement method based on deep neural network according to claim 1, characterized in that: The S4 specifically includes: S41, based on the time series characteristics of the image data, define an adaptive time window T for each sub-region w (i) Among them, T w (i) is the time window of sub-region i, V i is the image change speed of sub-region i, L i is the current light intensity of sub-region i, L ref is the illumination reference value, N i is the noise level of sub-region i, α, β, γ are weight coefficients; S42, based on the calculated time window T w (i) Dynamically adjust the processing frequency of each sub-region, assign a smaller time window to the sub-region with a faster change rate to optimize the processing frequency, and assign a larger time window to the sub-region with a slower change rate to optimize resource allocation; S43, based on the real-time processing status of each sub-area, monitor the time window T w (i) Applicability: combining real-time feedback to adjust the time window size of each sub-region to adapt to the dynamic changes of regional characteristics, so that the time window always maintains the optimal configuration; S44, based on the adaptive time window T w (i) The sub-region results after dynamic processing are synchronized, and the time series data of each sub-region are aligned through the time window alignment algorithm to eliminate the time difference between the sub-regions and ensure the consistency of the processing results in the time dimension; S45. Combined with the real-time feedback of environmental changes, the time window parameters of all sub-areas are optimized, and the balance between image clarity and processing frequency is refined by adjusting the weight coefficients α, β, and γ.

6. The real-time image denoising and enhancement method based on deep neural network according to claim 1, characterized in that: The S5 specifically includes: S51, based on real-time feedback of environmental changes, monitor the image characteristics of each sub-area to obtain real-time image quality information including signal-to-noise ratio, contrast, illumination change amount and image change speed; S52, using the image quality information, dynamically adjusting the denoising intensity, contrast and brightness enhancement parameters of each sub-region so that the processing of each sub-region matches its real-time characteristics; S53, after each sub-region is subjected to denoising and enhancement processing, a pixel-level dynamic association reconstruction method is used to gradually merge the processing results of each sub-region, and the pixels of each sub-region are integrated to obtain a fused image pixel value P(x, y): Among them, W i (x, y) is the weight coefficient of sub-region i at position (x, y), P i (x, y) is the pixel value of sub-region i at position (x, y), Q i is the denoising strength factor of sub-region i, L i is the current light intensity of sub-region i, L ref is the reference illumination value, SNR i is the signal-to-noise ratio of sub-region i, C i is the contrast of sub-region i, N i is the noise level of sub-region i, ∈ is a small positive number to avoid the denominator being zero, and N is the total number of sub-regions; S54. According to the environmental feedback data, the processing parameters are adaptively optimized, and the image clarity, denoising and enhancement effects are balanced by adjusting the weights of the parameters, so that the processing of each sub-area maintains consistency and stability in a changing environment.

7. The real-time image denoising and enhancement method based on deep neural network according to claim 1, characterized in that: The S6 specifically includes: S61. During the image processing process, each sub-region after denoising and enhancement is monitored in real time, and image characteristic information such as signal-to-noise ratio, contrast, illumination change, and image change speed of each sub-region is collected to generate real-time image quality assessment data; S62, using the real-time image quality assessment data, dynamically optimizing the denoising intensity, contrast and brightness enhancement parameters of each sub-region, so that the processing parameters of each sub-region are adapted to its current image characteristics; S63, after dynamic optimization, processing the image data of each sub-region based on the optimized denoising and enhancement parameters, integrating the processing results by a step-by-step fusion method, achieving spatial and temporal consistency of each sub-region through pixel-level integration, so that the image presents a seamless visual effect between regions; S64, combining environmental feedback data, adaptively adjusting processing parameters, and dynamically balancing denoising intensity, contrast, and brightness enhancement parameters to keep the image clarity and quality of each sub-area stable in a changing environment; S65. According to the feedback data of each sub-region, the parameter weight coefficients used are gradually adjusted to achieve multi-dimensional adaptive optimization.

Images