Image acquisition processing method and device, computer equipment and storage medium
By dynamically adjusting the 3DNR intensity value of the hunting camera and image noise reduction processing, the problems of image adhesion and noise suppression were solved, and high-quality image acquisition was achieved.
Patent Information
- Application Number
- CN202511401516.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-28
- Publication Date
- 2026-02-10
AI Technical Summary
The 3DNR intensity of existing hunting cameras is set to a fixed value, which leads to image adhesion and rough image quality during complex imaging processes, and cannot effectively suppress noise.
By obtaining the initial gain parameter to set the initial noise reduction intensity value, detecting whether the automatic exposure has completed the convergence of the first frame image, and adjusting to the optimal noise reduction intensity value after convergence, image noise reduction processing is performed by combining motion estimation and compensation, temporal filtering and spatial filtering.
It improves image clarity and quality, effectively suppresses noise, avoids image clogging, and enhances image usability.
Smart Images
Figure CN121509828A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image processing, and more particularly to an image acquisition and processing method, apparatus, computer equipment, and storage medium. Background Technology
[0002] In fields such as field surveillance, ecological research, and hunting activities, hunting cameras serve as crucial unattended imaging devices, and their image quality is paramount. Because they frequently operate in outdoor environments with complex and variable lighting conditions, hunting cameras require excellent low-light imaging capabilities and noise suppression. 3DNR (3D noise reduction) technology is a widely adopted and highly efficient image denoising method. Its core principle is to perform temporal filtering on multiple consecutive frames of images, utilizing the correlation of signals in the time dimension to suppress random noise, thereby significantly improving image clarity.
[0003] However, existing hunting cameras typically use a fixed 3DNR intensity setting. This "one-size-fits-all" static strategy reveals significant limitations when facing complex imaging processes. The root cause lies in the irreconcilable contradiction between 3DNR processing and the automatic exposure (AE) convergence process. Specifically, the goal of the automatic exposure module is to quickly adjust sensor parameters (such as gain and shutter speed) to rapidly stabilize image brightness to an ideal level. However, in solutions using specific main control chips, 3DNR and WDR (Wide Dynamic Range) processing reside in different processing domains. When the AE convergence speed is too fast, the instantaneous and drastic changes in data transfer and mapping between different processing modules during the conversion from the RAW domain (the sensor's raw data domain) to the YUV domain (luminance-chrominance separated color domain) can easily trigger image sticking. This sticking visually manifests as irregular blue or black spots in the image, severely impairing image usability.
[0004] To circumvent this inherent hardware challenge, existing technologies often adopt a conservative strategy: setting the 3DNR intensity to a low level at the initial stage of camera startup or scene transition. While this weak noise reduction effectively avoids potential sticking spots during rapid image convergence (AE), it comes at the cost of significant image noise. After AE convergence is complete and image brightness stabilizes, the 3DNR intensity remains at a low initial value, failing to effectively suppress existing noise, resulting in a coarse final image quality with severe loss of detail. Summary of the Invention
[0005] The purpose of this application is to provide an image acquisition and processing method and apparatus to solve the problem that images are easily blurred and have low image quality when a camera acquires images.
[0006] To address the aforementioned technical problems, this application provides an image acquisition and processing method, employing the following technical solution:
[0007] Obtain the initial gain parameters, and set the initial noise reduction intensity value according to the initial gain parameters;
[0008] Automatic exposure is performed based on the initial noise reduction intensity value, and it is detected whether the automatic exposure has completed the convergence of the first frame image;
[0009] If the automatic exposure completes the convergence of the first frame image, then the initial noise reduction intensity value is adjusted to the optimal value to obtain the optimal noise reduction intensity value;
[0010] Image denoising is performed based on the optimal denoising intensity value to obtain an effective target image;
[0011] The valid target image is output.
[0012] Furthermore, the step of obtaining the initial gain parameter and setting the initial noise reduction intensity value based on the initial gain parameter specifically includes:
[0013] Obtain the initial gain parameter from the sensor's parameter dataset;
[0014] The initial noise reduction intensity value is obtained by setting a preset initial value for the 3D noise reduction intensity value based on the initial gain parameter.
[0015] Further, the step of performing automatic exposure based on the initial noise reduction intensity value and detecting whether the automatic exposure has completed the convergence of the first frame image specifically includes:
[0016] Based on the initial noise reduction intensity value, the sensor is controlled to perform automatic exposure processing to obtain the first frame image;
[0017] Obtain the brightness statistics of the first frame image, and calculate the average brightness of the image based on the brightness statistics;
[0018] Determine whether the average brightness of the image is greater than or equal to a preset brightness threshold;
[0019] If the average brightness of the image is greater than or equal to the preset brightness threshold, then the automatic exposure is determined to have completed the convergence of the first frame image.
[0020] Furthermore, the step of adjusting the initial noise reduction intensity value to the optimal value to obtain the optimal noise reduction intensity value specifically includes:
[0021] Retrieve the preset optimal noise reduction intensity value from the database;
[0022] Obtain the noise variance and signal-to-noise ratio of the first frame image, and adjust the initial noise reduction intensity value according to the noise variance, the signal-to-noise ratio, and the preset optimal noise reduction intensity value to obtain an effective noise reduction intensity value;
[0023] Extract ambient light intensity information from the first frame image;
[0024] Obtain a preset intensity mapping table, and fine-tune the effective noise reduction intensity value according to the ambient light intensity information and the preset intensity mapping table to obtain the optimal noise reduction intensity value.
[0025] Further, the step of fine-tuning the effective noise reduction intensity value based on the ambient light intensity information and the preset intensity mapping table to obtain the optimal noise reduction intensity value specifically includes:
[0026] Based on the ambient light intensity information, a matching query is performed in the preset intensity mapping table to obtain the corresponding noise reduction intensity value;
[0027] The effective noise reduction intensity value is adjusted according to the corresponding noise reduction intensity value to obtain the optimal noise reduction intensity value.
[0028] Further, the step of performing image denoising processing based on the optimal denoising intensity value to obtain an effective target image specifically includes:
[0029] Image capture is performed based on the optimal noise reduction intensity value to obtain continuous frame images;
[0030] Inter-frame alignment based on motion estimation and compensation is performed on the consecutive frame images to obtain aligned consecutive frame images;
[0031] The aligned continuous frame images are subjected to time-domain filtering based on weighted average to obtain effective continuous frame images;
[0032] The effective continuous frame image is subjected to Gaussian-based spatial filtering to obtain a smooth continuous frame image;
[0033] Edge texture enhancement is performed on the smooth continuous frame image to obtain the effective target image.
[0034] Furthermore, the step of performing inter-frame alignment based on motion estimation and compensation on the consecutive frame images to obtain aligned consecutive frame images specifically includes:
[0035] Obtain the image pixel information of the consecutive frames;
[0036] Based on the motion estimation algorithm, the displacement vector between images is calculated according to the image pixel information;
[0037] Motion compensation is performed on the consecutive frame images based on the displacement vector to obtain the aligned consecutive frame images.
[0038] To address the aforementioned technical problems, this application also provides an image acquisition and processing device, which employs the following technical solution:
[0039] The intensity value setting module is used to obtain the initial gain parameter and set the initial noise reduction intensity value according to the initial gain parameter;
[0040] An exposure detection module is used to perform automatic exposure based on the initial noise reduction intensity value, and to detect whether the automatic exposure has completed the convergence of the first frame image;
[0041] The intensity value adjustment module is used to adjust the initial noise reduction intensity value to the optimal value to obtain the optimal noise reduction intensity value if the automatic exposure completes the convergence of the first frame image.
[0042] An image denoising module is used to perform image denoising processing based on the optimal denoising intensity value to obtain an effective target image;
[0043] The image output module is used to output the effective target image.
[0044] To address the aforementioned technical problems, this application also provides a computer device that employs the following technical solution:
[0045] A computer device includes a memory and a processor, the memory storing computer-readable instructions, the processor executing the computer-readable instructions to implement the steps of the image acquisition and processing method as described in any of the preceding claims.
[0046] To address the aforementioned technical problems, this application also provides a computer-readable storage medium, employing the technical solution described below:
[0047] A computer-readable storage medium storing computer-readable instructions, which, when executed by a processor, implement the steps of the image acquisition and processing method described in any of the preceding claims.
[0048] Compared with the prior art, the embodiments of this application have the following main advantages:
[0049] This embodiment obtains an initial gain parameter, sets an initial noise reduction intensity value based on the initial gain parameter, performs automatic exposure based on the initial noise reduction intensity value, and detects whether the automatic exposure has completed the convergence of the first frame image. If the automatic exposure has completed the convergence of the first frame image, the initial noise reduction intensity value is adjusted to the optimal value to obtain the optimal noise reduction intensity value. Image noise reduction processing is performed based on the optimal noise reduction intensity value to obtain an effective target image. The effective target image is then output. This effectively improves the clarity and quality of the acquired image. Attached Figure Description
[0050] To more clearly illustrate the solutions in this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0051] Figure 1 A flowchart of an embodiment of the image acquisition and processing method according to this application;
[0052] Figure 2 This is a schematic diagram of the structure of an embodiment of the image acquisition and processing apparatus according to this application;
[0053] Figure 3 This is a schematic diagram of the structure of one embodiment of the computer device according to this application. Detailed Implementation
[0054] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein in the specification of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having," and any variations thereof, in the specification, claims, and foregoing drawings of this application, are intended to cover non-exclusive inclusion. The terms "first," "second," etc., in the specification, claims, or foregoing drawings of this application are used to distinguish different objects, not to describe a particular order.
[0055] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.
[0056] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.
[0057] Continue to refer to Figure 1 A flowchart of an embodiment of the image acquisition and processing method according to this application is shown. The image acquisition and processing method includes the following steps:
[0058] Step S10: Obtain the initial gain parameters and set the initial noise reduction intensity value according to the initial gain parameters;
[0059] In this embodiment, after the camera is powered on, the system initializes. During this stage, the image signal processor (ISP) sets the 3DNR intensity (3D noise reduction intensity) to a low level (e.g., 10%), and the set 3DNR intensity is the initial noise reduction intensity value.
[0060] Step S20: Perform automatic exposure based on the initial noise reduction intensity value, and detect whether the automatic exposure has completed the convergence of the first frame image;
[0061] In this embodiment, first frame image convergence refers to the AE convergence of the first frame image. The automatic exposure (AE) system needs to quickly adjust the shutter speed, aperture, and sensor gain (ISO) to obtain an image with appropriate brightness; this process is called AE convergence. The system needs to know when AE convergence is complete. Convergence is detected when the image brightness value (or average pixel value) has reached the preset target brightness and remains stable for several consecutive frames without significant fluctuations. Once the system detects that the first frame exposure parameters are stable and the image brightness meets the target, it determines that the AE of the first frame image has completed its first convergence.
[0062] Step S30: If the automatic exposure completes the convergence of the first frame image, then the initial noise reduction intensity value is adjusted to the optimal value to obtain the optimal noise reduction intensity value.
[0063] In this embodiment, upon detecting AE convergence, the system immediately increases the 3DNR intensity from its initial low level to a preset "optimal value" (e.g., 80%) via software commands, forming the optimal noise reduction intensity value. Increasing the 3DNR intensity maximizes the image noise reduction effect, resulting in a cleaner and higher-quality image while maintaining image clarity. The optimal value is the intensity value that best balances "noise reduction effect" and "detail preservation."
[0064] Step S40: If the automatic exposure has not completed the convergence of the first frame image, then the automatic exposure is re-executed according to the initial noise reduction intensity value until the automatic exposure has completed the convergence of the first frame image.
[0065] In this embodiment, if the automatic exposure fails to achieve AE convergence for the first frame image, the system will re-perform the AE adjustment operation, re-acquire the first frame image, and check whether AE convergence has been completed after this adjustment operation. This process is repeated continuously until AE convergence for the first frame image is completed during automatic exposure. Specifically, the number of loops can be set. When the set number of loops is reached, the looping operation stops, and an automatic exposure failure message is generated and displayed. In this embodiment, the initial number of loops can be set to 5, and can be adjusted according to actual conditions.
[0066] Step S50: Perform image denoising processing based on the optimal denoising intensity value to obtain an effective target image;
[0067] In this embodiment, after the 3DNR intensity is enhanced to the optimal value, the camera continues to operate normally. The system controls the AE module to continuously fine-tune to maintain stable brightness, and all image frames undergo strong 3DNR processing. AE convergence and 3DNR adjustment are two relatively independent processes. Changes in 3DNR intensity do not directly affect how the AE calculates exposure; the goal of the AE is to control brightness, while the goal of the 3DNR is to process noise.
[0068] Step S60: Output the effective target image.
[0069] In this embodiment, after complete AE exposure and 3DNR noise reduction processing, a clear and effective target image is finally generated and compressed and stored in the camera's memory card.
[0070] This embodiment obtains an initial gain parameter, sets an initial noise reduction intensity value based on the initial gain parameter, performs automatic exposure based on the initial noise reduction intensity value, and detects whether the automatic exposure has completed the convergence of the first frame image. If the automatic exposure has completed the convergence of the first frame image, the initial noise reduction intensity value is adjusted to the optimal value to obtain the optimal noise reduction intensity value. Image noise reduction processing is performed based on the optimal noise reduction intensity value to obtain an effective target image. The effective target image is then output. This effectively improves the clarity and quality of the acquired image.
[0071] In some optional implementations of this embodiment, obtaining the initial gain parameter and setting the initial noise reduction intensity value based on the initial gain parameter includes the following steps:
[0072] Obtain the initial gain parameter from the sensor's parameter dataset;
[0073] In this embodiment, the sensor parameter dataset is a pre-configured database or configuration file that stores key parameters of the camera image sensor in various operating modes. The initial gain parameter refers to a default, safe starting ISO value (sensor gain) used by the camera after startup and before the automatic exposure algorithm begins calculation. In this embodiment, the initial gain parameter is set to 100, but can be set and adjusted according to actual conditions.
[0074] The initial noise reduction intensity value is obtained by setting a preset initial value for the 3D noise reduction intensity value based on the initial gain parameter.
[0075] In this embodiment, the system calculates the initial value of the 3D noise reduction intensity based on the initial gain parameter and a preset rule or mapping relationship. For example, if the initial gain parameter is a low ISO (e.g., 100), it indicates that the environment may be bright and the expected noise is low, so the system will set the initial intensity of 3DNR to a lower value (e.g., 20%). The preset mapping relationship can be expressed by the mapping formula: Initial Noise Reduction Intensity = A * log2(ISO / Base_ISO) + B; where ISO is the initial gain parameter; Base_ISO is the sensor's reference ISO (the native minimum ISO, e.g., 100); log2 is a logarithmic function with base 2. Because ISO values are usually multiples (100, 200, 400, 800...), using logarithms can convert them into a linear relationship; A is a scaling factor, controlling the rate at which the 3DNR intensity increases with ISO; and B is the base offset, representing the initial 3DNR intensity at the reference ISO. For example, setting Base_ISO = 100, A = 15, and B = 5. When ISO = 100, the initial strength = 15 * log2(100 / 100) + 5 = 15 * 0 + 5 = 5%; when ISO = 800, the initial strength = 15 * log2(800 / 100) + 5 = 15 * 3 + 5 = 50%.
[0076] This embodiment obtains the initial gain parameter from the sensor's parameter dataset; and sets a preset initial value for the 3D noise reduction intensity value based on the initial gain parameter to obtain the initial noise reduction intensity value. This effectively sets the initial noise reduction intensity value to avoid adhesion spots during rapid AE (automatic exposure) convergence, facilitating subsequent automatic exposure processing.
[0077] In some optional implementations of this embodiment, the step of performing automatic exposure based on the initial noise reduction intensity value and detecting whether the automatic exposure has completed the convergence of the first frame image includes the following steps:
[0078] Based on the initial noise reduction intensity value, the sensor is controlled to perform automatic exposure processing to obtain the first frame image;
[0079] In this embodiment, the camera system uses an initial noise reduction intensity value to control the image sensor to perform the first full exposure and capture the first frame image. The system sets an initial shutter speed and aperture based on the built-in metering mode, and combines this with the sensor gain to capture the first frame image.
[0080] Obtain the brightness statistics of the first frame image, and calculate the average brightness of the image based on the brightness statistics;
[0081] In this embodiment, the image signal processor in the system counts the brightness values of all pixels in the entire image or a specific area and calculates an average brightness value. The average image brightness value is calculated by adding the brightness values of all pixels (e.g., within the range of 0-255) and dividing by the total number of pixels.
[0082] Determine whether the average brightness of the image is greater than or equal to a preset brightness threshold;
[0083] In this embodiment, the preset brightness threshold is a preset target value used for conditional judgment. This threshold can be determined experimentally and fixed in the system (e.g., the brightness value corresponding to an 18% gray card).
[0084] If the average brightness of the image is greater than or equal to the preset brightness threshold, then it is determined that the automatic exposure has completed the convergence of the first frame image.
[0085] In this embodiment, if the comparison result shows that the average image brightness is greater than or equal to a preset brightness threshold, the system will determine that the automatic exposure has completed convergence based on the first frame image. Convergence means that the automatic exposure system has pushed the image brightness to an acceptable range through the first attempt. The above determination is a trigger signal; once the system confirms that the AE has completed the first frame exposure convergence, it will immediately execute the next key action.
[0086] If the average brightness of the image is less than the preset brightness threshold, it is determined that the automatic exposure has not completed the convergence of the first frame image.
[0087] In this embodiment, if the comparison result is that the average image brightness is less than the preset brightness threshold, the system will determine that the automatic exposure has not completed the convergence based on the first frame image.
[0088] This embodiment controls the sensor to perform automatic exposure processing based on the initial noise reduction intensity value to obtain a first frame image; acquires the brightness statistics of the first frame image, calculates the average image brightness based on the brightness statistics, and determines whether the average image brightness is greater than or equal to a preset brightness threshold; if the average image brightness is greater than or equal to the preset brightness threshold, it is determined that the automatic exposure has completed the convergence of the first frame image; if the average image brightness is less than the preset brightness threshold, it is determined that the automatic exposure has not completed the convergence of the first frame image. This effectively detects whether the first frame image of the automatic exposure has converged, facilitating subsequent adjustment of the initial noise reduction intensity value.
[0089] In some optional implementations of this embodiment, adjusting the initial noise reduction intensity value to the optimal value to obtain the optimal noise reduction intensity value includes the following steps:
[0090] Retrieve the preset optimal noise reduction intensity value from the database;
[0091] In this embodiment, the system queries and reads a preset optimal noise reduction intensity value from a built-in database. This value is a pre-determined and set universal optimal value. It has been determined through extensive experimental testing (evaluated by indicators such as PSNR and SSIM under different lighting conditions and scenes) and can balance noise reduction effect and detail preservation in most scenarios. In this embodiment, the preset optimal noise reduction intensity value is set to 80%, which can be adjusted accordingly based on actual conditions.
[0092] Obtain the noise variance and signal-to-noise ratio of the first frame image, and adjust the initial noise reduction intensity value according to the noise variance, the signal-to-noise ratio, and the preset optimal noise reduction intensity value to obtain an effective noise reduction intensity value;
[0093] In this embodiment, the system performs real-time analysis on the first frame image, calculating two key indicators: noise variance (representing the strength and dispersion of noise) and signal-to-noise ratio (SNR) (representing the ratio of useful information to noise in the signal). An adjustment coefficient is then determined based on the noise variance and SNR. This adjustment coefficient, along with a preset optimal noise reduction strength value, is used to calculate the adjusted value. The relationship between the adjustment coefficients and the noise variance and signal-to-noise ratio (SNR) is as follows: when the noise variance is 0-50 and the SNR is >30dB, the adjustment coefficient is 0.7-0.8; when the noise variance is 50-100 and the SNR is 25-30dB, the adjustment coefficient is 0.8-0.9; when the noise variance is 100-200 and the SNR is 20-25dB, the adjustment coefficient is 1; when the noise variance is 200-400 and the SNR is 15-20dB, the adjustment coefficient is 1.1-1.2; and when the noise variance is >400 and the SNR is <15dB, the adjustment coefficient is 1.2-1.3. After determining the adjustment coefficients, the effective noise reduction intensity value is obtained by multiplying the adjustment coefficients by the preset optimal noise reduction intensity value.
[0094] Extract ambient light intensity information from the first frame image;
[0095] In this embodiment, the ambient light intensity can be obtained by comprehensively considering parameters such as the average brightness of the image, sensor gain, and exposure time. The ambient light intensity is calculated using the following formula: Lux = K × (Average Image Brightness) / (Exposure Time × ISO Gain × Aperture Coefficient) 2 ); where Lux is the ambient light intensity, K is the system calibration constant, initially set to 1, and can be adjusted according to the actual situation, and the aperture coefficient is the square of the F value (e.g., the aperture coefficient of F2.8 is 2.8 = 7.84).
[0096] Obtain a preset intensity mapping table, and fine-tune the effective noise reduction intensity value according to the ambient light intensity information and the preset intensity mapping table to obtain the optimal noise reduction intensity value.
[0097] In this embodiment, the system reads a preset intensity mapping table of "scene-optimal 3DNR intensity". Then, using the extracted ambient light intensity information as input, it queries this mapping table to find a corresponding, more refined 3DNR intensity suggestion value, which is the effective noise reduction intensity value.
[0098] This embodiment obtains a preset optimal noise reduction intensity value from a database; acquires the noise variance and signal-to-noise ratio of the first frame image, and adjusts the initial noise reduction intensity value according to the noise variance, the signal-to-noise ratio, and the preset optimal noise reduction intensity value to obtain an effective noise reduction intensity value; extracts ambient light intensity information from the first frame image; obtains a preset intensity mapping table, and fine-tunes the effective noise reduction intensity value according to the ambient light intensity information and the preset intensity mapping table to obtain the optimal noise reduction intensity value. This effectively adjusts the initial noise reduction intensity value to the most suitable optimal noise reduction intensity value, facilitating subsequent image noise reduction processing based on the optimal noise reduction intensity value.
[0099] In some optional implementations of this embodiment, the step of fine-tuning the effective noise reduction intensity value based on the ambient light intensity information and the preset intensity mapping table to obtain the optimal noise reduction intensity value includes the following steps:
[0100] Based on the ambient light intensity information, a matching query is performed in the preset intensity mapping table to obtain the corresponding noise reduction intensity value;
[0101] In this embodiment, ambient light intensity information is used as a query condition to search and match in the database storing the data, so as to return a corresponding noise reduction intensity value that precisely corresponds to the lighting conditions of the ambient light intensity information.
[0102] The effective noise reduction intensity value is adjusted according to the corresponding noise reduction intensity value to obtain the optimal noise reduction intensity value.
[0103] In this embodiment, the effective noise reduction intensity value is a value derived based on a general preset and the noise conditions of the first frame. The optimal noise reduction intensity value can be calculated using the following formula: Optimal noise reduction intensity value = (effective noise reduction intensity value * A) + (corresponding noise reduction intensity value * B), where A is preset to 0.3 and B is preset to 0.7. These values can be adjusted according to actual conditions.
[0104] This embodiment obtains the corresponding noise reduction intensity value by matching and querying the ambient light intensity information in the preset intensity mapping table; then, it adjusts the effective noise reduction intensity value according to the corresponding noise reduction intensity value to obtain the optimal noise reduction intensity value. This effectively achieves the adjustment of the effective noise reduction intensity based on the ambient light intensity of the acquired image to obtain the optimal noise reduction intensity value.
[0105] In some optional implementations of this embodiment, the step of performing image denoising processing based on the optimal denoising intensity value to obtain an effective target image includes the following steps:
[0106] Image capture is performed based on the optimal noise reduction intensity value to obtain continuous frame images;
[0107] In this embodiment, the optimal noise reduction intensity value is used as a control parameter to guide the image acquisition process. Then, the sensor continuously captures multiple original images (e.g., 5-7 frames) at a certain frame rate, forming a set of consecutive frame images. Acquiring images based on the optimal noise reduction intensity value does not mean that this value directly controls the shutter speed, but rather that the system has entered a strong noise reduction mode, and all subsequent processing will be based on this optimized intensity.
[0108] Inter-frame alignment based on motion estimation and compensation is performed on the consecutive frame images to obtain aligned consecutive frame images;
[0109] In this embodiment, pixel displacement caused by slight camera shake or object movement between frames is detected by analyzing consecutive frame images. Then, an algorithm "moves" the pixels of subsequent frames to precisely align them with the reference frame (the first frame) in terms of scene content. Motion estimation and compensation are key preliminary steps in the 3DNR algorithm. This effectively ensures that during subsequent temporal filtering, each pixel is compared with information from the same scene location, thus averaging only the noise without blurring the image content.
[0110] The aligned continuous frame images are subjected to time-domain filtering based on weighted average to obtain effective continuous frame images;
[0111] In this embodiment, for each pixel position (x, y) in the aligned consecutive frame images, its value in the consecutive frame sequence is taken, and then a weighted average is calculated. The result of this weighted average is the value of that pixel in the new frame. Temporal filtering is the core noise reduction step of 3DNR. Random noise fluctuates in the temporal domain (i.e., across frames), while the real scene signal is stable. After averaging multiple frames, the noise is suppressed due to "positive and negative cancellation," while the scene signal is enhanced.
[0112] The effective continuous frame image is subjected to Gaussian-based spatial filtering to obtain a smooth continuous frame image;
[0113] In this embodiment, Gaussian filtering is applied to each frame image after temporal denoising. Gaussian filtering is a smoothing algorithm that observes the relationship between each pixel within a single frame and its surrounding pixels (e.g., a 3x3 or 5x5 area) and assigns higher weights to pixels closer to the center. Spatial filtering utilizes the spatial characteristic that adjacent pixels in an image typically have similar values to smooth out residual noise within a single frame image, making uniform areas (such as the sky or walls) cleaner and smoother.
[0114] Edge texture enhancement is performed on the smooth continuous frame image to obtain the effective target image.
[0115] In this embodiment, after temporal and spatial denoising, the smooth continuous frame image is used to selectively enhance the edge and texture details of objects in the image using edge detection and sharpening algorithms (such as UnsharpMask) to obtain the processed effective target image.
[0116] This embodiment acquires consecutive frame images by capturing images according to the optimal noise reduction intensity value; performs inter-frame alignment based on motion estimation and compensation on the consecutive frame images to obtain aligned consecutive frame images; applies weighted average-based temporal filtering to the aligned consecutive frame images to obtain effective consecutive frame images; applies Gaussian-based spatial filtering to the effective consecutive frame images to obtain smoothed consecutive frame images; and enhances the edge texture of the smoothed consecutive frame images to obtain the effective target image. This effectively achieves the acquisition of a set of consecutive frame images based on the optimal noise reduction intensity and processing them into high-quality effective target images.
[0117] In some optional implementations of this embodiment, the step of performing inter-frame alignment based on motion estimation and compensation on the consecutive frame images to obtain aligned consecutive frame images includes the following steps:
[0118] Obtain the image pixel information of the consecutive frames;
[0119] In this embodiment, the system reads the original image pixel information from multiple consecutively captured images, including basic information such as the brightness value, color value, and pixel coordinate information of each pixel.
[0120] Based on the motion estimation algorithm, the displacement vector between images is calculated according to the image pixel information;
[0121] In this embodiment, calculating the displacement vector based on image pixel information includes the following steps: Block processing: dividing the current frame into multiple macroblocks (e.g., 16×16 pixels); Search matching: within a search window around each macroblock, finding the most similar pixel block in the reference frame; Vector calculation: after finding the best matching block, calculating the displacement difference between the two blocks. The format for calculating the displacement vector is (Δx, Δy), where Δx represents the horizontal displacement and Δy represents the vertical displacement. For example, the displacement vector (2, -5) indicates that the image pixel moves 2 pixels to the right and 5 pixels upward.
[0122] Motion compensation is performed on the consecutive frame images based on the displacement vector to obtain the aligned consecutive frame images.
[0123] In this embodiment, motion compensation for consecutive frame images includes reverse translation: based on the displacement vector, each pixel block is moved back to its corresponding position in the reference frame; pixel resampling: since non-integer pixel positions may be generated after the movement, interpolation calculation (such as bilinear interpolation) is required; and hole filling: any blank areas that may appear after compensation are filled with adjacent pixels. By performing motion compensation on consecutive frame images including the above steps, aligned consecutive frame images are effectively obtained.
[0124] This embodiment obtains the image pixel information of the consecutive frame images; calculates the displacement vector between images based on the image pixel information using a motion estimation algorithm; and performs motion compensation on the consecutive frame images based on the displacement vector to obtain the aligned consecutive frame images. This effectively achieves alignment processing of consecutive frame images to obtain corresponding aligned consecutive frame images, facilitating subsequent temporal filtering processing.
[0125] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing related hardware through computer-readable instructions. These computer-readable instructions can be stored in a computer-readable storage medium. When executed, the program can include the processes of the embodiments of the above methods. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, optical disk, or read-only memory (ROM), or random access memory (RAM).
[0126] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.
[0127] Further reference Figure 2 As a response to the above Figure 1 The implementation of the method shown in this application provides an embodiment of an image acquisition and processing device, which is similar to... Figure 1 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.
[0128] like Figure 2As shown, the image acquisition and processing device 700 described in this embodiment includes: an intensity value setting module 701, an exposure detection module 702, an intensity value adjustment module 703, an exposure processing module 704, an image noise reduction module 705, and an image output module 706. Wherein:
[0129] The intensity value setting module 701 is used to obtain the initial gain parameter and set the initial noise reduction intensity value according to the initial gain parameter.
[0130] The exposure detection module 702 is used to perform automatic exposure based on the initial noise reduction intensity value, and to detect whether the automatic exposure has completed the convergence of the first frame image;
[0131] The intensity value adjustment module 703 is used to adjust the initial noise reduction intensity value to the optimal value to obtain the optimal noise reduction intensity value if the automatic exposure completes the convergence of the first frame image.
[0132] The exposure processing module 704 is used to re-execute automatic exposure according to the initial noise reduction intensity value if the automatic exposure fails to converge the first frame image, until the automatic exposure completes the convergence of the first frame image.
[0133] The image denoising module 705 is used to perform image denoising processing based on the optimal denoising intensity value to obtain an effective target image;
[0134] The image output module 706 is used to output the effective target image.
[0135] This embodiment employs an image acquisition and processing device 700 including the aforementioned modules. It can acquire initial gain parameters, set an initial noise reduction intensity value based on these parameters, perform automatic exposure based on the initial noise reduction intensity value, and detect whether the automatic exposure has completed the convergence of the first frame image. If the automatic exposure has completed the convergence of the first frame image, the initial noise reduction intensity value is adjusted to an optimal value to obtain the optimal noise reduction intensity value. Image noise reduction processing is performed based on the optimal noise reduction intensity value to obtain a valid target image. The valid target image is then output. This effectively improves the clarity and quality of the acquired image.
[0136] In some optional implementations of this embodiment, the intensity value setting module 701 includes: a parameter acquisition unit and an intensity value setting unit. Wherein:
[0137] The parameter acquisition unit is used to acquire the initial gain parameter from the parameter dataset of the sensor;
[0138] The intensity value setting unit is used to set a preset initial value for the 3D noise reduction intensity value according to the initial gain parameter, so as to obtain the initial noise reduction intensity value.
[0139] This embodiment effectively sets the initial noise reduction intensity value by setting an intensity value setting module 701, which includes a parameter acquisition unit and an intensity value setting unit, so as to avoid sticking spots when AE (automatic exposure) converges quickly, and facilitate subsequent automatic exposure processing.
[0140] In some optional implementations of this embodiment, the exposure detection module 702 includes: an exposure processing unit, a brightness statistics unit, a threshold judgment unit, a first determination unit, and a second determination unit. Wherein:
[0141] The exposure processing unit is used to control the sensor to perform automatic exposure processing based on the initial noise reduction intensity value to obtain the first frame image;
[0142] The brightness statistics unit is used to obtain brightness statistics data of the first frame image and calculate the average brightness of the image based on the brightness statistics data.
[0143] The threshold determination unit is used to determine whether the average brightness of the image is greater than or equal to a preset brightness threshold.
[0144] The first determining unit is configured to determine that the automatic exposure has completed the convergence of the first frame image if the average brightness of the image is greater than or equal to the preset brightness threshold.
[0145] The second determining unit is used to determine that the automatic exposure has not completed the convergence of the first frame image if the average brightness of the image is less than the preset brightness threshold.
[0146] This embodiment effectively detects whether the first frame of automatically exposed image has converged by setting an exposure detection module 702, which includes an exposure processing unit, a brightness statistics unit, a threshold judgment unit, a first determination unit, and a second determination unit, so as to facilitate the subsequent adjustment of the initial noise reduction intensity value.
[0147] In some optional implementations of this embodiment, the intensity value adjustment module 703 includes: a segmentation unit, a frequency domain conversion unit, and a signal filtering unit. Wherein:
[0148] The intensity value acquisition unit is used to obtain a preset optimal noise reduction intensity value from the database;
[0149] The intensity value adjustment unit is used to obtain the noise variance and signal-to-noise ratio of the first frame image, and adjust the initial noise reduction intensity value according to the noise variance, the signal-to-noise ratio, and the preset optimal noise reduction intensity value to obtain an effective noise reduction intensity value.
[0150] The intensity information extraction unit is used to extract ambient light intensity information from the first frame image;
[0151] The intensity value fine-tuning unit is used to obtain a preset intensity mapping table, and to fine-tune the effective noise reduction intensity value according to the ambient light intensity information and the preset intensity mapping table to obtain the optimal noise reduction intensity value.
[0152] This embodiment effectively adjusts the initial noise reduction intensity value to the most suitable optimal noise reduction intensity value by setting an intensity value adjustment module 703 that includes a segmentation unit, a frequency domain conversion unit, and a signal filtering unit, so as to facilitate subsequent image noise reduction processing based on the optimal noise reduction intensity value.
[0153] In some optional implementations of this embodiment, the intensity value fine-tuning unit includes: a matching query subunit and an intensity adjustment subunit. Wherein:
[0154] The matching query subunit is used to perform a matching query in the preset intensity mapping table based on the ambient light intensity information to obtain the corresponding noise reduction intensity value;
[0155] The intensity adjustment subunit is used to adjust the effective noise reduction intensity value according to the corresponding noise reduction intensity value to obtain the optimal noise reduction intensity value.
[0156] This embodiment effectively adjusts the effective noise reduction intensity based on the ambient light intensity of the acquired image by setting an intensity value fine-tuning unit that includes a matching query subunit and an intensity adjustment subunit, so as to obtain the optimal noise reduction intensity value.
[0157] In some optional implementations of this embodiment, the image noise reduction module 705 includes: an image acquisition unit, an alignment processing unit, a temporal filtering unit, a spatial filtering unit, and an edge enhancement unit. Wherein:
[0158] The image acquisition unit is used to capture images according to the optimal noise reduction intensity value to obtain continuous frame images;
[0159] The alignment processing unit is used to perform inter-frame alignment based on motion estimation and compensation on the continuous frame images to obtain aligned continuous frame images.
[0160] The temporal filtering unit is used to perform weighted average-based temporal filtering on the aligned continuous frame image to obtain an effective continuous frame image.
[0161] The spatial filtering unit is used to perform Gaussian-based spatial filtering on the effective continuous frame image to obtain a smooth continuous frame image.
[0162] The edge enhancement unit is used to enhance the edge texture of the smooth continuous frame image to obtain the effective target image.
[0163] This embodiment effectively achieves the acquisition of a set of consecutive frames of images based on the optimal noise reduction intensity and the processing into a high-quality effective target image by setting up an image denoising module 705 including an image acquisition unit, an alignment processing unit, a temporal filtering unit, a spatial filtering unit, and an edge enhancement unit.
[0164] In some optional implementations of this embodiment, the alignment processing unit includes: a pixel acquisition subunit, a displacement calculation subunit, and an image compensation subunit. Wherein:
[0165] The pixel acquisition subunit is used to acquire the image pixel information of the consecutive frame images;
[0166] The displacement calculation subunit is used to calculate the displacement vector between images based on the image pixel information using a motion estimation algorithm.
[0167] The image compensation subunit is used to perform motion compensation on the continuous frame images according to the displacement vector to obtain the aligned continuous frame images.
[0168] This embodiment effectively aligns consecutive frame images by setting up an alignment processing unit that includes a pixel acquisition subunit, a displacement calculation subunit, and an image compensation subunit, so as to obtain the corresponding aligned consecutive frame images, which facilitates subsequent temporal filtering processing.
[0169] To address the aforementioned technical problems, embodiments of this application also provide a computer device. Please refer to [link / reference needed]. Figure 3 , Figure 3 This is a basic structural block diagram of the computer device in this embodiment.
[0170] The computer device 8 includes a memory 81, a processor 82, and a network interface 83 that are interconnected via a system bus. It should be noted that only the computer device 8 with components 81-83 is shown in the figure; however, it should be understood that it is not required to implement all the shown components, and more or fewer components can be implemented alternatively. Those skilled in the art will understand that the computer device described here is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.
[0171] The computer device can be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer device can interact with the user via a keyboard, mouse, remote control, touchpad, or voice control.
[0172] The memory 81 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 81 may be an internal storage unit of the computer device 8, such as the hard disk or memory of the computer device 8. In other embodiments, the memory 81 may also be an external storage device of the computer device 8, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the computer device 8. Of course, the memory 81 may also include both the internal storage unit and its external storage device of the computer device 8. In this embodiment, the memory 81 is typically used to store the operating system and various application software installed on the computer device 8, such as computer-readable instructions for image acquisition and processing methods. In addition, the memory 81 can also be used to temporarily store various types of data that have been output or will be output.
[0173] In some embodiments, the processor 82 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor 82 is typically used to control the overall operation of the computer device 8. In this embodiment, the processor 82 is used to execute computer-readable instructions stored in the memory 81 or to process data, for example, to execute computer-readable instructions for the image acquisition and processing method.
[0174] The network interface 83 may include a wireless network interface or a wired network interface, which is typically used to establish communication connections between the computer device 8 and other electronic devices.
[0175] This embodiment, by employing the aforementioned computer equipment, can acquire initial gain parameters, set an initial noise reduction intensity value based on these parameters, perform automatic exposure based on the initial noise reduction intensity value, and detect whether the automatic exposure has completed the convergence of the first frame image. If the automatic exposure has completed the convergence of the first frame image, the initial noise reduction intensity value is adjusted to an optimal value to obtain the optimal noise reduction intensity value. Image noise reduction processing is performed based on the optimal noise reduction intensity value to obtain an effective target image. The effective target image is then output. This effectively improves the clarity and quality of the acquired image.
[0176] This application also provides another embodiment, namely, providing a computer-readable storage medium storing computer-readable instructions that can be executed by at least one processor to cause the at least one processor to perform the steps of the image acquisition and processing method described above.
[0177] This embodiment, by employing the aforementioned computer-readable storage medium, can acquire initial gain parameters, set an initial noise reduction intensity value based on the initial gain parameters, perform automatic exposure based on the initial noise reduction intensity value, and detect whether the automatic exposure has completed the convergence of the first frame image. If the automatic exposure has completed the convergence of the first frame image, the initial noise reduction intensity value is adjusted to an optimal value to obtain an optimal noise reduction intensity value. Image noise reduction processing is performed based on the optimal noise reduction intensity value to obtain an effective target image. The effective target image is then output. This effectively improves the clarity and quality of the acquired image.
[0178] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.
[0179] Obviously, the embodiments described above are only some embodiments of this application, not all embodiments. The accompanying drawings show preferred embodiments of this application, but do not limit the patent scope of this application. This application can be implemented in many different forms; rather, the purpose of providing these embodiments is to provide a more thorough and comprehensive understanding of the disclosure of this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or make equivalent substitutions for some of the technical features. Any equivalent structures made using the content of this application's specification and drawings, directly or indirectly applied to other related technical fields, are similarly within the scope of patent protection of this application.
Claims
1. An image acquisition and processing method, characterized in that, include: Obtain the initial gain parameters, and set the initial noise reduction intensity value according to the initial gain parameters; Automatic exposure is performed based on the initial noise reduction intensity value, and it is detected whether the automatic exposure has completed the convergence of the first frame image; If the automatic exposure completes the convergence of the first frame image, then the initial noise reduction intensity value is adjusted to the optimal value to obtain the optimal noise reduction intensity value; Image denoising is performed based on the optimal denoising intensity value to obtain an effective target image; The valid target image is output.
2. The image acquisition and processing method according to claim 1, characterized in that, The step of obtaining the initial gain parameter and setting the initial noise reduction intensity value based on the initial gain parameter specifically includes: Obtain the initial gain parameter from the sensor's parameter dataset; The initial noise reduction intensity value is obtained by setting a preset initial value for the 3D noise reduction intensity value based on the initial gain parameter.
3. The image acquisition and processing method according to claim 1, characterized in that, The step of performing automatic exposure based on the initial noise reduction intensity value and detecting whether the automatic exposure has completed the convergence of the first frame image specifically includes: Based on the initial noise reduction intensity value, the sensor is controlled to perform automatic exposure processing to obtain the first frame image; Obtain the brightness statistics of the first frame image, and calculate the average brightness of the image based on the brightness statistics; Determine whether the average brightness of the image is greater than or equal to a preset brightness threshold; If the average brightness of the image is greater than or equal to the preset brightness threshold, then the automatic exposure is determined to have completed the convergence of the first frame image.
4. The image acquisition and processing method according to claim 1, characterized in that, The step of adjusting the initial noise reduction intensity value to the optimal value to obtain the optimal noise reduction intensity value specifically includes: Retrieve the preset optimal noise reduction intensity value from the database; Obtain the noise variance and signal-to-noise ratio of the first frame image, and adjust the initial noise reduction intensity value according to the noise variance, the signal-to-noise ratio, and the preset optimal noise reduction intensity value to obtain an effective noise reduction intensity value; Extract ambient light intensity information from the first frame image; Obtain a preset intensity mapping table, and fine-tune the effective noise reduction intensity value according to the ambient light intensity information and the preset intensity mapping table to obtain the optimal noise reduction intensity value.
5. The image acquisition and processing method according to claim 4, characterized in that, The step of fine-tuning the effective noise reduction intensity value based on the ambient light intensity information and the preset intensity mapping table to obtain the optimal noise reduction intensity value specifically includes: Based on the ambient light intensity information, a matching query is performed in the preset intensity mapping table to obtain the corresponding noise reduction intensity value; The effective noise reduction intensity value is adjusted according to the corresponding noise reduction intensity value to obtain the optimal noise reduction intensity value.
6. The image acquisition and processing method according to claim 1, characterized in that, The step of performing image denoising processing based on the optimal denoising intensity value to obtain an effective target image specifically includes: Image capture is performed based on the optimal noise reduction intensity value to obtain continuous frame images; Inter-frame alignment based on motion estimation and compensation is performed on the consecutive frame images to obtain aligned consecutive frame images; The aligned continuous frame images are subjected to time-domain filtering based on weighted average to obtain effective continuous frame images; The effective continuous frame image is subjected to Gaussian-based spatial filtering to obtain a smooth continuous frame image; Edge texture enhancement is performed on the smooth continuous frame image to obtain the effective target image.
7. The image acquisition and processing method according to claim 6, characterized in that, The step of performing inter-frame alignment based on motion estimation and compensation on the consecutive frame images to obtain aligned consecutive frame images specifically includes: Obtain the image pixel information of the consecutive frames; Based on the motion estimation algorithm, the displacement vector between images is calculated according to the image pixel information; Motion compensation is performed on the consecutive frame images based on the displacement vector to obtain the aligned consecutive frame images.
8. An image acquisition and processing device, characterized in that, include: The intensity value setting module is used to obtain the initial gain parameter and set the initial noise reduction intensity value according to the initial gain parameter; An exposure detection module is used to perform automatic exposure based on the initial noise reduction intensity value, and to detect whether the automatic exposure has completed the convergence of the first frame image; The intensity value adjustment module is used to adjust the initial noise reduction intensity value to the optimal value to obtain the optimal noise reduction intensity value if the automatic exposure completes the convergence of the first frame image. An image denoising module is used to perform image denoising processing based on the optimal denoising intensity value to obtain an effective target image; The image output module is used to output the effective target image.
9. A computer device, characterized in that, The method includes a memory and a processor, wherein the memory stores computer-readable instructions, and the processor executes the computer-readable instructions to implement the steps of the image acquisition and processing method as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the image acquisition and processing method as described in any one of claims 1 to 7.