Improved 3a image preprocessing method based on DM8127

Abstract

The present application belongs to the technical field of image processing, and in particular to an improved 3A image preprocessing algorithm based on DM8127. Through the research on the automatic focusing algorithm, the definition evaluation function is analyzed and compared, the search mode for finding the optimal value is improved, the search times are reduced, and better search effect is achieved. Combined with the influence of the aperture on the depth of field blur and the influence of the gain on the noise, a new automatic exposure algorithm is proposed based on the quality evaluation function, and the best comprehensive image quality balance between the depth of field blur and the noise is achieved. In the automatic white balance part, the image is divided, the gray area is found through the YUV component, and the parameters are adjusted on the DM8127 platform to make the result converge, which achieves better effect than the commonly used automatic white balance algorithm.

Description

Technical Field

[0001] The invention belongs to the technical field of image processing, and in particular, is an improved 3A image preprocessing method based on DM8127. Background Art

[0002] Video is now an integral part of our daily lives. High-definition video, HDTV, and HD calls all demonstrate the importance of high resolution in wireless networks. However, if the video resolution received by users is low, affecting the viewing experience and the delivery of information, the value of the video is diminished, leaving it like garbage in the vast cyberspace. Every day, the same video data undergoes countless round trips. People need video to reach their devices quickly and receive the latest information in real time to stay ahead in this competitive world. Naturally, the timeliness of video requires fast transmission speeds. Therefore, in order to transmit large amounts of critical HD video data in open wireless environments, improving the quality of captured video and achieving fast and secure transmission has become a hot topic in the video processing field.

[0003] Currently, commonly used image preprocessing techniques include color interpolation, color correction, gamma correction, image enhancement, and white balance. The 3A algorithm analyzes images through autofocus, auto exposure, and auto white balance to maximize image contrast, correct over- or underexposure of the target object, and compensate for color differences in the image under different lighting conditions, resulting in higher-quality image information.

[0004] The DM8127 chip has been specially optimized for security monitoring. It has unique low-light technology and high-efficiency compression functions, outputs video at a full frame rate of 1080P60 frames / s, and integrates image signal processing technologies such as 3D noise reduction, wide dynamic range and strong light suppression processing technology, face detection, video stabilization, and zoom distortion correction. In addition, it also uses image acquisition technology that supports up to 16 million pixels, combined with a unique 750MHz DSP for intelligent analysis, to achieve high-definition monitoring of smart front-ends. Summary of the Invention

[0005] This paper takes image preprocessing as its fundamental starting point and proposes an improved 3A image preprocessing method based on the DM8127. The DM8127 chip offers advantages such as high resolution, a rich algorithm library, multi-core communication, and superior DSP computing performance. Therefore, this method uses the DM8127 chip as the computing platform and preprocesses the image at the receiving end based on the 3A algorithm. This method optimizes the process of finding the optimal value, finding the optimal balance between depth of field blur and noise, and improving image quality.

[0006] The specific technical solutions adopted in the present invention are as follows:

[0007] An improved 3A image preprocessing method based on DM8127 is proposed. The DM8127 video processing system sends video images to a PC for image acquisition. The acquisition process is based on the 3A algorithm. The 3A algorithm analyzes images through autofocus, autoexposure, and autowhite balance to maximize image contrast, improve overexposure or underexposure of the target object, and compensate for color differences under different lighting conditions, thereby presenting higher-quality image information.

[0008] In the above technical solution, the DM8127 primarily consists of an ARM core and a DSP core. The ARM core is a Cortex-A8 processor, and the DSP core is a digital signal processor. It also integrates two ARM Cortex-M3 cores, the M3-VPSS core and the M3-VIDEO core, as coprocessors, for controlling and managing the High-Definition Video Processing Subsystem (HDVPSS) and the High-Definition Video Image Coprocessor (HDVICP2), respectively. The DM8127 cores are connected by an internal L3 bus, enabling interoperability between components. The ARM core runs the Linux system, invoking the VPSS, Video, and DSP cores to control the functionality of each module. The M3-VPSS core, after running the BIOS, implements image acquisition, display, and scaling. The M3-VIDEO core, after running the BIOS, receives pre-processed image data and is responsible for video image encoding and decoding. The DSP core is responsible for algorithm application and runs the BIOS system to perform image encryption operations. Finally, the ARM core receives the data stream and transmits it over the network.

[0009] A further improvement of the present invention is to optimize the optimal value of the algorithm, optimize the global search, divide the global search range into different areas, and use a different step size for each area. For the focus search problem, the number of searches can be reduced, and a better search effect can be achieved.

[0010] In the above technical solution, the automatic exposure process is divided into two steps: automatic gain and automatic aperture. Automatic gain works with the aperture to adjust the amount of light entering. The H3A module of the DM8127 platform calculates the current brightness value curY, sets the target brightness as targetY, and calculates the ratio. The calculation formula is as follows:

[0011]

[0012] Then adjust the exposure time, sensor gain and chip digital gain parameters according to the ratio.

[0013] Automatic aperture is a calculation quality evaluation function that changes the aperture to find the maximum value. The image quality evaluation function F(I) for the aperture is defined as follows:

[0014]

[0015] The captured image I(x, y) is essentially identical to the scene image R(x, y). However, the quality of the scene image R(x, y) is unknown, so the quality of the captured image I(x, y) is used as the criterion for the optimal aperture. h(x, y, σ) is the blur function, and noise is affected by the gain g. The optimal image quality is a balance between depth of field blur and noise.

[0016] Automatic white balance includes illumination estimation and image color correction. The image is divided into regions to find gray areas. The white balance algorithm is implemented on the experimental platform, and the parameters are adjusted to make the results converge, which achieves better results than the commonly used automatic white balance algorithm. Automatic white balance includes illumination estimation and image color correction. Grayscale world and perfect reflection are two classic and common hypothesis algorithms. The core idea of ​​the grayscale world is to control the average values ​​of the three channels to be equal, and the core idea of ​​perfect reflection is to control the maximum values ​​of the three channels to be equal. The two methods have their own advantages and disadvantages, and their applicability to different scenes varies. In order to overcome the shortcomings of the two methods, the present invention proposes an improved algorithm in the illumination estimation step: partition the image, the closer to gray, the more it can satisfy the grayscale world hypothesis, adjust the gains of the three channels in each area, and make the mean value of each area close to gray. Define image I(x, y), and add coefficient μ r 、μ b , γ r , γ b For the pixels of the corrected image, new pixel values ​​can be calculated according to equations (1) and (2), new pixel means can be calculated according to equations (3) and (4), and new maximum values ​​can be calculated according to equations (5) and (6):

[0017]

[0018] meanG=μ r meanR 2 +γ r meanR (3)

[0019] meanG=μ b meanB 2 +γ b meanB (4)

[0020] max(R new )=max{Ig (x,y)} (5)

[0021] max(B new )=max{I g (x, y)} (6)

[0022] The improved method does not modify the green channel, and satisfies the gray world algorithm. The mean of the new red and blue channels is equal to that of the green channel. It satisfies the perfect reflection algorithm. The maximum value of the new red and blue channels is equal to that of the green channel. Substituting equations (1) and (2) into equations (5) and (6), we can obtain:

[0023]

[0024] According to equations (3), (4), (7), and (8), the mean and maximum values ​​of the red and blue channels are expressed in matrix form, as shown in equations (9) and (10):

[0025]

[0026] The Gaussian elimination method is used to solve the gain coefficients of the red and blue channels as follows:

[0027]

[0028] This method was experimented on the DM8127 platform, and the parameters were adjusted to make the results converge, achieving better results than the commonly used automatic white balance algorithm.

[0029] The beneficial effects of the present invention are as follows: Under no-light conditions, the present invention adjusts the black level to bring the values ​​of each channel as close to 0 as possible; the automatic exposure module adjusts parameters such as exposure time and aperture size to select the appropriate light intensity; the automatic focus determines the appropriate focal length; the automatic white balance corrects the influence of color according to different scenes; defective pixels in the image are processed in the bad pixel correction process; 2D noise reduction can reduce noise and improve interpolation accuracy; Bayer interpolation produces data for the three channels of red, green, and blue; gamma correction and color correction are used to correct the contrast and color of the image; the brightness contrast enhancement and edge enhancement modules can increase brightness and enhance image clarity; and finally, the distortion correction module corrects the distorted image. The various modules of image preprocessing are relatively independent and work together to obtain high-quality images. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 This is a preprocessing flow chart in an embodiment of the present invention.

[0031] Figure 2 4 is a flow chart of an algorithm in an embodiment of the present invention.

[0032] Figure 3 4 is a flowchart of automatic exposure in an embodiment of the present invention.

[0033] Figure 4 This is a block diagram of the H3A module in an embodiment of the present invention. DETAILED DESCRIPTION

[0034] In order to deepen the understanding of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. The embodiments are only used to explain the present invention and do not limit the scope of protection of the present invention.

[0035] Example: Figure 1 As shown in the figure, an improved 3A image preprocessing method based on the DM8127 is used to send video images to a PC via the DM8127 video processing system. Considering various factors affecting image quality, the method studies multiple modules, including image color, light intensity, and camera focal length, ensuring that each module coordinates with the others. In the absence of light, black level adjustment is used to keep the values ​​of each channel as close to zero as possible. The automatic exposure module adjusts parameters such as exposure time and aperture size to select the appropriate light intensity. Autofocus determines the appropriate focal length. Automatic white balance corrects color effects based on different scenes. Defective pixels in the image are addressed in the bad pixel correction process. 2D noise reduction reduces noise and improves interpolation accuracy. Bayer interpolation generates data for the red, green, and blue channels. Gamma correction and color correction are used to adjust image contrast and color. The brightness contrast enhancement and edge enhancement modules increase brightness and enhance image clarity. Finally, the distortion correction module corrects distorted images. Each image preprocessing module is relatively independent and works together to produce high-quality images.

[0036] like Figure 2 As shown, based on the DM8127 video processing system, the present invention optimizes and adjusts the H3A module to achieve optimal image quality. Image preprocessing is performed through three hardware modules: ISIF, H3A, and IPIPE. The preprocessing process also adds 2A (auto exposure and auto white balance), scene-adaptive local dynamic range enhancement, video stabilization, lens distortion correction, noise filtering, and screen software display algorithms for optimization and adjustment.

[0037] Figure 3The following is a flowchart of the automatic exposure process in an embodiment of the present invention. First, the initial aperture is set and the brightness is calculated based on the actual scene. If it is within the threshold range, the parameters remain unchanged. If it exceeds the threshold, it is determined whether the low-light threshold condition has been met. If it is less than the low-light threshold, the maximum aperture and maximum exposure time are selected for gain; if it is greater than the low-light threshold, the experiment is conducted according to normal lighting. Under normal lighting, the aperture size is adjusted according to the quality evaluation function to find the maximum value of the function, and the automatic exposure process ends. If the maximum value is not found, the following steps are repeated until the maximum value is found: adjust the aperture direction according to the scene brightness. If the light intensity increases, the aperture is closed; if the light intensity decreases, the aperture is opened.

[0038] Under normal lighting conditions, first adjust the gain and use the H3A module of the DM8127 platform for calculation. Then, based on the set target brightness targetY, the calculation formula is as follows:

[0039]

[0040] Then calculate the quality evaluation function of the current image as follows:

[0041]

[0042] The maximum value is found based on the quality evaluation function, and the aperture is set to that maximum value, ending the automatic exposure process. If no maximum value is found, the gain is adjusted based on the lighting direction. If the scene becomes brighter, the aperture should be closed to reduce depth of field blur; if the scene becomes darker, the aperture should be opened to reduce the influence of noise. This process is repeated until the optimal value is found.

[0043] like Figure 4 As shown in the figure, the DM8127 hardware 3A module provides hardware support for pixel statistics for automatic exposure. The acquisition process is as follows:

[0044] First, the image is downsampled. Each frame is divided into windows, and each window is divided into 2×2 blocks. Each pixel in each block is counted separately. Then, the image is checked for saturation. If the number of pixels in a block exceeds the limit, it is not counted as an unsaturated block. The limit is replaced and the count is repeated. Finally, the number of pixels in each pixel window is accumulated and output.

[0045] After in-depth research and analysis of the H3A module, this paper studies depth of field blur, incorporates the blur caused by depth of field into the considerations of aperture adjustment, and proposes an evaluation function for optimizing the aperture. Combined with this evaluation function, the optimal aperture size can be found, finding the best balance between depth of field blur and noise, thereby optimizing the overall image quality.

[0046] The above shows and describes the basic principles, main features and advantages of the present application. Those skilled in the art should understand that the present application is not limited to the above-mentioned embodiments, and the above-mentioned embodiments and descriptions in the specification are only to illustrate the principles of the present application. Without departing from the spirit and scope of the present application, various changes and improvements can be made to the present application, and these changes and improvements all fall within the scope of the claimed present application. The scope of protection of the present application is defined by the appended claims and their equivalents.

Claims

1. An improved 3A image preprocessing method based on DM8127, characterized in that: The DM8127 video processing system sends video images to the PC for image acquisition. The acquisition process is based on the 3A algorithm, which specifically includes autofocus, autoexposure, and auto white balance. The DM8127 video processing system consists of an ARM core and a DSP core. The ARM core is a Cortext-A8 processor, and the DSP core is a digital signal processor. It also integrates two ARM Cortex-M3 cores, the M3-VPSS core and the M3-VIDEO core, as coprocessors, which are used to control and manage the high-definition video processing subsystem and the high-definition video image coprocessor, respectively. The DM8127 cores are connected by the L3 internal bus, and mutual access between the various components is achieved through the L3 bus. The automatic exposure process is divided into two steps: automatic gain and automatic aperture. Let the target brightness be targetY and calculate the ratio. The calculation formula is as follows: Then adjust the exposure time, sensor gain and chip digital gain according to the ratio; Automatic aperture is a calculation quality evaluation function that changes the aperture to find the maximum value. The image quality evaluation function F(I) for the aperture is defined as follows: Among them, the acquired image I(x, y) is consistent with the scene image R(x, y), but the quality of the scene image R(x, y) is unknown, so the quality of the acquired image I(x, y) is used as the standard for the optimal aperture, h(x, y, σ) is the blur function, and the noise is affected by the gain g. The optimal image quality should be a balance between depth of field blur and noise.

2. The improved 3A image preprocessing method based on DM8127 according to claim 1, characterized in that: During the automatic exposure process, the image is partitioned. The closer to gray, the better it satisfies the grayscale world assumption. The gains of the three channels in each area are adjusted so that the mean of each area is gray. The image I(x, y) is defined and the coefficient μ is added. r 、μ b , γ r , γ b For the pixels of the corrected image, the new pixel values ​​are calculated according to equations (1) and (2), the new pixel mean values ​​are calculated according to equations (3) and (4), and the new maximum values ​​are calculated according to equations (5) and (6): meanG=μ r meanR 2 +γ r meanR (3) meanG=μ b meanB 2 +γ b meanB (4) max(R new )=max{I g (x,y)} (5) max(B new )=max{I g (x,y)} (6), Substituting equations (1) and (2) into equations (5) and (6), we can obtain: According to equations (3), (4), (7), and (8), the mean and maximum values ​​of the red and blue channels are expressed in matrix form, as shown in equations (9) and (10): The Gaussian elimination method is used to solve the gain coefficients of the red and blue channels as follows:

3. The improved 3A image preprocessing method based on DM8127 according to claim 2, characterized in that: The 3A module of the DM8127 video processing system provides hardware support for pixel counting for automatic exposure. The acquisition process is as follows: First, the image is downsampled, and each frame is divided into windows. Each window is divided into 2×2 blocks. Each pixel in each block is counted separately. Then, the image is checked for saturation. If the number of pixels in a block exceeds the limit, it is not counted in the number of unsaturated blocks. The limit value is replaced and counted again. Finally, the pixel count of each pixel window is accumulated and output.

Images