Electronic screen image quality enhancement method and system for exposure correction and moire synchronization

Abstract

The application discloses an electronic screen image quality enhancement method and system for exposure correction and moire removal synchronization, which aims to simultaneously obtain defocus reference images, ISO reference images and exposure reference images by simultaneously referring to image changes caused by imaging parameters related to moire and exposure, to synchronously realize exposure correction and moire removal, to obtain better image quality enhancement effect for electronic screen images, and to solve the technical problem that the prior art uses single-factor reference pictures to respectively perform exposure correction and moire removal, resulting in detail loss or ghosting artifacts.

Description

Technical Field

[0001] This invention belongs to the field of imaging technology, and more specifically, relates to a method and system for enhancing the image quality of electronic screens by synchronizing exposure correction and moiré removal. Background Technology

[0002] With the rapid development of mobile photography devices, people can use them to capture various complex scenes in life anytime, anywhere, with high quality. Imaging electronic screens, acquiring images of electronic screens, and extracting their image information has become a common imaging scenario and requirement. Typical examples of imaging electronic screens include concert venues, areas near electronic screens in shopping malls, and office workstations.

[0003] However, due to the display principle of electronic screens, the acquired images exhibit a series of unique image quality degradation problems: the most obvious is the degradation caused by moiré patterns, resulting from the frequency aliasing effect between the high-frequency repetitive textures on the display device surface (the pixel array on the screen) and the pixel array of the image sensor in the photographic equipment; in addition, electronic images often suffer from overexposure / underexposure problems. This problem arises because the ambient light in the scene is relatively low while the brightness of the electronic display screen is significantly high, making it difficult to balance exposure when imaging images containing the electronic screen. Exposure defects and meaningless moiré patterns in electronic screen images often damage image details, destroy image integrity, reduce the aesthetic value of the image, and may also significantly reduce the performance of downstream image analysis and recognition algorithms.

[0004] In recent years, researchers both domestically and internationally have conducted a series of studies on exposure problems and moiré removal problems, achieving significant progress. For exposure problems, the mainstream solution is to compensate for the inherent limitation of camera sensor dynamic range through software algorithms. This involves intelligently integrating well-exposed area information from images with different exposure levels of the same scene to directly generate a standard dynamic range image with uniform exposure, complete details, and a natural visual appearance. Currently, most multi-exposure fusion methods combine multi-exposure image sequences with multi-ISO image sequences. Leveraging the complementary characteristics of multi-ISO images, low-ISO images ensure area cleanliness, while high-ISO images supplement effective signals in dark areas. Post-processing weighting strategies suppress noise in high-ISO images, ultimately resulting in a fused image sequence with lower overall noise and clearer texture. For moiré pattern removal, multi-focus image demoiré methods are often used. A typical approach is to achieve video demoiré through a dual-camera system (X.Dong, X. Sun, X. Wang, J. Song, Y. Li, W. Li. Video Demoireing Using Focused-Defocused Dual-Camera System. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025, 47(12): 11283-11297): Two cameras are set to focus and defocus states respectively, and a focus image with moiré patterns and a clean defocus image without moiré patterns are acquired in the same scene. Then, an optical flow algorithm is used to align the defocus image and the focus image. The aligned defocus image is used as a reference image to provide image information without moiré patterns for the focus image, thereby guiding the neural network to complete the moiré removal process of the focus image more efficiently and accurately.

[0005] Because the two types of problems mentioned above (exposure problems and moiré removal problems related to image quality enhancement) often occur simultaneously, and both attempt to solve image quality defects by fusing information from multiple images, and perform exposure compensation and moiré removal independently, the electronic screen image loses detail or produces ghosting artifacts, resulting in limited image enhancement effects and difficulty in meeting the image processing needs of complex scenes. Summary of the Invention

[0006] To address the aforementioned deficiencies or improvement needs of existing technologies, this invention provides a method and system for enhancing the image quality of electronic screen images by simultaneously performing exposure correction and moiré removal. The aim is to obtain a defocus reference image, an ISO reference image, and an exposure reference image in one step by simultaneously referencing image changes caused by imaging parameters related to both moiré and exposure. This allows for simultaneous exposure correction and moiré removal, resulting in better image quality enhancement for electronic screen images. This solves the technical problem of existing technologies that use a single-factor reference image for separate exposure correction and moiré removal, leading to detail loss or ghosting artifacts.

[0007] To achieve the above objectives, according to one aspect of the present invention, a method for enhancing the image quality of an electronic screen by synchronizing exposure correction and demoiré removal is provided, comprising the following steps: (1) Acquire the image of the electronic screen to be corrected, as well as the defocus reference image, ISO reference image, and exposure reference image of the electronic screen; The defocus reference image is used to reflect the differences in images acquired by imaging the electronic screen under different focusing states, including a front defocus image and a rear defocus image; the only difference between the defocus reference image and the electronic screen image to be corrected is the focal length. The ISO reference image is used to reflect the differences in images acquired by imaging an electronic screen at different ISO values. The only difference between the ISO reference image and the electronic screen image to be corrected is the ISO value. It includes low ISO images and high ISO images. The low ISO images, the electronic screen image to be corrected, and the high ISO images form a multi-ISO image sequence, wherein the number of low ISO images and high ISO images is the same. The exposure reference image is used to reflect the differences in images obtained by imaging the electronic screen at different exposure times. The only difference between the imaging parameters of the exposure reference image and the electronic screen image to be corrected is the exposure time. It includes low-exposure images and high-exposure images. The low-exposure images, the electronic screen image to be corrected, and the high-exposure images form a multi-exposure image sequence, wherein the number of low-exposure images and high-exposure images is the same. The number of images in the exposed image sequence is the same as the number of images in the ISO image sequence; (2) Obtain exposure correction residual map: fuse the ISO image sequence and the multi-exposure image sequence and superimpose the texture information extracted from the defocused image, input it into the HDR model obtained by training the MEF model to extract the exposure correction residual map; (3) Demoiré image extraction: The moiré is initially removed using multi-focus images to obtain a multi-focus demoiré image. The moiré region coordinates extracted from the multi-focus demoiré image are then used to finely remove the moiré to obtain a demoiré image. (4) Fusion correction: The exposure correction residual image obtained in step (2) and the demoiré image extracted in step (3) are added together to obtain an enhanced electronic screen image.

[0008] Preferably, in the method for enhancing the image quality of an electronic screen by synchronizing exposure correction and demoiring, the defocus reference image, ISO reference image, and exposure reference image of the electronic screen are imaged on the same electronic screen as the image of the electronic screen to be corrected. The other imaging parameters of the defocus reference image, ISO reference image, and exposure reference image of the electronic screen are the same as those of the above three types of images imaged on the same electronic screen.

[0009] Preferably, the method for enhancing the image quality of an electronic screen by synchronizing exposure correction and demoiring includes step (2) as follows: (2-1) The multi-ISO image sequence and multi-exposure image sequence obtained in step (1) are fused to obtain the fused image sequence with exposure correction; the fusion weight used during fusion is determined by the local noise perception algorithm according to the principle that the higher the noise intensity, the lower the fusion weight. (2-2) The gradient of the defocused image in different focal images obtained in step (1) is used as texture information. After being superimposed with the fused image sequence obtained in step (2-1), it is input into the HDR model obtained by training the MEF model to obtain the exposure correction residual map.

[0010] Preferably, the electronic screen image quality enhancement method for exposure correction and moiré removal synchronization includes the following sub-steps in step (3): (3-1) Preliminary removal of moiré patterns: The moiré patterns of the electronic screen image to be corrected obtained in step (1) are removed using the ESDNet network to obtain a preliminary moiré pattern removed image; (3-2) Using multifocal images to remove moiré patterns: Using the preliminary moiré pattern removal image obtained in step (3-1) as the registration target, the defocus reference image is subjected to optical flow estimation and the dense displacement field between the defocus image and the reference image is calculated for registration, thereby obtaining a defocus registration image that is registered to the focus image based on pixels; the coordinates of the moiré pattern region are extracted based on the multi-ISO image sequence and / or multi-exposure image sequence, and the defocus registration image and the preliminary moiré pattern removal image are weighted and multifocally fused within the moiré pattern region to obtain a multifocal demoiré pattern image.

[0011] Preferably, the electronic screen image quality enhancement method that synchronizes exposure correction and moiré removal includes step (3-1) performing image enhancement on the defocused and registered image; the image enhancement includes ghosting removal and color correction. The ghost removal algorithm calculates the difference between the registered image and the focused image, binarizes the result, calculates the ratio of the sum of bright spots in the image to the total area of ​​the image to adaptively select the kernel size for median filtering and closing operation, and finally achieves ghost removal through adaptive closing operation and median filtering. The color correction is based on the characteristic that the low-frequency information of the focused image and the defocused image are consistent. It implements the color correction mapping algorithm by calculating the average RGB channels of the focused image and the defocused image, as follows:

[0012] in , These represent the single-channel focused image after demoiring, the registered front defocused image pixels, and the registered rear defocused image pixels, respectively. This represents the pixels of the focused image after single-channel color correction, and finally, the focused image after color correction of all three color channels is obtained. .

[0013] Preferably, the method for enhancing the image quality of electronic screen images with exposure correction and demoiring synchronization, wherein the extraction of moiré region coordinates based on multi-ISO image sequences specifically involves: comparing the intra-sequence image information of the multi-ISO image sequences obtained in step (1) to extract the coordinate range of the moiré region; Comparing the image information within the multi-ISO image sequence obtained in step (1), the specific steps for extracting the coordinate range of the moiré region are as follows: The pattern moiré recognition algorithm is used to extract the pattern moiré coordinates of the multi-ISO image sequence: the gradient maps of the high-ISO image and the low-ISO image are calculated and binarized to obtain the binarized gradient maps of the high-ISO image and the low-ISO image. The difference operation is performed on the binarized gradient maps of the high-ISO image and the low-ISO image and Gaussian blur is performed to obtain the difference map. The area with a pixel value greater than a preset threshold in the difference map is taken as the moiré region. The maximum coordinates and minimum coordinates of the moiré region are extracted as the coordinate range of the moiré region. Preferably, in the electronic screen image quality enhancement method for simultaneous exposure correction and demoiring, the high-ISO images of the multi-ISO image sequence are subjected to noise reduction processing.

[0014] Preferably, the method for enhancing the image quality of electronic screen images with exposure correction and demoiring synchronization, wherein the extraction of moiré region coordinates based on multi-exposure image sequences specifically involves: comparing the intra-sequence image information of the multi-exposure image sequence obtained in step (1) to extract the coordinate range of the moiré region; Comparing the image information obtained in step (1) within the multi-ISO image sequence, the coordinate range of the moiré region is extracted as follows: The pattern moiré recognition algorithm is used to extract the pattern moiré coordinates of the multi-ISO image sequence: the gradient maps of the high-ISO image and the low-ISO image are calculated and binarized to obtain the binarized gradient maps of the high-ISO image and the low-ISO image. The difference operation is performed on the binarized gradient maps of the high-ISO image and the low-ISO image and Gaussian blur is performed to obtain the difference map. The area with a pixel value greater than a preset threshold in the difference map is taken as the moiré region. The maximum coordinate and minimum coordinate of the moiré region are extracted as the coordinate range of the moiré region.

[0015] Preferably, in the electronic screen image quality enhancement method of exposure correction and demoiring synchronization, when comparing the intra-sequence image information of the multi-ISO image sequence and the multi-exposure image sequence obtained in step (1) to extract the coordinate range of the moiré region, the moiré region extracted based on the multi-ISO image sequence and the moiré region extracted based on the multi-exposure image sequence are merged as the coordinate range of the finally extracted moiré region.

[0016] According to another aspect of the present invention, an electronic screen image quality enhancement system for exposure correction and moiré de-aliasing synchronization is provided, which is an electronic device or a non-transitory computer-readable storage medium. The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the program to implement the steps of the electronic screen image quality enhancement method for exposure correction and moiré de-synchronization provided by the present invention. The non-transitory computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of the electronic screen image quality enhancement method for exposure correction and demoiré synchronization provided by the present invention.

[0017] In summary, compared with the prior art, the above-described technical solutions conceived by this invention can achieve the following beneficial effects: This invention performs image demoiring and exposure correction by acquiring image sequences with different focal lengths, exposure levels, and ISOs. All of these optical parameters affect the distribution of moiré patterns, with focal length being the key parameter. Furthermore, multiple exposure and ISO sequences can further address exposure issues in the corrected image, improving the dynamic range and better meeting the demoiring requirements in complex environments. This invention simultaneously performs exposure correction and fine-tuning of moiré patterns by referencing multiple pieces of information, avoiding the problems of separate image enhancement processes for different types of electronic screen images. Differences in the reference information used for correction can lead to variations in image enhancement effects, resulting in the loss of low-frequency information or color details, increased noise, or artifact distortion. Attached Figure Description

[0018] Figure 1 This is a flowchart of the electronic screen image quality enhancement method for exposure correction and moiré removal synchronization provided by the present invention. Figure 2 These are images of dark electronic screen images after exposure correction and moiré removal according to embodiments of the present invention; wherein, (a) is the electronic screen image to be corrected, (b) is the image quality enhancement image processed by the ESDNet network, and (c) is the image quality enhancement image processed according to an embodiment of the present invention. Figure 3 These are images of a bright electronic screen image after exposure correction and moiré removal according to an embodiment of the present invention; wherein, (a) is the electronic screen image to be corrected, (b) is the image quality enhancement image processed by the ESDNet network, and (c) is the image quality enhancement image processed according to an embodiment of the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0020] The electronic screen image quality enhancement method with exposure correction and moiré removal synchronization provided by this invention, such as... Figure 1 As shown, it includes the following steps: (1) Acquire the image of the electronic screen to be corrected, as well as the defocus reference image, ISO reference image, and exposure reference image of the electronic screen; The defocus reference image is used to reflect the differences in images acquired from the electronic screen under different focusing states, including a front defocus image and a rear defocus image; the only difference between the defocus reference image and the electronic screen image to be corrected is the focal length.

[0021] The ISO reference image is used to represent the image differences obtained by imaging an electronic screen at different ISO values. The only difference between the ISO reference image and the electronic screen image to be corrected is the ISO value. It includes low ISO images and high ISO images, wherein the ISO value of the electronic screen image to be corrected is 4 times or more the ISO value of the low ISO image, and the ISO value of the high ISO image is 4 times or more the ISO value of the electronic screen image to be corrected. The low ISO images, the electronic screen image to be corrected, and the high ISO images form a multi-ISO image sequence, wherein the number of low ISO images and high ISO images is the same. The exposure reference image is used to reflect the differences in images acquired from the electronic screen at different exposure times. The only difference between the imaging parameters of the exposure reference image and the electronic screen image to be corrected is the exposure time. It includes low-exposure images and high-exposure images. The exposure value of the low-exposure image is 2 eV or more lower than the exposure value of the electronic screen image to be corrected, and the exposure value of the high-exposure image is 2 eV or more higher than the exposure value of the electronic screen image to be corrected. The low-exposure images, the electronic screen image to be corrected, and the high-exposure images form a multi-exposure image sequence, wherein the number of low-exposure images and high-exposure images is the same. The number of images in the exposure image sequence is the same as the number of images in the ISO image sequence.

[0022] All three types of images are projected onto the same electronic screen, and all other imaging parameters are the same as those for the three types of images projected onto the same electronic screen.

[0023] (2) Obtain the exposure correction residual map: fuse the ISO image sequence and the multi-exposure image sequence and superimpose the texture information extracted from the defocused image, input the MEF model to extract the exposure correction residual map; (2-1) The multi-ISO image sequence and multi-exposure image sequence obtained in step (1) are fused to obtain the fused image sequence with exposure correction; the fusion weight used during fusion is determined by the local noise perception algorithm according to the principle that the higher the noise intensity, the lower the fusion weight. (2-2) The gradient of the defocused image in different focal images obtained in step (1) is used as texture information. After being superimposed with the fused image sequence obtained in step (2-1), it is input into the HDR model obtained by training the MEF model to obtain the exposure correction residual map. (3) Demoiré Image Extraction: A multi-focus image is initially used to remove moiré patterns to obtain a multi-focus demoiré image. The moiré patterns in the multi-focus demoiré image are then finely removed using the coordinates of the moiré regions extracted from the multi-ISO image sequence and / or multi-exposure image sequence to obtain a demoiré image. Specifically, this includes the following sub-steps: (3-1) Preliminary removal of moiré patterns: The moiré patterns of the electronic screen image to be corrected obtained in step (1) are removed using the ESDNet network to obtain a preliminary moiré pattern removed image; (3-2) Using multifocal images to remove moiré patterns: Using the preliminary moiré pattern removal image obtained in step (3-1) as the registration target, the defocus reference image is subjected to optical flow estimation and the dense displacement field between the defocus image and the reference image is calculated for registration, thereby obtaining a defocus registration image that is registered to the focus image based on pixels; the coordinates of the moiré pattern region are extracted based on the multi-ISO image sequence and / or multi-exposure image sequence, and the defocus registration image and the preliminary moiré pattern removal image are weighted and multifocally fused within the moiré pattern region to obtain a multifocal demoiré pattern image.

[0024] Preferably, image enhancement is performed on the defocused and registered image; the image enhancement includes ghosting removal and color correction. The specific steps for extracting moiré region coordinates based on multi-ISO image sequences are as follows: compare the intra-sequence image information of the multi-ISO image sequences obtained in step (1) to extract the coordinate range of the moiré region; Comparing the image information obtained in step (1) within the multi-ISO image sequence, the coordinate range of the moiré region is extracted as follows: The pattern moiré recognition algorithm is used to extract the pattern moiré coordinates of the multi-ISO image sequence: the gradient maps of the high-ISO image and the low-ISO image are calculated and binarized to obtain the binarized gradient maps of the high-ISO image and the low-ISO image. The difference operation is performed on the binarized gradient maps of the high-ISO image and the low-ISO image and Gaussian blur is performed to obtain the difference map. The area with a pixel value greater than a preset threshold in the difference map is taken as the moiré region. The maximum and minimum coordinates of the moiré region are extracted as the coordinate range of the moiré region. In the preferred scheme, the high-ISO image of the multi-ISO image sequence is denoised.

[0025] The specific steps for extracting the coordinates of the moiré region based on the multi-exposure image sequence are as follows: compare the intra-sequence image information of the multi-exposure image sequence obtained in step (1) to extract the coordinate range of the moiré region; Compare the intra-sequence image information of the multi-exposure image sequence obtained in step (1) to extract the coordinate range of the moiré region. Specifically, the intensity moiré recognition algorithm is used to extract the coordinates of the intensity moiré by multi-exposure image sequence images: Gaussian blur the high-exposure image and the electronic screen image to be corrected, calculate their gradient map and binarize it to obtain the binarized gradient map of the high-exposure image and the electronic screen image to be corrected. Perform a difference operation on the binarized gradient map of the high-exposure image and the electronic screen image to be corrected to obtain a difference map. The area with a pixel value greater than a preset threshold in the difference map is taken as the moiré region. The maximum and minimum coordinates of the moiré region are extracted as the coordinate range of the moiré region.

[0026] When comparing the intra-sequence image information of the multi-ISO image sequence and the multi-exposure image sequence obtained in step (1) to extract the coordinate range of the moiré region, the moiré region extracted based on the multi-ISO image sequence and the moiré region extracted based on the multi-exposure image sequence are merged as the coordinate range of the finally extracted moiré region.

[0027] While using deep learning neural networks to remove moiré patterns from images can meet the requirements, it can lead to side effects such as color degradation and texture distortion in non-moiré areas. Therefore, by extracting and identifying both moiré and non-moiré regions in an image, the advantages of multiple images can be used to repair image information in non-moiré areas, further improving image quality. Furthermore, based on the characteristics of image sequences obtained under different parameters, refined information can be supplemented. For example, by fusing with defocused images, the depth information provided by the defocused images can be used to perform color and texture correction in non-focal areas.

[0028] (4) Fusion correction: The exposure correction residual image obtained in step (2) and the demoiré image extracted in step (3) are added together to obtain an enhanced electronic screen image.

[0029] The following is an example: The electronic screen image quality enhancement method for simultaneous exposure correction and moiré removal provided in this embodiment is as follows: Figure 1 As shown, it includes the following steps: (1) Acquire the image of the electronic screen to be corrected, as well as the defocus reference image, ISO reference image, and exposure reference image of the electronic screen; Acquire images of the electronic screen at different focal lengths, ISO image sequences, and multi-exposure image sequences; obtain these using a fixed portable device (such as a mobile phone) at a fixed subject distance. Acquire the focused image as the electronic screen image to be calibrated. ISO is set to [250, 1250], and exposure intensity is 0 ev. Other parameters are set automatically by the camera's built-in algorithm.

[0030] The defocus reference image is used to illustrate the differences in images acquired from the electronic screen under different focusing states, including a front defocus image and a rear defocus image; the only difference between the defocus reference image and the electronic screen image to be corrected is the focal length. This embodiment obtains the image by quickly acquiring the front defocus image by adjusting the focal length. and defocused image .

[0031] The ISO reference image is used to represent the image differences obtained by imaging the electronic screen at different ISO values. The only difference between the ISO reference image and the electronic screen image to be corrected is the ISO value. It includes low-ISO images and high-ISO images, which, along with the electronic screen image to be corrected and the high-ISO image, form a multi-ISO image sequence. , , The number of low-ISO images and high-ISO images is the same in this embodiment; the method of obtaining the images in this embodiment is: low-ISO images taken when the ISO is [50, 250). High ISO image taken at ISO (1250, 3200). .

[0032] The exposure reference image is used to represent the image differences obtained by imaging the electronic screen at different exposure times. The only difference between the exposure reference image and the electronic screen image to be corrected is the exposure time. It includes low-exposure images and high-exposure images. The low-exposure images, the electronic screen image to be corrected, and the high-exposure images constitute a multi-exposure image sequence. , , The number of low-exposure images and high-exposure images are the same in this embodiment; the images are obtained as follows: low-exposure images taken at -2ev; and high-exposure images taken at +2ev. .

[0033] The number of images in the exposure image sequence is the same as the number of images in the ISO image sequence.

[0034] All three types of images are projected onto the same electronic screen, and all other imaging parameters are the same as those for the three types of images projected onto the same electronic screen.

[0035] (2) Obtain the exposure correction residual map: fuse the ISO image sequence and the multi-exposure image sequence and superimpose the texture information extracted from the defocused image, input the MEF model to extract the exposure correction residual map; (2-1) The multi-ISO image sequence and multi-exposure image sequence obtained in step (1) are fused to obtain an exposure-corrected fused image sequence; the fusion weight used during fusion is determined by the local noise perception algorithm according to the principle that the higher the noise intensity, the lower the fusion weight; specifically: Multi-exposure image sequence [ , , ] and multi-ISO image sequences[ , , Image block segmentation is performed using a local noise perception algorithm, where: The gradient maps of each low-exposure and low-ISO image patch are used to calculate the corresponding fusion weights using the following formula, and then fused into a low-exposure low-ISO fused image in the fused image sequence:

[0036]

[0037] in, Weighting for low-exposure images, For low ISO image weights, For low-exposure, low-ISO blended images; is the base of the natural logarithm of the exponential function. Indicates gradient calculation; The gradient maps of each high-exposure and high-ISO image patch are used to calculate the corresponding fusion weights using the following formula, and then fused into a high-exposure high-ISO fused image in the fused image sequence:

[0038]

[0039] in, Weighting for low-exposure images, For low ISO image weights, For high-exposure, high-ISO blended images; is the base of the natural logarithm of the exponential function. Indicates gradient calculation; Normally exposed electronic screen image to be corrected No fusion required:

[0040] Finally, the images are fused according to the following formula to obtain the fused image sequence. , , ] (2-2) The gradients of the defocused images in the different focal images obtained in step (1) are used as texture information. After being superimposed with the fused image sequence obtained in step (2-1), the result is input into the trained and converged MEF model to obtain the exposure correction residual map; specifically: Defocused image and The gradient map of the defocused image is calculated using a gradient calculation algorithm. and , gradient map and With multi-exposure image sequences [ , , Add them together to output a fused image sequence with texture information. , , ],as follows:

[0041] in, [ , , ], [ , , ].

[0042] Fuse image sequences [ , , Input the converged MEF model and infer the exposure correction residual map. Training the multi-exposure image fusion U-Net model, also known as the MEF model: The model is trained using the Kalanitari dataset with a batch size of 6 training images, a training image size of 256×256 pixels, an optimizer set to Adam, a learning rate of 0.0025, and 100 training epochs. The model's output is an exposure correction residual map, which, when added to a low dynamic range image, yields the exposure correction result.

[0043] (3) Demoiré Image Extraction: A multi-focus image is initially used to remove moiré patterns to obtain a multi-focus demoiré image. The moiré patterns in the multi-focus demoiré image are then finely removed using the coordinates of the moiré regions extracted from the multi-ISO image sequence and / or multi-exposure image sequence to obtain a demoiré image. Specifically, this includes the following steps: (3-1) Preliminary removal of moiré patterns: The moiré patterns in the electronic screen image to be corrected obtained in step (1) are removed using the ESDNet network to obtain a preliminary image with moiré pattern removal; specifically: The ESDNet demoiring model was trained using the UHDM dataset. The training set consisted of 5000 paired images, with a batch size of 8 and image size of 256×256 pixels. The optimizer was Adam, the learning rate was 0.0001, and the number of training epochs was 100. After training, the electronic screen images to be corrected were... Inference was performed to obtain a preliminary image with moiré pattern removal. .

[0044] (3-2) Using multifocal images to remove moiré patterns: Using the preliminary moiré pattern removal image obtained in step (3-1) as the registration target, the defocus reference image is subjected to optical flow estimation and the dense displacement field between the defocus image and the reference image is calculated for registration, thereby obtaining a defocus registration image that is registered to the focus image based on pixels; the coordinates of the moiré pattern region are extracted based on the multi-ISO image sequence and / or multi-exposure image sequence, and the defocus registration image and the preliminary moiré pattern removal image are weighted and multifocally fused within the moiré pattern region to obtain a multifocal demoiré pattern image.

[0045] The defocused and registered image is obtained using the following method: Forward defocused image and the focused image after moiré removal The input is fed into a trained and converged optical flow estimation model, and the "optical flow" is inferred. Figure 1The dense displacement field was calculated, and the forward defocused image was then processed. and the focused image after moiré removal Registration, obtaining a front defocused image registered to the focused image based on pixels. The focused image after removing moiré patterns and defocused image The input is fed into a trained and converged optical flow estimation model, and the "optical flow" is inferred. Figure 2 The dense displacement field was calculated, and the defocused image was then processed. and the focused image after moiré removal Registration, obtaining a defocused image registered to the focused image based on pixels. The U-Net optical flow estimation model was trained using the FlyingChairs dataset. The batch size of training images was set to 6, the size of training images was 256×256 pixels, the optimizer was set to Adam, the learning rate was set to 0.0025, and the number of training epochs was set to 100.

[0046] Preferably, image enhancement is performed on the defocused and registered image; the image enhancement includes ghosting removal and color correction. After optical flow registration, local registration ghosting may occur, affecting the subsequent demoiring results. This invention addresses this by designing a dedicated deghosting algorithm. This algorithm calculates the difference between the registered image and the focused image, binarizes the result, calculates the ratio of the sum of bright spots in the image to the total image area to adaptively select the kernel size for median filtering and closing operations, and finally achieves ghosting removal through adaptive closing operations and median filtering. The input is the registered pre-defocused image. and defocused image Output the registered and ghost-free front-defocused image. and defocused image .

[0047] Neural network-based demoiring algorithms often result in varying degrees of color distortion after removing moiré patterns. This invention, based on the principle that low-frequency information in both focused and defocused images remains consistent, implements a color correction mapping algorithm by calculating the average RGB channels of both images. Specifically:

[0048] in , These represent the single-channel focused image after demoiring, the registered front defocused image pixels, and the registered rear defocused image pixels, respectively. This represents the pixels of the focused image after single-channel color correction, and finally, the focused image after color correction of all three color channels is obtained. .

[0049] The specific steps for extracting moiré region coordinates based on multi-ISO image sequences are as follows: compare the intra-sequence image information of the multi-ISO image sequences obtained in step (1) to extract the coordinate range of the moiré region; Comparing the image information obtained in step (1) within the multi-ISO image sequence, the coordinate range of the moiré region is extracted as follows: The pattern moiré recognition algorithm is used to extract the pattern moiré coordinates of the multi-ISO image sequence: the gradient maps of the high-ISO image and the low-ISO image are calculated and binarized to obtain the binarized gradient maps of the high-ISO image and the low-ISO image. The difference operation is performed on the binarized gradient maps of the high-ISO image and the low-ISO image and Gaussian blur is performed to obtain the difference map. The area with a pixel value greater than a preset threshold in the difference map is taken as the moiré region. The maximum and minimum coordinates of the moiré region are extracted as the coordinate range of the moiré region. The high-ISO image of the multi-ISO image sequence is denoised.

[0050] Specific steps: High ISO image Low ISO image and the output denoised high-ISO image The coordinates of the moiré region are calculated as input to the pattern moiré recognition algorithm. ,in , Input a high-ISO image. High-ISO images denoised using the U-Net denoising model The U-Net denoising model was trained using the SIDD dataset. The training set consisted of 160 pairs of paired images, with a batch size of 8 and each training image being 256×256 pixels. The optimizer was set to Adam, the learning rate to 0.0001, and the number of training epochs to 100.

[0051] The specific algorithm for pattern moiré recognition is as follows:

[0052]

[0053] in, Indicates gradient calculation, This indicates a binarization operation. Indicates Gaussian blur, In the image The number of pixels in the denoised high-ISO image output at the given location. In the image Pixels of a low-ISO image at that location. The minimum coordinates representing the moiré pattern region. Representing the maximum coordinates of the moiré region, we obtain... ,in , .

[0054] The specific steps for extracting the coordinates of the moiré region based on the multi-exposure image sequence are as follows: compare the intra-sequence image information of the multi-exposure image sequence obtained in step (1) to extract the coordinate range of the moiré region; Comparing the intra-sequence image information of the multi-exposure image sequence obtained in step (1), the coordinate range of the moiré region is extracted. Specifically, the intensity moiré coordinates are extracted using an intensity moiré recognition algorithm on the multi-exposure image sequence: the gradient maps of the high-exposure image and the electronic screen image to be corrected are Gaussian blurred and binarized to obtain the binarized gradient maps of the high-exposure image and the electronic screen image to be corrected. The difference map is obtained by performing a difference operation on the binarized gradient maps of the high-exposure image and the electronic screen image to be corrected. The region with a pixel value greater than a preset threshold in the difference map is taken as the moiré region. The maximum and minimum coordinates of the moiré region are extracted as the coordinate range of the moiré region. In this embodiment, the threshold is 5.

[0055] Specific operations: High-exposure images and the image on the electronic screen to be focused / corrected The coordinates of the moiré region are calculated as input to the intensity moiré pattern recognition algorithm. ,in , .

[0056] The formula for the intensity moiré pattern recognition algorithm is shown below:

[0057]

[0058] in, Indicates Gaussian blur, Indicates binarization, Indicates gradient calculation, The minimum coordinates representing the moiré pattern region. Representing the maximum coordinates of the moiré region, we obtain... ,in , .

[0059] When comparing the intra-sequence image information of the multi-ISO image sequence and the multi-exposure image sequence obtained in step (1) to extract the coordinate range of the moiré region, the moiré region extracted based on the multi-ISO image sequence and the moiré region extracted based on the multi-exposure image sequence are merged as the final extracted coordinate range of the moiré region. In this embodiment, the threshold is set to 5.

[0060] A rectangular region is extracted based on the coordinate range of the moiré pattern region for multi-focus fusion.

[0061] The specific steps involve performing weighted multi-focus fusion of the defocused registered image and the preliminary moiré removal image within the moiré region to obtain a multi-focus demoiré image: For the registered front defocused image Post-defocus image and the focused image after moiré removal Weighted multifocal fusion is performed to effectively recover information in the low-frequency region while preserving details. The fusion formula is shown below:

[0062] in, , , , The images are: multi-image fusion, image with preliminary moiré removal, registered pre-defocused image, and registered post-defocused image, representing the number of pixels. The value of . As weight.

[0063] To ensure clearer texture details, among Taking a value of 0.7, the above fusion method performs image fusion in the moiré coordinate region, while the non-moiré region is... The original pixel values ​​are merged to obtain a demoiré pattern. .

[0064] (4) Fusion Correction: The exposure correction residual image obtained in step (2) and the demoiré image extracted in step (3) are added together to obtain an enhanced electronic screen image. Specifically: Exposure correction residual map With demoiré pattern The images are added together to obtain the exposure-corrected image with demoired edges. .

[0065] Following the above method, the dark electron screen image ( Figure 2 ) and bright electronic screen images ( Figure 3 Exposure correction and moiré removal were performed, and the results were compared with those of the ESDNet network moiré removal method. The results showed that the method provided in this embodiment can not only remove moiré better, but also further correct the exposure problem of low-quality images, improve the dynamic range of images, and enhance the visual effect of images.

[0066] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for enhancing the image quality of an electronic screen by synchronizing exposure correction and moiré removal, characterized in that, Includes the following steps: (1) Acquire the image of the electronic screen to be corrected, as well as the defocus reference image, ISO reference image, and exposure reference image of the electronic screen; The defocus reference image is used to reflect the differences in images acquired by imaging the electronic screen under different focusing states, including a front defocus image and a rear defocus image; the only difference between the defocus reference image and the electronic screen image to be corrected is the focal length. The ISO reference image is used to reflect the differences in images obtained by imaging the electronic screen at different ISO values. The only difference between the ISO reference image and the electronic screen image to be corrected is the ISO value. The sequence includes low-ISO images and high-ISO images. The low-ISO images, the electronic screen image to be corrected, and the high-ISO images form a multi-ISO image sequence, wherein the number of low-ISO images and high-ISO images is the same. The exposure reference image is used to reflect the differences in images obtained by imaging the electronic screen at different exposure times. The only difference between the imaging parameters of the exposure reference image and the electronic screen image to be corrected is the exposure time. It includes low-exposure images and high-exposure images. The low-exposure images, the electronic screen image to be corrected, and the high-exposure images form a multi-exposure image sequence, wherein the number of low-exposure images and high-exposure images is the same. The number of images in the exposed image sequence is the same as the number of images in the ISO image sequence; (2) Obtain exposure correction residual map: fuse the ISO image sequence and the multi-exposure image sequence and superimpose the texture information extracted from the defocused image, input it into the HDR model obtained by training the MEF model to extract the exposure correction residual map; (3) Demoiré image extraction: The moiré is initially removed using multi-focus images to obtain a multi-focus demoiré image. The moiré region coordinates extracted from the multi-focus demoiré image are then used to finely remove the moiré to obtain a demoiré image. (4) Fusion correction: The exposure correction residual image obtained in step (2) and the demoiré image extracted in step (3) are added together to obtain an enhanced electronic screen image.

2. The method for enhancing the image quality of an electronic screen by synchronizing exposure correction and demoiring as described in claim 1, characterized in that, The defocus reference image, ISO reference image, and exposure reference image of the electronic screen are all imaged from the same electronic screen as the image of the electronic screen to be corrected. The other imaging parameters of the defocus reference image, ISO reference image, and exposure reference image of the electronic screen are the same as those of the above three types of images imaged from the same electronic screen.

3. The method for enhancing the image quality of an electronic screen by synchronizing exposure correction and demoiré removal as described in claim 1 or 2, characterized in that, Step (2) is as follows: (2-1) The multi-ISO image sequence and multi-exposure image sequence obtained in step (1) are fused to obtain the fused image sequence with exposure correction; the fusion weight used during fusion is determined by the local noise perception algorithm according to the principle that the higher the noise intensity, the lower the fusion weight. (2-2) The gradient of the defocused image in different focal images obtained in step (1) is used as texture information. After being superimposed with the fused image sequence obtained in step (2-1), it is input into the HDR model obtained by training the MEF model to obtain the exposure correction residual map.

4. The method for enhancing the image quality of an electronic screen by synchronizing exposure correction and demoiring as described in claim 1 or 2, characterized in that, Step (3) includes the following sub-steps: (3-1) Preliminary removal of moiré patterns: The moiré patterns of the electronic screen image to be corrected obtained in step (1) are removed using the ESDNet network to obtain a preliminary moiré pattern removed image; (3-2) Remove moiré patterns using multifocal images: Using the preliminary moiré pattern removal image obtained in step (3-1) as the registration target, perform optical flow estimation on the defocused reference image and calculate the dense displacement field between the defocused image and the reference image to obtain a defocused registration image registered to the focus image based on pixels. Moiré region coordinates are extracted based on multi-ISO image sequences and / or multi-exposure image sequences. Within the moiré region, the defocused registered image and the image with preliminary moiré removal are weighted and multi-focused fused to obtain a multi-focus de-moiré image.

5. The electronic screen image quality enhancement method for exposure correction and moiré removal synchronization as described in claim 4, characterized in that, Step (3-1) performs image enhancement on the defocused and registered image; the image enhancement includes ghosting removal and color correction; The ghost removal algorithm calculates the difference between the registered image and the focused image, binarizes the result, calculates the ratio of the sum of bright spots in the image to the total area of ​​the image to adaptively select the kernel size for median filtering and closing operation, and finally achieves ghost removal through adaptive closing operation and median filtering. The color correction is based on the characteristic that the low-frequency information of the focused image and the defocused image are consistent. It implements the color correction mapping algorithm by calculating the average RGB channels of the focused image and the defocused image, as follows: in , These represent the single-channel focused image after demoiring, the registered front defocused image pixels, and the registered rear defocused image pixels, respectively. This represents the pixels of the focused image after single-channel color correction, and finally, the focused image after color correction of all three color channels is obtained. .

6. The electronic screen image quality enhancement method for exposure correction and moiré removal synchronization as described in claim 4, characterized in that, The specific steps for extracting moiré region coordinates based on multi-ISO image sequences are as follows: compare the intra-sequence image information of the multi-ISO image sequences obtained in step (1) to extract the coordinate range of the moiré region; Comparing the image information obtained in step (1) within the multi-ISO image sequence, the coordinate range of the moiré region is extracted as follows: The pattern moiré recognition algorithm is used to extract the pattern moiré coordinates of the multi-ISO image sequence: the gradient maps of the high-ISO image and the low-ISO image are calculated and binarized to obtain the binarized gradient maps of the high-ISO image and the low-ISO image. The difference operation is performed on the binarized gradient maps of the high-ISO image and the low-ISO image and Gaussian blur is performed to obtain the difference map. The area with a pixel value greater than a preset threshold in the difference map is taken as the moiré region. The maximum coordinate and minimum coordinate of the moiré region are extracted as the coordinate range of the moiré region.

7. The method for enhancing the image quality of an electronic screen by synchronizing exposure correction and demoiré pattern removal as described in claim 6, characterized in that, The high-ISO images of the multi-ISO image sequence are subjected to noise reduction processing.

8. The method for enhancing the image quality of an electronic screen by synchronizing exposure correction and demoiring as described in claim 4, characterized in that, The specific steps for extracting the coordinates of the moiré region based on the multi-exposure image sequence are as follows: compare the intra-sequence image information of the multi-exposure image sequence obtained in step (1) to extract the coordinate range of the moiré region; Comparing the image information obtained in step (1) within the multi-ISO image sequence, the coordinate range of the moiré region is extracted as follows: The pattern moiré recognition algorithm is used to extract the pattern moiré coordinates of the multi-ISO image sequence: the gradient maps of the high-ISO image and the low-ISO image are calculated and binarized to obtain the binarized gradient maps of the high-ISO image and the low-ISO image. The difference operation is performed on the binarized gradient maps of the high-ISO image and the low-ISO image and Gaussian blur is performed to obtain the difference map. The area with a pixel value greater than a preset threshold in the difference map is taken as the moiré region. The maximum coordinate and minimum coordinate of the moiré region are extracted as the coordinate range of the moiré region.

9. The method for enhancing the image quality of an electronic screen by synchronizing exposure correction and demoiring as described in claim 4, characterized in that, When comparing the intra-sequence image information of the multi-ISO image sequence and the multi-exposure image sequence obtained in step (1) to extract the coordinate range of the moiré region, the moiré region extracted based on the multi-ISO image sequence and the moiré region extracted based on the multi-exposure image sequence are merged as the coordinate range of the finally extracted moiré region.

10. An electronic screen image quality enhancement system with simultaneous exposure correction and moiré removal, characterized in that, For electronic devices or non-transitory computer-readable storage media; The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, when the processor executes the program, it implements the steps of the electronic screen image quality enhancement method for exposure correction and demoiring synchronization as described in any one of claims 1 to 9. The non-transitory computer-readable storage medium, on which a computer program is stored, is characterized in that, when the computer program is executed by a processor, it implements the steps of the electronic screen image quality enhancement method for exposure correction and demoiring synchronization as described in any one of claims 1 to 9.

Images