Under-screen image processing method, electronic equipment and computer program product

Abstract

The invention provides an under-screen image processing method, electronic equipment and a computer program product. The under-screen image processing method comprises the following steps: acquiring an under-screen image to be processed; extracting a mask to be processed of the under-screen image to be processed; and inputting the to-be-processed under-screen image and the to-be-processed mask into a large model to repair an image area corresponding to the to-be-processed mask in the to-be-processed under-screen image, and obtaining an optimized under-screen image. By means of the under-screen image processing method, the mask area, needing to be optimized, of the under-screen image is extracted, the mask area in the under-screen image is optimized through the large model, and restoration of the under-screen image is achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, and in particular to a screen-under image processing method, an electronic device, and a computer program product. BACKGROUND

[0002] In today's mobile phone market, screen-under imaging technology uses a transparent material as the base of the display layer and embeds light sources and pixel points below it. When light passes through the transparent base layer and pixel points, the size of the light gap can reach 100 times the wavelength of light, which meets the Fraunhofer diffraction theory. Therefore, light diffraction occurs, and after transmission to the camera imaging, the image obtained by the camera becomes blurred.

[0003] In addition, the multiple layers of screen materials will cause multiple reflections of light, so the image signal output by the camera will not only have a serious decrease in signal-to-noise ratio, but also the image will have a fogging phenomenon. In general, ghosting, color glare, noise, and fogging are all factors that cannot be avoided in screen-under images, resulting in low image quality of screen-under images and failing to provide users with a good shooting experience. SUMMARY

[0004] To at least solve the above technical problems, the present application provides a screen-under image processing method, an electronic device, and a computer program product.

[0005] To at least solve the above technical problems, the present application provides a screen-under image processing method, which comprises: acquiring a to-be-processed screen-under image; extracting a to-be-processed mask of the to-be-processed screen-under image; inputting the to-be-processed screen-under image and the to-be-processed mask into a large model to repair an image region corresponding to the to-be-processed mask in the to-be-processed screen-under image, and acquiring an optimized screen-under image.

[0006] To at least solve the above technical problems, the present application further provides an electronic device, which comprises a memory and a processor coupled with the memory; wherein the memory is used to store program data, and the processor is used to execute the program data to realize the screen-under image processing method as described above.

[0007] To at least solve the above technical problems, the present application further provides a computer program product, which comprises a computer program, and the computer program realizes the screen-under image processing method as described above when executed by a processor.

[0008] Compared with existing technologies, the beneficial effects of this application include at least the following: the electronic device acquires an under-display image to be processed; extracts the mask to be processed from the under-display image; inputs the under-display image and the mask to be processed into a large model to repair the image region corresponding to the mask in the under-display image, thereby obtaining an optimized under-display image. Through the above under-display image processing method, the mask region that needs optimization in the under-display image is extracted, and the mask region in the under-display image is optimized using a large model, thus achieving the repair of the under-display image. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] in:

[0011] Figure 1 This is a flowchart illustrating the first embodiment of the under-display image processing method provided in this application;

[0012] Figure 2 This is a schematic diagram of the overall processing flow of the under-display image processing method provided in this application;

[0013] Figure 3 This is a flowchart illustrating the second embodiment of the under-display image processing method provided in this application;

[0014] Figure 4 This is a schematic diagram of an embodiment of the mask extraction method provided in this application;

[0015] Figure 5 This is a flowchart illustrating the third embodiment of the under-display image processing method provided in this application;

[0016] Figure 6 This is a flowchart illustrating the fourth embodiment of the under-display image processing method provided in this application;

[0017] Figure 7 This is a flowchart illustrating the fifth embodiment of the under-display image processing method provided in this application;

[0018] Figure 8 This is a schematic diagram of the structure of an embodiment of the electronic device provided in this application;

[0019] Figure 9 This is a schematic diagram of the structure of an embodiment of the computer program product provided in this application. Detailed Implementation

[0020] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0021] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the application described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0022] Currently, under-display image restoration mainly falls into two categories: algorithm optimization and hardware optimization. Hardware optimization primarily focuses on increasing screen light transmittance, but the narrow light-transmitting gaps and the screen's coating itself both affect the unavoidable problem of under-display image quality. There is currently no good solution. Algorithm optimization mainly categorizes problems into two types: glare and ghosting, primarily using AI (Artificial Intelligence) networks. One approach repairs glare, while the other addresses ghosting through degradation. However, both methods only solve single problems and do not address the various under-display issues in a comprehensive way, making it difficult to achieve truly perfect results.

[0023] There are two main approaches to glare repair in existing patents. The first is through HDR (High Dynamic Range Imaging), which reduces exposure and uses a mask to integrate glare from underexposed lighting areas. While this method is effective in indoor lighting environments, it cannot completely eliminate glare from very strong sunlight, non-overexposed glare at specific angles, or colored light caused by dispersion. Furthermore, it doesn't optimize for ghosting caused by diffraction in non-light-lit areas. The second approach uses generative networks to repair glare individually in the lit area. While this method can improve glare to some extent, the improvement is not significant.

[0024] Regarding the ghosting problem, a good approach is to construct a dataset using PSF (point spread function) degradation, which can help eliminate ghosting and enhance images to some extent. However, the complexity of everyday lighting far exceeds that of simulated data and cannot cover all scenes. Furthermore, glare in illuminated areas cannot be resolved using this method.

[0025] Therefore, to improve under-display image quality, this application provides an under-display image processing solution to enhance processing performance. This application identifies glare and ghosting effects outside the light source itself and extracts a mask. A large model is then used to repair the masked areas. Mask extraction requires capturing two images at the same location: one under-display and one not under-display. These two images are then aligned and registered to identify the differences, which represent the locations of glare and ghosting effects, thus obtaining their masked areas. This constructs a paired dataset for extracting glare and ghosting masks. An AI network is then used to train this dataset to obtain a model capable of directly extracting glare and ghosting masks from under-display images. Finally, the large model is used to repair the masked areas in the input under-display image, resulting in an optimized under-display image. This not only mitigates the negative impact of under-display cameras but also allows users to enjoy a superior photography experience.

[0026] The solutions proposed in this application are described below from the perspectives of under-display image processing application and under-display image processing training:

[0027] See Figure 1 and Figure 2 , Figure 1 This is a flowchart illustrating the first embodiment of the under-display image processing method provided in this application. Figure 2 This is a schematic diagram of the overall processing flow of the under-display image processing method provided in this application.

[0028] The under-display image processing method of this application can be applied to an under-display image processing device, which can be a server, a terminal device, an electronic device, or a system in which a server and a terminal device cooperate with each other. Accordingly, the various parts of the under-display image processing device, such as various units, sub-units, modules, and sub-modules, can all be set in the server, all in the terminal device, or separately in the server and the terminal device.

[0029] Furthermore, the aforementioned server can be either hardware or software. When the server is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When the server is software, it can be implemented as multiple software programs or software modules, such as software or software modules used to provide distributed server functionality, or as a single software program or software module; no specific limitations are made here.

[0030] In one exemplary embodiment, the under-display image processing device of this application can be a processing unit or processor in a mobile terminal with shooting function, such as a smartphone or tablet, or a processing platform mounted on the mobile terminal or used to remotely control the mobile terminal.

[0031] like Figure 1 As shown, its under-screen image processing method may include the following operations:

[0032] Step S11: Obtain the under-screen image to be processed.

[0033] In one exemplary embodiment, the under-display image processing device acquires an under-display image to be processed, and the acquisition method includes, but is not limited to: controlling the camera to capture the under-display image in real time, and acquiring the preview image captured and displayed by the camera on the terminal device.

[0034] In one exemplary embodiment, the under-display image processing device can also acquire and process under-display images from other storage media.

[0035] In one exemplary implementation, such as Figure 2 As shown, the under-display image processing device can also enhance the contrast of the under-display image to be processed, thereby removing haze and improving image clarity and transparency. Transparency, in particular, refers to the comprehensive visual and psychological experience of an image, encompassing rich and clear color tones, a deep and bright sense of space, and clear and distinct object outlines.

[0036] In one exemplary embodiment, the under-display image processing device may employ adaptive histogram equalization (AHE) technology to improve the visual effect of under-display images during actual processing. AHE optimizes the contrast of each region by performing histogram equalization on local areas of the image, making image details more clearly visible.

[0037] In one exemplary embodiment, when performing adaptive histogram equalization, the under-display image processing device can use the grayscale histogram of a standard non-under-display image as a contrast enhancement target. This standard non-under-display image is an image collected and processed by staff, designed for optimal display performance on the human visual system.

[0038] The under-display image processing device determines a standard grayscale histogram. By increasing the brightness of dark areas and decreasing the brightness of bright areas in the grayscale histogram of the under-display image, it approaches and eventually reaches the standard grayscale histogram.

[0039] It should be noted that the under-display image processing device of this application can also use other contrast enhancement methods, such as linear enhancement, non-linear enhancement, etc.

[0040] Step S12: Extract the mask to be processed from the under-screen image to be processed.

[0041] In one exemplary embodiment, the under-display image processing device inputs the under-display image to be processed after step S11 into a pre-trained mask extraction model, such as... Figure 2 The AI ​​network shown is used to extract the mask to be processed from the under-screen image, such as... Figure 2 The image shows a glare mask. Furthermore, the mask extraction model can be trained to extract corresponding masks for different under-screen image problems, such as ghosting masks, noise masks, and fogging masks.

[0042] Step S13: Input the under-screen image to be processed and the mask to be processed into the large model to repair the image region corresponding to the mask in the under-screen image to be processed, and obtain the target under-screen image.

[0043] In one exemplary embodiment, the under-display image processing device inputs the under-display image to be processed and the mask to be processed into a large model, and uses the large model to optimize the image region corresponding to the mask in the under-display image to be processed. Specifically, the mask region to be processed extracted from the under-display image by the mask extraction model in this application is essentially the image region with poor quality in the under-display image. The large model then repairs the under-display image, that is, it restores the image region with poor quality or fills it with other image regions of better quality, thereby improving the overall performance quality of the under-display image.

[0044] In one exemplary embodiment, the large model used in this application can be a generative large model, which can generatively fill the mask area extracted from the under-screen image to achieve repair and reach an image quality level similar to that of the non-under-screen image.

[0045] In this application, an under-display image processing device acquires an under-display image to be processed; extracts a mask from the under-display image to be processed; and inputs the under-display image to be processed and the mask into a large model to repair the image region corresponding to the mask in the under-display image to be processed, thereby obtaining an optimized under-display image. Through the above under-display image processing method, the mask region that needs optimization in the under-display image is extracted, and the mask region in the under-display image is optimized using a large model, thereby achieving the repair of the under-display image.

[0046] Figure 1 Please continue to refer to the model training process of the mask extraction model mentioned in step S12 of the illustrated embodiment. Figure 2 See Figure 3 ,Figure 3 This is a flowchart illustrating the second embodiment of the under-display image processing method provided in this application.

[0047] like Figure 3 As shown, its under-screen image processing method may include the following operations:

[0048] Step S21: Acquire the under-screen image and the off-screen image of the same region to be trained.

[0049] In one exemplary embodiment, the under-screen image processing device captures two images at the same location: an under-screen image and an off-screen image, which are used as training material for the model.

[0050] It should be noted that during model training, the under-display image processing device can also enhance the contrast of the under-display image to more closely resemble the image quality in actual use, ensuring the quality and consistency of the under-display and non-under-display images in subsequent processing. The logic behind the contrast enhancement will not be elaborated upon here.

[0051] In one exemplary implementation, to ensure the consistency and comparability of the dataset, the under-display image processing device must use the same type of camera to capture under-display and non-under-display images at fixed positions and angles to ensure consistency in the perspective of the two images. This step is fundamental to building a high-quality dataset, ensuring accurate image processing and mask extraction in subsequent steps.

[0052] Step S22: Obtain the training mask based on the pixel value difference of each pixel at the same position in the under-screen image to be trained and the non-under-screen image to be trained.

[0053] Since illuminated areas are typically overexposed and lack useful difference information, we focus on extracting the differences, namely glare and ghosting areas, to obtain the corresponding mask image. This approach doesn't simply process the illuminated areas; instead, it specifically targets the colored glare produced by the light and strong light in the image. Although these areas are not completely overexposed, the colored glare obscures the original image information, affecting the overall image quality.

[0054] In one exemplary embodiment, the under-display image processing device extracts glare masks and ghosting masks. See details... Figure 4 , Figure 4 This is a schematic diagram of an embodiment of the mask extraction method provided in this application. By performing difference processing on the under-screen image and the non-under-screen image, a threshold K is set. The mask value of regions with a difference greater than the threshold K is set to 0, and the mask value of the remaining regions is set to 255, thereby obtaining the mask_k of glare and ghosting.

[0055] Step S23: Use the mask to be trained and the under-screen image to be trained as the training set to train the mask extraction model.

[0056] In one exemplary embodiment, the under-display image processing device combines a transparency-enhanced under-display image with a mask_k image into a paired dataset, which is then input into an end-to-end AI network, including but not limited to U-net, for training. After training, the network model, i.e. Figure 1 The mask extraction model used can be directly applied to under-screen images to quickly and accurately obtain the mask for glare and ghosting.

[0057] Furthermore, in order to achieve Figure 3 The model training process shown requires the under-screen image processing device to capture two images—one under the screen and one not—from the same location. Since these two types of images differ in brightness, color, and contrast, they first need to undergo consistency processing and pixel-level alignment.

[0058] In one exemplary implementation, to train an AI network to extract masks of colored glare and ghosting, a high-quality dataset is first required. This dataset is created by capturing images of the same scene, both on-screen and off-screen. However, because these two types of images differ significantly in brightness, color, and contrast, they cannot be directly used for mask extraction. To address this issue, a series of preprocessing steps must be performed on the images to ensure consistency of irrelevant conditions.

[0059] Therefore, after enhancing the contrast of the under-display image, the under-display image processing device also needs to adjust the color and brightness of the non-under-display image to match it. This matching process can be achieved through color space conversion and statistical parameter adjustment. After completing these preprocessing steps, the two images are pixel-level aligned to facilitate accurate comparison of their differences. During this process, areas with significant differences are identified and used as masks.

[0060] The following section explains the color space conversion. The under-display image processing device migrates and matches the colors and brightness of the non-under-display image to the under-display image. For details, please refer to [link to relevant documentation]. Figure 5 , Figure 5 This is a flowchart illustrating the third embodiment of the under-display image processing method provided in this application.

[0061] like Figure 5 As shown, its under-screen image processing method may include the following operations:

[0062] Step S31: Obtain the average value and standard deviation of the first pixel of the under-screen image to be trained in the preset color space.

[0063] In one exemplary embodiment, the under-display image processing device needs to unify under-display images and non-under-display images into a preset color space.

[0064] In one exemplary embodiment, to match the color and brightness of the non-under-screen image with the under-screen image, the under-screen image processing device first converts the two images from the RGB color space to the LAB color space. The RGB color space is a two-dimensional representation consisting of three channels: red (R), green (G), and blue (B). The LAB color space is commonly used in printing and image processing because it is closer to the human visual system and provides more accurate color description and control. Therefore, this application chooses the LAB color space as the preset color space, which ensures that the final optimized under-screen image achieves the best effect in the human visual system.

[0065] It should be noted that the image color space of this application is also applicable to the conversion of other color spaces to the preset color space, which will not be listed here.

[0066] Step S32: Obtain the average value of the second pixel and the standard deviation of the second pixel in the preset color space of the off-screen image to be trained.

[0067] In one exemplary embodiment, the under-screen image processing device calculates the mean and standard deviation of the under-screen image and the non-under-screen image in the LAB color space, respectively.

[0068] Step S33: Adjust the pixel values ​​of the off-screen image to be trained using the first pixel average value, the first pixel standard deviation, the second pixel average value, and the second pixel standard deviation.

[0069] In one exemplary embodiment, the under-display image processing device uses the statistical data from step S32 to adjust the color channels of the non-under-display image so that its color distribution matches that of the under-display image. The specific adjustment formula is as follows:

[0070] Off-screen subplot LAB = {(Off-screen subplot LAB - Off-screen subplot mean) / Off-screen subplot variance * On-screen subplot variance + On-screen subplot mean}

[0071] Using the adjustment formula described above, the color and brightness of the off-screen image in the LAB color space are adjusted, and then converted back to the BGR color space, thus obtaining an off-screen image that is essentially identical to the on-screen image in terms of color and brightness. This step reduces visual inconsistencies between the two images caused by differences in shooting conditions, providing an accurate basis for subsequent pixel-level difference comparison and mask extraction.

[0072] After color and brightness matching is completed in steps S31 to S33, the under-display image processing device performs pixel-level registration between the non-under-display image obtained in step S33 and the under-display image. Please refer to [link to details]. Figure 6 , Figure 6 This is a flowchart illustrating the fourth embodiment of the under-display image processing method provided in this application.

[0073] like Figure 6 As shown, its under-screen image processing method may include the following operations:

[0074] Step S41: Obtain the optical flow field between the off-screen image to be trained and the on-screen image to be trained.

[0075] In one exemplary embodiment, the under-display image processing device calculates the optical flow field between the off-screen image and the under-display image. The optical flow field refers to the apparent motion of the image's grayscale pattern. It is a two-dimensional vector field containing the instantaneous velocity vector information of each pixel. The purpose of studying the optical flow field is to approximate the motion field, which cannot be directly obtained, from a sequence of images.

[0076] In one exemplary embodiment, regarding the calculation of the optical flow field, the under-display image processing device employs an advanced optical flow algorithm to calculate the optical flow field from the off-screen image to the under-display image. The optical flow field is a two-dimensional vector field representing the motion vector of each pixel between the two images, i.e., the displacement in the horizontal direction (x-direction) and the vertical direction (y-direction), that is, the difference between pixels in the two images.

[0077] Step S42: Use optical flow field to register the pixels of the off-screen image to be trained with the pixels of the on-screen image to be trained.

[0078] In one exemplary embodiment, the under-display image processing device generates two-dimensional grid coordinates of the non-under-display image, applies the optical flow field to the non-under-display image to calculate new two-dimensional coordinates that align the non-under-display image with the under-display image, and finally maps the coordinates of the original non-under-display image onto the two-dimensional coordinates to complete the registration with the under-display image.

[0079] Finally, the under-display image processing device remaps the image, using image remapping technology to adjust the off-display image according to the new pixel coordinate matrix, aligning it with the under-display image. This process involves deforming the off-display image to match the pixel layout of the under-display image.

[0080] In one exemplary implementation, Figure 6 For details on the specific technical implementation of using optical flow fields to register under-screen and off-screen images, please refer to [link / reference needed]. Figure 7 , Figure 7This is a flowchart illustrating the fifth embodiment of the under-display image processing method provided in this application.

[0081] like Figure 7 As shown, its under-screen image processing method may include the following operations:

[0082] Step 51: Based on the off-screen image to be trained, generate a two-dimensional grid and determine the initial grid coordinates of each pixel in the off-screen image to be trained.

[0083] In one exemplary embodiment, the under-display image processing device generates a two-dimensional grid coordinate system, creating a two-dimensional grid coordinate matrix using the dimensions of the off-display image. Each intersection of the grid corresponds to a pixel position in the image. For an image of size h (height) × w (width), two matrices, grid_x and grid_y, representing pixel positions are generated.

[0084] Step 52: Calculate the registration grid coordinates of each pixel according to the motion vector of each pixel in the optical flow field and the initial grid coordinates of each pixel.

[0085] In one exemplary embodiment, the under-display image processing device applies an optical flow field, applying the optical flow field calculated in step S41 to the grid coordinates of the non-under-display image to generate new pixel coordinates. The purpose of this step is to move each pixel of the non-under-display image to its corresponding position in the under-display image based on the optical flow field.

[0086] For each pixel position (i, j), the displacement vector (u, v) is used to calculate the new pixel position using the following formula:

[0087] new_x = grid_x + u

[0088] new_y = grid_y + v

[0089] Where u and v represent the horizontal and vertical displacements of the optical flow field at position (i, j), respectively. In this way, the under-screen image processing device obtains new pixel position matrices map_x and map_y for the off-screen image.

[0090] Step 53: Remap the registration grid coordinates of each pixel to the image pixel coordinates to obtain the non-screen image to be trained, which is registered with the screen image to be trained.

[0091] In one exemplary embodiment, the under-display image processing device remaps the image, using image remapping technology to adjust the off-display image according to a new pixel coordinate matrix to align it with the under-display image. This process involves deforming the off-display image to match the pixel layout of the under-display image.

[0092] This application identifies and extracts masks from glare and ghosting areas outside the light source itself, and then uses a large model to repair the masked areas. Obtaining the mask involves taking two images from the same location—one under the screen and one not under the screen—and aligning and registering them to identify the differences that indicate the location of the glare and ghosting, thus obtaining the masked areas. This constructs a paired dataset for extracting glare and ghosting masks. An AI network is then used to train this dataset to obtain a model capable of directly extracting glare and ghosting masks from under-screen images. Finally, the large model is used to repair the masked areas in the input under-screen image, resulting in an optimized under-screen image.

[0093] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0094] To implement the above-described under-display image processing method, this application also proposes an electronic device, for details please refer to [link / reference needed]. Figure 8 , Figure 8 This is a schematic diagram of the structure of an embodiment of the electronic device provided in this application.

[0095] The under-display image processing device 400 of this embodiment includes a processor 41, a memory 42, an input / output device 43, and a bus 44.

[0096] The processor 41, memory 42, and input / output device 43 are respectively connected to the bus 44. The memory 42 stores program data, and the processor 41 is used to execute the program data to implement the under-screen image processing method described in the above embodiments.

[0097] In this embodiment, processor 41 can also be referred to as a CPU (Central Processing Unit). Processor 41 may be an integrated circuit chip with signal processing capabilities. Processor 41 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor can be a microprocessor, or processor 41 can be any conventional processor.

[0098] This application also provides a computer program product; please refer to further details. Figure 9 , Figure 9 This is a schematic diagram of the structure of an embodiment of the computer program product provided in this application. The computer program product 600 stores a computer program 61, which, when executed by a processor, is used to implement the under-screen image processing method of the above embodiment.

[0099] When the embodiments of this application are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0100] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. An under-display image processing method, characterized in that, The under-display image processing method includes: Acquire the under-screen image to be processed; Extract the mask to be processed from the under-screen image to be processed; The under-screen image to be processed and the mask to be processed are input into a large model to repair the image region corresponding to the mask in the under-screen image to be processed, thereby obtaining the target under-screen image.

2. The under-display image processing method according to claim 1, characterized in that, After acquiring the under-screen image to be processed, the method further includes: The contrast of the under-screen image to be processed is enhanced.

3. The under-screen image processing method according to claim 2, characterized in that, The step of enhancing the contrast of the under-screen image to be processed includes: Obtain the grayscale histogram of a standard off-screen image; According to the grayscale histogram, increase the brightness of the dark areas in the under-screen image to be processed, and decrease the brightness of the bright areas in the under-screen image to be processed.

4. The under-display image processing method according to claim 1, characterized in that, The step of extracting the mask to be processed from the under-screen image includes: The mask to be processed for the under-screen image to be processed is extracted using a preset mask extraction model, and the preset mask extraction model is obtained in the following way: Acquire the under-screen image and the off-screen image of the same training region; The training mask is obtained based on the pixel value difference of each pixel at the same position in the under-screen image to be trained and the off-screen image to be trained. The mask to be trained and the under-screen image to be trained are used as the training set to train the mask extraction model.

5. The under-display image processing method according to claim 4, characterized in that, After acquiring the under-screen image and the off-screen image of the same region to be trained, the method further includes: Obtain the average value and standard deviation of the first pixel of the under-screen image to be trained in a preset color space; Obtain the average value of the second pixel and the standard deviation of the second pixel of the off-screen image to be trained in the preset color space; The pixel values ​​of the off-screen image to be trained are adjusted using the first pixel average value, the first pixel standard deviation, the second pixel average value, and the second pixel standard deviation. The training mask is obtained based on the pixel value difference of each pixel at the same position in the under-screen image to be trained and the adjusted off-screen image to be trained.

6. The under-display image processing method according to claim 5, characterized in that, The preset color space is the LAB color space.

7. The under-screen image processing method according to claim 4, characterized in that, After acquiring the under-screen image and the off-screen image of the same region to be trained, the method further includes: Obtain the optical flow field between the off-screen image to be trained and the on-screen image to be trained; The optical flow field is used to register the pixels of the off-screen image to be trained with the pixels of the on-screen image to be trained. The training mask is obtained based on the pixel value difference of each pixel at the same position in the under-screen image to be trained and the registered off-screen image to be trained.

8. The under-display image processing method according to claim 7, characterized in that, The step of registering the pixels of the off-screen image to be trained with the pixels of the on-screen image to be trained using the optical flow field includes: Based on the off-screen image to be trained, a two-dimensional grid is generated, and the initial grid coordinates of each pixel in the off-screen image to be trained are determined. Calculate the registration grid coordinates of each pixel according to the motion vector of each pixel in the optical flow field and the initial grid coordinates of each pixel. By remapping the registration grid coordinates of each pixel to the image pixel coordinates, a non-screen image to be trained, registered with the screen image to be trained, is obtained.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor coupled to the memory; The memory is used to store program data, and the processor is used to execute the program data to implement the under-screen image processing method as described in any one of claims 1 to 8.

10. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the under-display image processing method as described in any one of claims 1 to 8.

Images