An under-screen depth camera

Abstract

An under-display depth camera, located beneath a display screen, includes: a transmitter for emitting infrared laser light towards a target area; an infrared receiver for receiving the reflected signal of the infrared laser light to obtain a first depth map; an RGB camera for receiving signals from the target area and generating a first RGB image; a processor for aligning the first depth map with the first RGB image to obtain a second depth map; removing rainbow-like glare from the first RGB image using a first depth network model based on the second depth map and the first RGB image to obtain a second RGB image; fusing the light sources in the second RGB image with those in the first RGB image to generate a third RGB image; eliminating step defects in the third RGB image using the second depth network model to obtain a fourth RGB image; and fusing the fourth RGB image with the second depth map to obtain an RGBD image. This invention effectively removes rainbow-like glare from under-display depth cameras.

Description

Technical Field

[0001] This invention relates to the field of image processing, and more specifically, to an under-display depth camera. Background Technology

[0002] In recent years, with the rapid development of the full-screen design concept for mobile phones, many manufacturers have been trying to place the front-facing camera behind the display to achieve a truly full-screen effect. Due to the need for data acquisition, front-facing cameras are increasingly becoming depth cameras with RGB lighting. However, placing the front-facing camera behind the display brings new problems, namely a significant decrease in the image quality of the under-display camera. In particular, when there is a strong light source in the field of view, the RGB color image captured by the under-display camera will exhibit severe rainbow-like glare. This glare surrounds the light source in concentric circles, obscuring background information and resulting in a poor user experience.

[0003] Currently, researchers are beginning to study the quality issues caused by under-display cameras, but there is still no specific solution for the rainbow-like glare problem. Previous studies have also investigated how to remove glare from RGB images, but these studies focused on images not captured by under-display cameras, and the glare patterns in these images differ significantly from those in images captured by under-display cameras.

[0004] Traditional glare is formed by the scattering and reflection of strong light by the optical elements within the sensor. However, glare in under-display cameras is caused not only by the characteristics of the optical elements within the sensor but also by changes in the light path due to the display screen. Because the under-display camera is some distance from the screen, and because end-users may apply a protective film, these changes in the optical form of light before it enters the camera are further amplified, resulting in a final image that differs significantly from traditional imaging.

[0005] In under-display images, rainbow glare is wider in range than traditional glare, presenting a closed diffusion pattern, and the light intensity also changes gradually, which is significantly different from traditional rainbows. Therefore, traditional solutions cannot effectively solve the problem of under-display rainbow glare.

[0006] The difficulty of this problem lies in:

[0007] 1. How to remove rainbow-like glare around a light source while preserving the light source itself;

[0008] 2. Ensure a natural transition between the light source edge and the surrounding background after glare removal, thus maintaining an overall natural effect;

[0009] 3. How to recover information obscured by excessive glare in RGB images. Summary of the Invention

[0010] To solve the above problems, or at least one of them, the present invention targets an under-display depth camera, which utilizes the features of the light source region in the depth map and combines them with RGB image features to achieve effective identification of the light source region in three-dimensional space, and can effectively remove rainbow-like glare in RGB images.

[0011] This invention provides an under-display depth camera, located under a display screen, characterized in that it comprises:

[0012] The transmitter is used to emit infrared laser light into the target area.

[0013] An infrared receiver is used to receive the reflected signal of the infrared laser and obtain a first depth map;

[0014] An RGB camera is used to receive signals from the target area and generate a first RGB image;

[0015] The processor is configured to align the first depth map with the first RGB image to obtain a second depth map; based on the second depth map and the first RGB image, remove rainbow-like glare from the first RGB image using a first depth network model to obtain a second RGB image; fuse the light source in the second RGB image with the first RGB image to generate a third RGB image; eliminate step defects in the third RGB image using a second depth network model to obtain a fourth RGB image; and fuse the fourth RGB image with the second depth map to obtain an RGBD image.

[0016] Optionally, the under-display depth camera is characterized in that the RGB camera is located at the optical center of the transmitter and the infrared receiver.

[0017] Optionally, the under-display depth camera is characterized in that padding is applied to the periphery of the second depth map so that the resolution of the second depth map is similar to that of the first RGB... Figure 1 To.

[0018] Optionally, the under-display depth camera is characterized in that the first depth network model extracts information from the first RGB image and information from the second depth map respectively, and after feature fusion transformation, obtains the second RGB image without light source and glare, so as to effectively remove glare and restore the lost information in the second RGB image.

[0019] Optionally, the under-display depth camera is characterized in that, during training, the first depth network model fits multiple light sources through data augmentation.

[0020] Optionally, the under-display depth camera is characterized in that the step of fusing the light source in the second RGB image with the first RGB image includes:

[0021] Step S31: Segment the light source in the first RGB image to generate a mask;

[0022] Step S32: The mask is expanded unevenly, and a weight is set for each expansion. As the mask diffuses from the edge of the light source to the effective edge, the weight gradually decreases from 1 to 0, thus obtaining a single-channel weight alpha map.

[0023] Step S33: Multiply the weighted alpha image with the light source edge diffusion to the effective edge in the first RGB image, and then fuse it with the second RGB image to generate a third RGB image.

[0024] Optionally, the under-display depth camera is characterized by weighting. Where d is the shortest distance between a pixel in the image and the boundary of the light source. max It is the shortest distance between the outermost edge of the diffuse and the boundary of the light source.

[0025] Optionally, the under-display depth camera is characterized by weighting. Where d is the shortest distance between a pixel in the image and the boundary of the light source. max It is the shortest distance between the outermost edge of the diffuse and the boundary of the light source.

[0026] Optionally, the under-display depth camera is characterized in that the third RGB image is calculated according to the fusion formula output = input_1 * mask + input_2 * (1 - mask), wherein input_1 is the RGB value of a pixel in the first RGB image; input_2 is the RGB value of a pixel in the second RGB image; and mask is the weight.

[0027] Optionally, the under-display depth camera is characterized in that the second depth network model is a generative adversarial network model, which includes a generator and a discriminator; in the training set, the real image is labeled as true and the fused image is labeled as false, so as to cross-train the discriminator and the generator; finally, the trained generator is used to adjust the third RGB image to make the details in the transition area smoother.

[0028] Compared with the prior art, the present invention has the following beneficial effects:

[0029] This invention targets under-display depth cameras and uses the depth map obtained by the depth camera to identify rainbow-like glare in RGB images. Since the information in the depth map is different from that in the RGB image, it can achieve a more accurate recognition effect than using two-dimensional images.

[0030] This invention, while obtaining the precise size of the light source, can set the light source diffusion ratio to control the size of the light source area on the RGB image, thereby controlling the proportion of the light source in the RGBD image and adjusting between highlighting and not highlighting the light source to adapt to different application scenarios.

[0031] This invention further processes the fused image to eliminate the stepped defects in under-screen glare, making the image more natural and realistic. Attached Figure Description

[0032] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort. Other features, objects, and advantages of the present invention will become more apparent by reading the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0033] Figure 1 This is a schematic diagram of the structure of an under-display depth camera according to an embodiment of the present invention;

[0034] Figure 2 This is a second depth map in an embodiment of the present invention;

[0035] Figure 3 This is a first RGB image captured by an under-display camera in an embodiment of the present invention;

[0036] Figure 4 This is a second RGB image in an embodiment of the present invention;

[0037] Figure 5 This is a third RGB image in an embodiment of the present invention;

[0038] Figure 6 This is an enlarged third RGB image in an embodiment of the present invention;

[0039] Figure 7 This is a final image in an embodiment of the present invention;

[0040] Figure 8 This is a flowchart illustrating the steps for generating a third RGB image in an embodiment of the present invention;

[0041] Figure 9 This is a weighted curve diagram in an embodiment of the present invention;

[0042] Figure 10 This is another weighting curve diagram in an embodiment of the present invention. Detailed Implementation

[0043] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention. These all fall within the scope of protection of the present invention.

[0044] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0045] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0046] The present invention provides an under-display depth camera, which aims to solve the problems existing in the prior art.

[0047] The technical solutions of the present invention and how they solve the above-mentioned technical problems will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present invention will now be described with reference to the accompanying drawings.

[0048] This invention targets under-display depth cameras and utilizes the characteristics of light source regions in depth maps, combined with RGB image features, to achieve effective identification of light source regions in three-dimensional space, effectively removing rainbow-like glare in RGB images.

[0049] Figure 1 This is a schematic diagram of an under-display depth camera according to an embodiment of the present invention. Figure 1 As shown, an under-display depth camera in an embodiment of the present invention is located under the display screen and includes:

[0050] Transmitter 1 is used to emit infrared lasers toward the target area.

[0051] Specifically, the transmitter can be any type of infrared laser transmitter, such as a speckle projector, a floodlight projector, or a switchable laser projector. This embodiment does not limit the type of transmitter, as long as it is used to emit infrared laser light towards the target area. The target area is the area projected by the transmitter, the signal source area received by the infrared receiver, and also the signal source area received by the RGB camera. The infrared laser can be either structured light or floodlight.

[0052] Infrared receiver 2 is used to receive the reflected signal of the infrared laser and obtain a first depth map.

[0053] Specifically, the infrared receiver operates synchronously with the transmitter to receive signals from the transmitter. When the transmitter is a structured light projector, the infrared receiver is a structured light receiver. When the transmitter is a floodlight projector, the infrared receiver is a Time-of-Flight (TOF) receiver. When the transmitter is a switchable laser projector, the infrared receiver is a TOF receiver. The infrared receiver obtains depth values ​​based on time difference or parallax, thereby generating a first depth map.

[0054] RGB camera 3 is used to receive signals from the target area and generate a first RGB image.

[0055] Specifically, the RGB camera acquires the natural reflection signals of objects within the target area to generate a first RGB image, without relying on actively projected light. The RGB camera and infrared receiver are exposed synchronously to obtain images at the same moment. The RGB camera and infrared receiver receive different wavelength ranges and do not interfere with each other. In some embodiments, the RGB camera is located at the optical center of the transmitter and the infrared receiver, so that the first RGB image is aligned with the first depth map. The transmitter 1, infrared receiver 2, and RGB camera 3 are located on the same straight line.

[0056] Processor 4 is configured to align the first depth map with the first RGB image to obtain a second depth map; based on the second depth map and the first RGB image, remove rainbow-like glare from the first RGB image using a first depth network model to obtain a second RGB image; fuse the second RGB image with the light source in the first RGB image to generate a third RGB image; eliminate step defects in the third RGB image using a second depth network model to obtain a fourth RGB image; and fuse the fourth RGB image with the second depth map to obtain an RGBD image.

[0057] Specifically, when the light source is strong enough, the depth camera located under the screen will capture an RGB image of the light source and the rainbow-like glare surrounding it. Since the wavelength of infrared images is typically above 850nm, such as... Figure 2 As shown, the depth map represents the distance between the target object and the depth camera. Glare from strong light sources is completely invisible in the depth map, so the information in the depth map can be used to help determine the location and range of the light source.

[0058] like Figure 3 As shown, unlike the glare from RGB cameras directly capturing strong light, the glare in images captured by under-display cameras exhibits a wider, continuous diffuse pattern because light passes behind the screen before reaching the camera. Furthermore, the color changes with increasing radius, resembling a rainbow. Further, in Figure 3 In the outer region of the image, the glare is weaker, and information about other objects remains clear, essentially adding a mask to other objects, unlike traditional glare where other objects are not visible. In this embodiment, the first RGB image is captured by a camera under the screen, which is typically a glass screen. The camera is positioned at a certain distance from the glass screen, usually between 0.1mm and 5mm, depending on the application scenario. For example, the distance in a mobile phone's front-facing camera is smaller than the corresponding distance in a robot.

[0059] In some embodiments, padding is applied around the second depth map to make the resolution of the second depth map equal to that of the first RGB. Figure 1 When aligning the first depth map with the first RGB image, the first depth map will be distorted and deformed, especially with data loss in the edge areas. Padding is used to fill in the missing pixels in the RGB image, but the number of pixels cannot exceed that in the first depth map. Padding makes it possible to process the second depth map using the first depth network model.

[0060] In some embodiments, the first deep network model fits multiple light sources during training through data augmentation. This embodiment obtains images with multiple light sources by copying, cutting, overlapping, and adding noise to the light sources. When the training set only contains data for a single light source, this embodiment can simulate the effect of multiple light sources, thereby obtaining image effects under multiple light source conditions and achieving effective identification of multiple light sources with a smaller training set.

[0061] The first deep network model extracts information from the first RGB image and the second depth map, respectively. After feature fusion and transformation, it obtains the second RGB image without light source and glare, effectively removing glare and restoring lost information in the second RGB image. Figure 4As shown, although infrared images can be used to recover background information obscured by glare and remove rainbow-like glare, the color of the center of the recovered light source is somewhat distorted.

[0062] In some embodiments, the first deep network model is trained using a dataset that corresponds one-to-one with a light source being on / off. During training, the dataset consists of RGB images and infrared images for each set of light source on / off conditions. Figure 1 A training set consists of four images in each corresponding data pair. The number of samples in the training set can be any of 10,000, 20,000, 30,000, 40,000, 50,000, or 60,000, or any value not listed. All data in the training set are samples captured under the screen. These samples can be taken by different cameras at different distances from the glass screen, but must exhibit rainbow-like glare. Different cameras will capture images of varying sizes and ranges due to their different parameters. Different distances between the camera and the glass screen will produce rainbow-like glare of varying sizes, resulting in different images. The distance between a strong light source and the glass screen will also affect the size of the rainbow-like glare in the image. The training set, composed of image pairs with rainbow-like glare captured under these various conditions, increases the robustness of the first deep network model. Each data pair consists of an image with rainbow-like glare when the light source is on and a background image without light when the light source is off. The first deep network model can be of any type, including but not limited to convolutional neural networks, recurrent neural networks, and generative adversarial networks.

[0063] When fusing the light source from the second RGB image with that from the first RGB image, various segmentation methods can be used to segment the light source and surrounding information in the first RGB image, including but not limited to: threshold-based segmentation methods, region-based segmentation methods, edge-based segmentation methods, and segmentation methods based on specific theories. This embodiment does not limit the segmentation method, as long as it can segment the light source and surrounding information. Figure 5 As shown, after this fusion step, the distorted light source center is covered by the light source, and the light source and surrounding information can transition naturally.

[0064] A second deep network model is used to further process the third RGB image to eliminate the step-like defects in the third RGB image. For example... Figure 6 As shown, when rainbow-like glare and radial glare are mixed together, there are concentric stepped defects in the transition region, resulting in a strong grainy appearance, and some rainbow-like glare still remains. By using a deep learning model to learn and process the stepped defects, rainbow-like glare can be further removed while simultaneously eliminating the stepped defects.

[0065] In some embodiments, the second deep network model is a generative adversarial network (GAN) model. The GAN is trained using backpropagation. No Markov chains or unfolded approximate inference networks are required during training or sample generation. The GAN includes a generator and a discriminator; in the training set, the real images are labeled "true" and the fused images are labeled "false," thus cross-training the discriminator and the generator; finally, the trained generator is used to adjust the third RGB image, making the details in the transition regions smoother. Figure 7 The image after adjustment according to this embodiment shows that the stepped defects have been eliminated and the rainbow glare has been further reduced, making the overall image look more natural.

[0066] The following is the optimization objective when training a generative adversarial network, where x and y are the fused image and the real image, respectively, and D... Y G and L are the discriminator and generator, respectively. The goal of training a generative adversarial network model is to make L... GAN Maximum value:

[0067]

[0068] Figure 8 This is a flowchart illustrating the steps involved in generating a third RGB image according to an embodiment of the present invention. Figure 8 As shown, a method for generating a third RGB image in an embodiment of the present invention includes:

[0069] Step S31: Segment the light source in the first RGB image to generate a mask.

[0070] In this step, when acquiring the light source, a threshold segmentation method can be used to segment the areas with higher brightness to obtain the light source region. In an RGB image, white is represented by (255, 255, 255), meaning the R, G, and B channels are all 255. During threshold determination, the R, G, and B channels are evaluated separately. When the values ​​of all three channels are greater than a preset threshold, it is identified as a light source region. The preset threshold is a fixed value, including but not limited to 255, 254, 253, 252, 251, and other arbitrarily settable values. The preset threshold is typically greater than 250. The thresholds for the R, G, and B channels can be different. A higher threshold results in a smaller range of the obtained light source region. A lower threshold results in a larger range of the obtained light source region. A mask is then generated for the identified light source.

[0071] Step S32: The mask is expanded unevenly, and a weight is set for each expansion. As the mask diffuses from the edge of the light source to the effective edge, the weight gradually decreases from 1 to 0, thus obtaining a single-channel weight alpha map.

[0072] In this step, the mask expands outward in concentric rings, with each ring assigned a unique weight. The weight at the edge of the light source is 1, and the weight at the end of the diffusion layer is 0. The weights differ between rings, decreasing with distance from the light source. The weight decay is non-linear. Due to the irregular shape of the light source, the expansion is also irregular and uneven. Based on the weight distribution, a single-channel weight alpha map representing the weights can be obtained, which can be used for image fusion calculations. The portion formed by a strong light source on the first RGB image is defined from the inside out as the light source, the light source periphery, and the glare. The light source is a strong region with brightness close to white light, representing the area where the light source itself is located. The light source periphery is the natural area formed when the light source shines outward, not the light diffused by lenses, glass plates, etc. The glare is the glare formed by diffusion from lenses, glass plates, etc., mainly manifested as rainbow-like glare. These three can be distinguished by their brightness values.

[0073] In some embodiments, such as Figure 9 As shown, weights Where d is the shortest distance between a pixel in the image and the boundary of the light source. max It is the shortest distance between the outermost edge of the diffuse and the boundary of the light source. As the number of pixels changes, the value of d ranges from [0, d...]. max The value changes between d and d'. When d = 0, a = 1. When d = d'', a = 1. max When a = 0, this embodiment ensures that the weight changes are more adapted to the human eye's perception of light, resulting in a more realistic fusion effect. max

[0074] In some embodiments, such as Figure 10 As shown, weights Where d is the shortest distance between a pixel in the image and the boundary of the light source. max It is the shortest distance between the outermost edge of the diffuse and the boundary of the light source. As the number of pixels changes, the value of d ranges from [0, d...]. max The value changes between d and d'. When d = 0, a = 1. When d = d'', a = 1. max When a = 0. Compared to the previous embodiments, the weights in this embodiment are linearly related to the distance d, resulting in faster calculation speed and lower resource consumption.

[0075] In some embodiments, the expansion speed in different directions is determined based on the angle of the light source. When the light source is perpendicular to the camera or webcam, the expansion speed in each direction is the same in this step. When the light source is not perpendicular to the camera or webcam, the light source does not appear circular in the image, but rather as an ellipse or other shape. This embodiment can adjust the expansion speed in different directions according to the angle of the light source, making the expansion more uniform.

[0076] In some embodiments, the line width at different positions during expansion is determined based on the angle of the light source. When the light source is perpendicular to the camera or webcam, the line width during expansion in all directions is the same in this step. When the light source is not perpendicular to the camera or webcam, the light source does not appear circular in the image, but rather as an ellipse or other shape. In this embodiment, the line width during expansion in different directions can be adjusted according to the angle of the light source, making the expansion more uniform.

[0077] In some embodiments, the line width at different positions during expansion is determined based on the distance from the center of the light source to its edge. When the light source is perpendicular to the camera or video camera, the line width during expansion in all directions is the same in this step. The closer the center of the light source is to its edge, the thinner the line during expansion; the farther the center of the light source is from its edge, the wider the line during expansion, so that all lines reach the edges surrounding the light source simultaneously.

[0078] Step S33: Multiply the weighted alpha image with the light source edge diffusion to the effective edge in the first RGB image, and then fuse it with the second RGB image to generate a third RGB image.

[0079] In this step, the third RGB image is calculated according to the fusion formula output = input_1 * mask + input_2 * (1 - mask), where input_1 is the RGB value of a pixel in the first RGB image, input_2 is the RGB value of a pixel in the second RGB image, and mask is the weight. This step calculates and fuses the data obtained in the previous steps to generate the third RGB image. When calculating the third RGB image, the calculation is performed according to the position and order of the pixels in the alpha image. For example, in the alpha image, all pixels are calculated sequentially from top to bottom and from left to right; or multiple pixels are calculated row by row or column by column to obtain the fused pixel values ​​corresponding to all alpha images, which are then concatenated with the corresponding pixels in the first RGB image outside the alpha image to form the third RGB image.

[0080] This embodiment uses a method that segments the light source to obtain a pickling film, gradually expands the pickling film outward to obtain a weighted alpha map, and then fuses the images. This method preserves the areas with strong light sources, removes rainbow-like glare, and can be adjusted for different light source angles, making it widely applicable.

[0081] The various embodiments described in this specification are presented in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0082] The specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention.

Claims

1. An under-display depth camera located under a display screen, characterized in that, include: The transmitter is used to emit infrared laser light into the target area. An infrared receiver is used to receive the reflected signal of the infrared laser and obtain a first depth map; An RGB camera is used to receive signals from the target area and generate a first RGB image; A processor is configured to align the first depth map with the first RGB image to obtain a second depth map; Based on the second depth map and the first RGB image, the rainbow-like glare in the first RGB image is removed using a first depth network model to obtain the second RGB image; The second RGB image is fused with the light source in the first RGB image to generate a third RGB image; the step defects in the third RGB image are eliminated using a second deep network model to obtain a fourth RGB image; the fourth RGB image is fused with the second depth map to obtain an RGBD image; the rainbow glare presents a closed diffusion shape, and the light intensity also changes gradually; The step of fusing the light source in the second RGB image with that in the first RGB image includes: Step S31: Segment the light source in the first RGB image to generate a mask; Step S32: The mask is expanded unevenly, and a weight is set for each expansion. As the light source diffuses from the edge to the effective edge, the weight gradually decreases from 1 to 0, thus obtaining a single-channel weight alpha map; the effective edge is the outermost edge of the light source diffusion. Step S33: Multiply the weighted alpha image with the region in the first RGB image where the light source edge diffuses to the effective edge, and then fuse it with the second RGB image to generate a third RGB image; weight where d is the shortest distance between a pixel point on the image and the edge of the light source, is the shortest distance between the effective edge and the edge of the light source.

2. The under-screen depth camera according to claim 1, wherein, The RGB camera is located at the optical center of the transmitter and the infrared receiver.

3. The under-screen depth camera according to claim 1, wherein, Padding is applied around the second depth map to make the resolution of the second depth map the same as that of the first RGB map.

4. The under-screen depth camera of claim 1, wherein, The first deep network model extracts information from the first RGB image and the second depth map respectively. After feature fusion and transformation, it obtains the second RGB image without light source and glare, so as to effectively remove glare and restore the lost information in the second RGB image.

5. The under-screen depth camera according to claim 1, wherein, During training, the first deep network model fits multiple light sources through data augmentation.

6. The under-screen depth camera of claim 1, wherein, The third RGB image is calculated according to the fusion formula output = input_1 * mask + input_2 * (1-mask), where input_1 is the RGB value of a pixel in the first RGB image; input_2 is the RGB value of a pixel in the second RGB image; and mask is the weight.

7. The under-screen depth camera according to claim 1, wherein, The second deep network model is a generative adversarial network model, which includes a generator and a discriminator. In the training set, the real image is labeled as true and the fused image is labeled as false, so as to cross-train the discriminator and the generator. Finally, the trained generator is used to adjust the third RGB image to make the details in the transition region smoother.

Images