Electronic device, method for acquiring image of electronic device, and computer-readable storage medium

Abstract

This application discloses an electronic device and an image acquisition method for an electronic device. The electronic device includes a display screen, a camera, and a processor. The camera is disposed below the display screen and corresponds to the area of the display screen that generates the picture. The camera can capture the light passing through the display screen to take pictures. The processor is used to obtain the actual display picture in the display screen, start the camera to take pictures to obtain the actual captured image, and input the actual display picture and the actual captured image into an image processing model for processing to obtain a target image. The actual captured image contains interference elements related to the actual display picture, and the target image is the image obtained by removing the interference elements from the actual captured image. This application avoids opening holes in the display screen, ensuring a high screen-to-body ratio. Although the actual captured image contains interference elements, the processor inputs the actual captured image and the actual display picture into the image processing model for processing, and obtains a target image with the interference elements removed, and the image quality is better.

Description

Technical Field

[0001] This application relates to the field of imaging technologies, and more particularly, to an electronic device and an image acquisition method for the electronic device. Background Art

[0002] Currently, electronic devices such as mobile phones, iPads, and laptop computers mainly have slots or holes opened on the display screen, and then the camera is installed below the slot or hole, which can avoid the interference of the display screen above the camera on the imaging of the camera when taking pictures. However, this structural design results in a relatively small screen-to-body ratio of the display screen, which is not competitive today when the large screen-to-body ratio is increasingly pursued. To solve the problem of the small screen-to-body ratio of the electronic device, the electronic device uses an under-display camera, that is, the camera is arranged below the display screen, and the area corresponding to the camera is not slotted or perforated. Although this design can ensure the screen-to-body ratio of the electronic device, when the display screen is lit, since the pixels in the display screen emit light out of the display screen and are reflected by the cover plate into the under-display camera, the image quality captured by the under-display camera is poor. Summary of the Invention

[0003] Embodiments of this application provide an electronic device and an image acquisition method for the electronic device, which are used to at least solve the problem of poor image quality.

[0004] The electronic device according to the embodiments of this application includes a display screen, a camera, and a processor. The display screen is used to display to generate a picture. The camera is arranged below the display screen and corresponds to the area of the display screen that generates the picture, and the camera can capture the light passing through the display screen to take pictures. The processor is used to: obtain the actual display picture in the display screen of the electronic device, start the camera located below the display screen to take pictures to obtain an actual captured image, and input the actual display picture and the actual captured image into an image processing model for processing to obtain a target image. The actual captured image contains interference elements related to the actual display picture, and the target image is the image obtained by removing the interference elements from the actual captured image.

[0005] The image acquisition method of the electronic device according to the embodiments of this application includes: obtaining the actual display picture in the display screen of the electronic device; starting the camera located below the display screen to take pictures to obtain an actual captured image, where the actual captured image contains interference elements related to the actual display picture; and inputting the actual display picture and the actual captured image into an image processing model for processing to obtain a target image, where the target image is the image obtained by removing the interference elements from the actual captured image.

[0006] In the electronic device and the image acquisition method of the electronic device according to the embodiments of the present application, the camera is disposed below the display screen, avoiding opening holes in the display screen and ensuring a high screen-to-body ratio of the electronic device. Moreover, since the display screen still displays normally when the camera acquires the actual captured image, although the "actual captured image" contains interference elements related to the actual display screen of the display screen, the processor can input the actual captured image and the actual display screen into the image processing model for processing, so as to obtain a target image after removing the interference elements, and the image quality of the target image is relatively good.

[0007] Additional aspects and advantages of the embodiments of the present application will be given in part in the following description, become apparent in part from the following description, or be learned through the practice of the embodiments of the present application. Description of the Drawings

[0008] The above and / or additional aspects and advantages of the present application will become apparent and be readily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0009] Figure 1 is a schematic flowchart of an image acquisition method of an electronic device according to some embodiments of the present application;

[0010] Figure 2 is a schematic structural diagram of an electronic device according to some embodiments of the present application;

[0011] Figure 3 is another schematic structural diagram of an electronic device according to some embodiments of the present application;

[0012] Figure 4 is a schematic diagram of the principle of an electronic device executing an image acquisition method according to some embodiments of the present application;

[0013] Figure 5 is a schematic flowchart of an image acquisition method of an electronic device according to some embodiments of the present application;

[0014] Figure 6 is a schematic diagram of the principle of an electronic device executing an image acquisition method according to some embodiments of the present application;

[0015] Figure 7 is a schematic flowchart of an image acquisition method of an electronic device according to some embodiments of the present application;

[0016] Figure 8 is a schematic flowchart of an image acquisition method of an electronic device according to some embodiments of the present application;

[0017] Figure 9 is a schematic diagram of the principle of an electronic device executing an image acquisition method according to some embodiments of the present application;

[0018] Figure 10 It is a schematic diagram of the principle for an electronic device according to some embodiments of the present application to execute an image acquisition method;

[0019] Figure 11 It is a schematic diagram of the principle for an electronic device according to some embodiments of the present application to execute an image acquisition method;

[0020] Figure 12 It is a schematic diagram of the connection state between a non - volatile computer - readable storage medium and a processor according to some embodiments of the present application.

[0021] Description of main component symbols:

[0022] Electronic device 100

[0023] Housing 10

[0024] Display screen 20, first area 21, second area 22

[0025] Camera 30

[0026] Processor 40

[0027] Storage unit 50. Specific embodiments

[0028] The following details the embodiments of the present application. The examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals denote the same or similar elements or elements with the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary only for explaining the embodiments of the present application and should not be construed as a limitation to the embodiments of the present application.

[0029] Currently, for electronic devices such as mobile phones, iPads, and laptop computers, a slot or a hole is mainly made on the display screen, and then the camera is installed below the slot or the hole, so as to avoid the interference of the display screen above the camera on the imaging of the camera when taking pictures. However, this structural design results in a relatively small screen - to - body ratio of the display screen, and it is not competitive today when the large screen - to - body ratio is increasingly pursued. To solve the problem of the small screen - to - body ratio of the electronic device, the electronic device adopts an under - screen camera, that is, the camera is arranged below the display screen, and the area corresponding to the camera is not slotted or perforated. Although this design can ensure the screen - to - body ratio of the electronic device, when the display screen is lit, since the pixels in the display screen emit light outwards and are reflected by the cover plate into the under - screen camera, the image quality captured by the under - screen camera is poor. To solve this problem, the present application provides an electronic device 100 and an image acquisition method for the electronic device 100.

[0030] Please refer to Figure 1 and Figure 2, A method for acquiring an image of an electronic device 100 provided by an embodiment of the present application includes:

[0031] 01: Obtain the actual display screen in the display screen 20 of the electronic device 100;

[0032] 03: Activate the camera 30 located below the display screen 20 to take a picture to obtain an actual captured image, and the actual captured image includes interference elements related to the actual display screen; and

[0033] 07: Input the actual display screen and the actual captured image into an image processing model for processing to obtain a target image, where the target image is the image obtained by removing the interference elements from the actual captured image.

[0034] In some embodiments, the electronic device 100 includes a display screen 20, a camera 30, and a processor 40. The display screen 20 is used for displaying to generate a screen. The camera 30 is disposed below the display screen 20 and corresponds to the area of the display screen 20 that generates the screen. The camera 30 can capture the light passing through the display screen 20 to take an image. The processor 40 is used for: obtaining the actual display screen in the display screen 20 of the electronic device 100, activating the camera 30 located below the display screen 20 to take a picture to obtain an actual captured image, and inputting the actual display screen and the actual captured image into an image processing model for processing to obtain a target image, where the actual captured image includes interference elements related to the actual display screen, and the target image is the image obtained by removing the interference elements from the actual captured image.

[0035] Among them, the electronic device 100 may be, but is not limited to, a device with a shooting function and a display function such as a mobile phone, a tablet computer (PAD), a notebook computer, a handheld game console, a smart watch, a smart bracelet, smart glasses, or a smart helmet. In this application, only the case where the electronic device 100 is a mobile phone is taken as an example for illustration.

[0036] Please refer to Figure 2 and Figure 3 , Further, the electronic device 100 further includes a housing 10, and the display screen 20 is installed on the housing 10. In the embodiment of the present application, the housing 10 is the outer shell of the mobile phone, and the display screen 20 and the housing 10 jointly form a receiving cavity 15. Both the camera 30 and the processor 40 are received in the receiving cavity 15.

[0037] The display screen 20 includes a plurality of image pixels for generating a picture (displaying an image). The display screen 20 may include a first area 21 corresponding to the camera 30 and a second area 22 surrounding the first area 21. Among them, the first area 21 is a light-transmitting solid area, and the second area 22 is a non-light-transmitting solid area. The light transmittance of the light-transmitting solid area is greater than or equal to 50%. Both the first area 21 and the second area 22 contain image pixels for displaying data information. That is to say, the first area 21 can still display the picture normally. In one example, the projection of the center point of the first area 21 in the direction perpendicular to the display screen 20 coincides with the projection of the center point of the camera 30 in the direction perpendicular to the display screen 20. Specifically, the projection of the first area 21 in the direction perpendicular to the display screen 20 covers the projection of the lens of the camera 30 in the direction perpendicular to the display screen 20. In this way, light can pass through the first area 21 to reach the camera 30. The first area 21 and the surrounding second area 22 are of the same thickness and continuous. The first area 21 can be arranged at the upper end of the display screen 20, or at the lower end of the display screen 20, or at the middle position of the display screen 20. In the embodiment of the present invention, the first area 21 is arranged at the upper end of the display screen 20.

[0038] Please refer to Figure 3 , the display screen 20 may include a display layer 23 and a cover plate 26 arranged on the display layer 23. The cover plate 26 is fixed to the display layer 23 by full lamination or frame lamination. Specifically, the cover plate 26 and the display layer 23 can be combined through an OCA optical adhesive 25. The display layer 23 can be an OLED (Organic Light-Emitting Diode) display layer, a flexible OLED display layer, or an AMOLED (Active-matrix organic light emitting diode) display layer. In the embodiment of the present application, the display layer 23 is an OLED display layer. Of course, the display layer 23 is not limited to the above types, and a suitable display layer 23 can be selected according to actual needs in practical applications.

[0039] In some embodiments, when designing the display layer 23, a small light-transmitting area 212 can be designed in the display area of the display layer 23, and the corresponding partial area of the cover plate 26 and the light-transmitting area 212 together form the light-transmitting solid area (the first area 21) in the embodiment of the present application, and the upper and lower surfaces of the display layer 23 are still complete surfaces. The cover plate 26 covers the entire display layer 23. It can be understood that the upper and lower surfaces of the cover plate 26 are also still complete surfaces. That is to say, neither the display layer 23 nor the cover plate 26 of the display screen 20 in the embodiment of the present application is provided with an opening for light to pass through. The small light-transmitting area 212 can be designed when producing the display layer 23 for forming the light-transmitting solid area (the first area 21), and at this time, the light-transmitting area 212 can still display the image normally.

[0040] In one example, the cover plate 26 is a touch screen cover plate 26. That is to say, the touch control circuit of the touch screen is integrated in the cover plate 26, and the touch control function can be realized by clicking on the cover plate 26. In some embodiments, the cover plate 26 includes a two-layer structure, namely a touch panel layer and a protective cover plate layer disposed on the touch panel layer. The touch panel layer is used to implement the touch operation of the display screen 20, and the protective cover plate layer is used to protect the touch panel layer to prevent damage to the touch panel layer caused by directly touching the touch panel layer. The cover plate 26 can be made of materials such as glass, sapphire, polyvinyl chloride (PVC), etc. Of course, the material of the cover plate 26 is not limited to the above types, and appropriate materials can be selected to manufacture the cover plate 26 according to actual needs in practical applications.

[0041] Furthermore, the display screen 20 may further include a polarizer 24. The polarizer 24 is disposed between the display layer 23 and the cover plate 26. Since there are image pixels inside the display layer 23, and each image pixel is composed of some electronic components (such as metal electrodes) and electronic circuits, the shiny metal electrodes are similar to mirrors. When a person's eyes stare at the display screen 20, the reflected light of the metal electrodes will have a great impact on the person's eyes. In the display screen 20 of the present application, the polarizer 24 is disposed between the display layer 23 and the cover plate 26. When external light enters the display layer 23 after passing through the polarizer 24, it becomes circularly polarized light. The circularly polarized light is reflected by the metal electrodes and is absorbed by the polarizer 24 when passing through the polarizer 24 again. That is, the polarizer 24 can be used to eliminate the reflected light generated by the reflection of the display layer 23, thereby protecting the comfort of the person's eyes when viewing the display screen 20 of the display screen.

[0042] Please refer to Figure 3 and Figure 4 , when the display screen 20 is working normally, the picture displayed on the display screen 20 is called the actual display picture, that is, as shown in the left picture in Figure 4 . Among them, the actual display picture is any picture displayed on the display screen 20 when using the electronic device 100, including but not limited to various types of static pictures and dynamic pictures (video pictures). When the actual display picture is a static picture, the static picture can be a background picture for locking the screen (such as a solid color background picture, a building background picture, a landscape background picture, a portrait background picture), various photos stored by the user in the album, or an application interface picture when using the mobile phone, etc. When the actual display picture is a dynamic picture, the dynamic picture can be a dynamic video picture, for example: a video picture in the album, a video picture associated with various APPs, a movie picture, or a TV drama picture, etc.

[0043] The camera 30 is located below the display screen 20. When the display screen 20 is not activated, that is, when the display screen 20 is not lit and does not display an image, if the processor 40 controls the camera 30 to take a picture, the image taken by the camera 30 generally will not be affected by the display screen 20, and the quality of the image at this time can meet the user's requirements. However, when the display screen 20 is activated, that is, when the display screen 20 is lit and displays an image, if the processor 40 controls the camera 30 to take a picture, the image taken by the camera 30 generally will be affected by the light emitted by the display screen 20 (for the specific influence, please refer to the introduction of the background technology). At this time, the image taken by the camera 30 is called the "actual captured image". As shown in the middle figure of Figure 4 , the "actual captured image" contains interference elements related to the actual display screen, such as horizontal and vertical stripes or a certain degree of blurriness. At this time, the image quality does not meet the user's requirements. In the image acquisition method of the present application, instead of directly using the actual captured image, after the camera 30 captures the actual captured image, the processor 40 obtains the actual display screen, and inputs the actual captured image and the actual display screen into a pre-set image processing model to obtain a target image, where the target image is the image after removing the interference elements from the actual captured image, that is, the image after removing the horizontal and vertical stripes or eliminating the blurriness. In other words, the quality of the target image is the same as the quality of the image taken by the camera 30 when the display screen 20 is not lit and does not display an image; even better, the quality of the target image is the same as the quality of the image directly captured by the camera 30 (the light entering the camera 30 does not pass through the display screen 20. At this time, the display screen 20 has a slot or hole, and the camera 30 is located below the slot or hole).

[0044] In the electronic device 100 of the present application and the image acquisition method of the electronic device 100, a small area of light-transmitting area 212 is reserved at the position corresponding to the camera 30 during the production of the display screen 20 to form a part of the light-transmitting solid area (the first area 21). Since the cover plate 26 placed in the area directly above the light-transmitting area 212 is transparent, the light passes through the first area 21 and enters the camera 30, realizing that the camera 30 can receive the external light of the electronic device 100 without opening a hole in the display screen 20, ensuring a high screen-to-body ratio of the electronic device 100. Further, since the display screen 20 still displays normally when the camera 30 obtains the actual captured image, although the "actual captured image" contains interference elements related to the actual display screen of the display screen 20, the processor 40 can input the actual captured image and the actual display screen into the image processing model for processing, so as to obtain a target image after removing the interference elements, making the image quality of the target image better.

[0045] In addition, since the projection of the center point of the first region 21 in the direction perpendicular to the display screen 20 coincides with the projection of the center point of the camera 30 in the direction perpendicular to the display screen 20, the camera 30 can obtain more light and thus has a better shooting effect. Moreover, the electronic device 100 reserves a light-transmitting solid region 122 on the display screen 20, avoiding opening holes on the display screen 20 and enhancing the aesthetic feeling of the electronic device 100.

[0046] Please refer to Figure 5 and Figure 6 In some embodiments, 01: obtaining the actual display screen in the display screen 20 of the electronic device 100, including:

[0047] 011: obtaining the actual display screen of the first region 21 in the display screen 20 of the electronic device 100.

[0048] In some embodiments, the processor 40 is further configured to obtain the actual display screen of the first region 21 in the display screen 20 of the electronic device 100.

[0049] The display screen 20 includes a first region 21 and a second region 22 surrounding the first region 21, and the camera 30 only corresponds to the first region 21. Then, most of the interference elements in the actual captured image taken by the camera 30 are caused by the actual display screen of the first region 21 ( Figure 6 the rightmost figure in). Then, when obtaining the target image, on the one hand, directly inputting the actual display screen of the first region 21 and the actual captured image into the image processing model for processing makes the image processing more accurate and the interference elements can be found more accurately, thereby improving the quality of the target image. On the other hand, only inputting the actual display screen of the first region 21 and the actual captured image into the image processing model for processing, avoiding processing the entire-screen display image of the display screen 20 ( Figure 6 the middle figure in), and only processing the image displayed in the first region 21 can reduce the data volume of image processing and improve the image processing efficiency.

[0050] Please refer to Figure 7 In some embodiments, the image acquisition method of the electronic device 100 may further include:

[0051] 05: obtaining an image processing model through model training.

[0052] Please refer to Figure 2 In some embodiments, the processor 40 is further configured to obtain an image processing model through model training.

[0053] In some embodiments, the electronic device 100 may further include a storage unit 50. The image processing model may be trained before the electronic device 100 leaves the factory and stored in the storage unit 50. When the subsequent processor 40 needs to use it, that is, when executing the method in 07, it can be read from the storage unit 50. In this case, the image processing model is obtained by training with a large amount of data, and the amount of sample data is very large, which can basically adapt to various actual shooting situations, and can also adapt to multiple electronic devices 100 of the same model, simplifying the production process. This embodiment uses the method of directly calling the existing image processing model to obtain the target image, which can save the user's photo-taking time and provide a good user experience.

[0054] However, as time goes by, the internal structure of many electronic devices 100 may change significantly compared to when they left the factory due to impacts, drops, etc. Using the image processing model at the time of factory production may no longer be suitable; or, due to the use environment, the differences between multiple electronic devices 100 of the same model become larger and larger. When they use the same image processing model, the effects of the captured target images will be uneven, which may lead to dissatisfaction among different users of the same model of electronic device 100; or, even if the internal environment of the electronic device 100 has not changed and different users of the same model of electronic device 100 are not dissatisfied with the image quality differences, there will always be differences between the image processing model trained before the electronic device 100 leaves the factory and the actual shooting scene. These differences themselves will cause the image processing model not to always be adapted, and using an inadapted image processing model will result in the obtained target images not all being high-quality images.

[0055] To solve these problems, in some other embodiments, the electronic device 100 may store an initial image processing model in the storage unit 50 before leaving the factory. The initial image processing model is trained before leaving the factory as in the above embodiments. And each time during actual shooting, various relevant data will be stored in the storage unit 50. After the data accumulates to a predetermined quantity, a new data processing model will be trained according to the stored data and updated and stored in the storage unit 50. When taking pictures later (executing the method in 07), the updated data processing model can be used for image processing. And before the data accumulates to a certain predetermined quantity, the initial processing model can be used for image processing. Of course, during each subsequent actual shooting, various relevant data can be stored in the storage unit 50. After the data accumulates to the next predetermined quantity, a second new data processing model will be trained according to the total stored data (a total of two predetermined quantities) and updated and stored in the storage unit 50. When taking pictures later (executing the method in 07), the image processing model updated for the second time can be used for image processing. And before the data accumulates to the next predetermined quantity, the image processing model updated for the first time is still used for image processing. Iterating like this, the image processing model can always be adapted to the internal structure, external environment, and usage scenario of the electronic device 100, so that the quality of the finally obtained target image can always be maintained at a relatively high level.

[0056] Please refer to 8. In some embodiments, 05: obtaining an image processing model through model training includes:

[0057] 051: obtaining different images displayed on the display screen 20 multiple times as multiple frames of training display images;

[0058] 053: simulating multiple frames of interference images with interference elements according to the pixel structure of the display screen 20, multiple frames of training display images, and direct-shot images. Each frame of interference image corresponds to a frame of training display image and a frame of direct-shot image. The direct-shot image is an image directly captured by the camera 30 without being blocked by the display screen 20. The interference elements include one of horizontal and vertical stripes and blur; and

[0059] 055: training an image processing model according to multiple frames of training display images, multiple frames of interference images, and direct-shot images.

[0060] Please refer to Figure 2, in some embodiments, the processor 40 is further configured to: obtain different images displayed on the display screen 20 multiple times as multiple frames of training display images; simulate multiple frames of interference images with interference elements according to the pixel structure of the display screen 20, the multiple frames of training display images, and the directly captured images, where each frame of interference image corresponds to one frame of training display image and one frame of directly captured image, the directly captured image is an image directly captured by the camera 30 without being blocked by the display screen 20, and the interference elements include one of horizontal and vertical stripes and blur; and train an image processing model according to the multiple frames of training display images, the multiple frames of interference images, and the directly captured images.

[0061] As described above, obtaining the image processing model through model training can be completed before the electronic device 100 leaves the factory, or can be executed during the use of the electronic device 100. That is, 051, 053, and 055 in this embodiment can all be completed before the electronic device 100 leaves the factory, or can all be executed during the use of the electronic device 100.

[0062] Please combine Figure 9 and Figure 10 , the processor 40 can first control the display screen 20 to present different images. For example Figure 10 When the first frame of the first row is an image without stray light interference (the display screen 20 is not lit), the interface presented by the display screen 20 can be referred to as Figure 9 "Training display image 1" in the leftmost first row of Figure 10 When the second frame of the first row is an image when the display screen 20 emits white light, the image displayed by the display screen 20 can be referred to as Figure 9 "Training display image 2" in the leftmost second row of Figure 10 When the third frame of the first row is an image when the display screen 20 emits gray light, the image displayed by the display screen 20 can be referred to as Figure 9 "Training display image 3" in the leftmost third row of Figure 10 When the fourth frame of the first row is an image when the display screen 20 emits yellow light, the image displayed by the display screen 20 can be referred to as Figure 9 "Training display image 4" in the leftmost fourth row of Figure 10 When the fifth frame of the first row is an image when the display screen 20 emits blue light, the image displayed by the display screen 20 can be referred to as Figure 9 "Training display image 5" in the leftmost fifth row of. It should be noted that due to the limitation of the drawing boundary, "Training display image 4" and "Training display image 5" are not presented in Figure 9 , but they can actually exist.

[0063] In method 053, in one example, the "direct capture image" can be an image captured by the camera 30 of the electronic device 100 when the display screen 20 is not lit, such as Figure 10 the image in the third line in. At this time, the "direct capture image" can be implemented when the display screen 20 and the camera 30 are assembled together, that is, the method of 053 can be implemented during the use of the electronic device 100. Among them, "direct capture image 1" to "direct capture image n" can be the same. For example, they are all images obtained by the electronic device 100 using the camera 30 to take pictures of the same face; or, "direct capture image 1" to "direct capture image n" can be different. For example, they are images obtained by the electronic device 100 using the camera 30 to take pictures of n different faces respectively; or some of "direct capture image 1" to "direct capture image n" can be the same and some can be different.

[0064] In another example, the "direct capture image" can be an image directly captured by the camera 30 when the camera 30 and the display screen 20 are not assembled together, that is, the light entering the camera 30 does not pass through the display screen 20 but directly enters the camera 30 for imaging, such as Figure 10 the image in the third line in. That is, the method of 053 can be implemented during the process of training the image processing model before the electronic device 100 leaves the factory. Similarly, "direct capture image 1" to "direct capture image n" can be the same; or, "direct capture image 1" to "direct capture image n" can be different; or some of "direct capture image 1" to "direct capture image n" can be the same and some can be different.

[0065] Please refer to Figure 11 , the pixel structure of the display screen 20 is known, mainly including the distribution of the pixel structure, and can be pre-stored in the storage unit 50. After obtaining multiple frames of "training display screens" and multiple frames of "direct capture images", interference images with interference elements can be simulated together with the known pixel structure of the display screen 20 ( Figure 10 the image in the second line in).

[0066] It should be noted that there is a one-to-one correspondence among the "training display screen", the "direct capture image", and the "interference image". As Figure 9 shown, "training display screen 1", "direct capture image 1", and "interference image 1" are in one-to-one correspondence, "training display screen 2", "direct capture image 2", and "interference image 2" are in one-to-one correspondence, "training display screen 3", "direct capture image 3", and "interference image 3" are in one-to-one correspondence, and "training display screen n", "direct capture image n", and "interference image n" are in one-to-one correspondence.

[0067] To achieve a one-to-one correspondence, the following first method can be adopted when obtaining the "training display screen", "direct shot image", and "interference image" in one-to-one correspondence: Keep the electronic device 100 stationary. When there is no stray light interference (the display screen 20 is not lit), obtain the interface presented on the display screen 20 as the "training display screen 1". Then, keep the display screen 20 unlit, and the electronic device 100 captures an image through the camera 30. This image is the "direct shot image 1" and can also be used as the subsequent "direct shot image 2" to "direct shot image n". Next, the processor 40 controls the display screen 20 to emit white light and obtains the image displayed on the display screen as the "training display screen 2". Then, the processor 40 controls the display screen 20 to emit gray light and obtains the image displayed on the display screen as the "training display screen 3". Then, the processor 40 controls the display screen 20 to emit yellow light and obtains the image displayed on the display screen as the "training display screen 4". Then, the processor 40 controls the display screen 20 to emit blue light and obtains the image displayed on the display screen as the "training display screen 5", until the "training display screen n" is obtained. Thus, n frames of "direct shot images" and n frames of "training display screens" in one-to-one correspondence can be obtained. Among them, the n frames of "direct shot images" can be completely identical images. Finally, using the n frames of "direct shot images" and n frames of "training display screens" in one-to-one correspondence, and the pixel structure of the display screen 20, Figure 11 train n frames of interference images in the manner of

[0068] Of course, the following second method can also be used to obtain the one-to-one corresponding "display screen for training", "direct shot image", and "interference image": When there is no stray light interference (the display screen 20 is not lit), the interface presented by the display screen 20 is obtained as the "display screen for training 1", and then the display screen 20 remains unlit. The electronic device 100 captures an image through the camera 30, and this image is the "direct shot image 1"; then, the electronic device 100 is moved. The processor 40 first controls the display screen 20 to be unlit, and the electronic device 100 captures the "direct shot image 2" through the camera 30. Then, the processor 40 controls the display screen 20 to emit white light and obtains the image displayed on the display screen 20 as the "display screen for training 2"; then, the electronic device 100 is moved. The processor 40 first controls the display screen 20 to be unlit, and the electronic device 100 captures the "direct shot image 3" through the camera 30. Then, the processor 40 controls the display screen 20 to emit gray light and obtains the image displayed on the display screen 20 as the "display screen for training 3"; then, the electronic device 100 is moved. The processor 40 first controls the display screen 20 to be unlit, and the electronic device 100 captures the "direct shot image 4" through the camera 30. Then, the processor 40 controls the display screen 20 to emit yellow light and obtains the image displayed on the display screen 20 as the "display screen for training 4"; then, the electronic device 100 is moved. The processor 40 first controls the display screen 20 to be unlit, and the electronic device 100 captures the "direct shot image 5" through the camera 30. Then, the processor 40 controls the display screen 20 to emit blue light and obtains the image displayed on the display screen 20 as the "display screen for training 5", until the "display screen for training n" is obtained. Thus, n frames of one-to-one corresponding "direct shot images" and n frames of "display screens for training" can be obtained. Among them, the n frames of "direct shot images" can be completely different or partially the same, with partially the same images. Finally, by using the n frames of one-to-one corresponding "direct shot images" and n frames of "display screens for training", and the pixel structure of the display screen 20, Figure 11 the method in

[0069] In some embodiments, when the display screen for training includes picture areas of different colors, the interference elements corresponding to the picture areas of different colors are different.

[0070] Specifically, please refer to Figure 10, when the training display screen includes a white screen area, the interference elements corresponding to the white screen area include horizontal and vertical stripes and a first blur degree. The horizontal and vertical stripes have a first thickness, and the blur has a first degree. When the training display screen includes a gray screen area, the interference elements corresponding to the gray screen area include horizontal and vertical stripes and a second blur degree. The horizontal and vertical stripes have a second thickness, the second thickness is less than the first thickness, and the second blur degree is less than the first blur degree. When the training display screen includes a yellow screen area, the interference elements corresponding to the yellow screen area include horizontal and vertical stripes and a third blur degree. The horizontal and vertical stripes have a third thickness, the third thickness is less than the first thickness, and the third blur degree is less than the first blur degree. When the training display screen includes a blue screen area, the interference elements corresponding to the blue screen area include horizontal and vertical stripes and a fourth blur degree. The horizontal and vertical stripes have a fourth thickness, the fourth thickness is less than the first thickness, and the fourth blur degree is less than the first blur degree.

[0071] Please refer to Figure 12 , a non-volatile computer-readable storage medium 200 storing a computer program 202 according to an embodiment of the present application. When the computer program 202 is executed by one or more processors 40, the processor 40 can execute the image generation method of any of the above embodiments.

[0072] For example, please combine Figure 1 , when the computer program 202 is executed by one or more processors 40, the processor 40 executes the following method:

[0073] 01: Obtain the actual display screen in the display screen 20 of the electronic device 100;

[0074] 03: Start the camera 30 located below the display screen 20 to take a picture to obtain an actual captured image, and the actual captured image includes interference elements related to the actual display screen; and

[0075] 07: Input the actual display screen and the actual captured image into an image processing model for processing to obtain a target image, and the target image is the image obtained by removing the interference elements from the actual captured image.

[0076] For another example, please combine Figure 5 , when the computer program 202 is executed by one or more processors 40, the processor 40 can also execute the following method:

[0077] 011: Obtain the actual display screen of the first area 21 in the display screen 20 of the electronic device 100.

[0078] For another example, please combine Figure 7 , when the computer program 202 is executed by one or more processors 40, the processor 40 can also execute the following method:

[0079] 05: Obtain an image processing model through model training.

[0080] For another example, please combine Figure 8 , when the computer program 202 is executed by one or more processors 40, the processor 40 can also execute the following method:

[0081] 051: Obtain different images displayed on the display screen 20 multiple times as multiple frames of training display images.

[0082] 053: Simulate multiple frames of interference images with interference elements according to the pixel structure of the display screen 20, multiple frames of training display images, and direct capture images. Each frame of interference image corresponds to one frame of training display image and one frame of direct capture image. The direct capture image is an image directly captured by the camera 30 without being blocked by the display screen 20. The interference elements include one of horizontal and vertical stripes and blur. And

[0083] 055: Train an image processing model according to multiple frames of training display images, multiple frames of interference images, and direct capture images.

[0084] In the description of this specification, the descriptions referring to terms such as "certain embodiments", "in an example", "exemplarily", etc. mean that the specific features, structures, materials or characteristics described in connection with the embodiments or examples are included in at least one embodiment or example of the present application. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiments or examples. Moreover, the specific features, structures, materials or characteristics described can be combined in a suitable manner in any one or more embodiments or examples. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0085] Any process or method description shown in the flowchart or described in other ways herein can be understood as representing a module, segment or part of the code of executable instructions including one or more steps for implementing a specific logical function or process. The scope of the preferred embodiments of the present application includes additional implementations, where the functions can be executed in a substantially simultaneous manner or in a reverse order according to the involved functions, rather than in the order shown or discussed, which should be understood by those skilled in the technical field to which the embodiments of the present application belong.

[0086] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present application. Those of ordinary skill in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present application.

Claims

1. An electronic device, characterized in that, Comprising: A display screen for displaying to generate a picture; A camera, disposed below the display screen and corresponding to the area of the display screen that generates the picture, the camera being capable of capturing light passing through the display screen to take an image; And A processor for obtaining the actual display picture in the display screen of the electronic device, starting the camera located below the display screen to take a picture to obtain an actual captured image, and inputting the actual display picture and the actual captured image into an image processing model for processing to obtain a target image, the actual captured image containing interference elements related to the actual display picture, the target image being an image obtained by removing the interference elements from the actual captured image; the image processing model is trained according to multiple frames of training display pictures, multiple frames of interference images, and direct capture images, the multiple frames of training display pictures including the picture displayed when the display screen is not lit, and the pictures displayed when the display screen emits different colors of light, the direct capture images including the images directly captured by the camera without being blocked by the display screen.

2. The electronic device according to claim 1, characterized in that The display screen includes a first area corresponding to the camera and a second area surrounding the first area, and the processor is further configured to obtain the actual display picture of the first area in the display screen of the electronic device.

3. The electronic device according to claim 1, wherein The processor is further configured to: Obtain different pictures presented by the display screen multiple times as multiple frames of training display pictures; Simulate multiple frames of interference images with the interference elements according to the pixel structure of the display screen, multiple frames of the training display pictures, and the direct capture images, each frame of the interference images corresponding to one frame of the training display picture and one frame of the direct capture image, the interference elements including one of horizontal and vertical stripes and blur; And Train the image processing model according to multiple frames of the training display pictures, multiple frames of the interference images, and the direct capture images.

4. The electronic device according to claim 3, wherein When the training display pictures include picture areas of different colors, the interference elements corresponding to the picture areas of different colors are different.

5. An image acquisition method for an electronic device, characterized in that, Comprising: Obtain the actual display picture in the display screen of the electronic device; Start the camera located below the display screen to take a picture to obtain an actual captured image, the actual captured image containing interference elements related to the actual display picture; And Input the actual display picture and the actual captured image into an image processing model for processing to obtain a target image, the target image being an image obtained by removing the interference elements from the actual captured image; the image processing model is trained according to multiple frames of training display pictures, multiple frames of interference images, and direct capture images, the multiple frames of training display pictures including the picture displayed when the display screen is not lit, and the pictures displayed when the display screen emits different colors of light, the direct capture images including the images directly captured by the camera without being blocked by the display screen.

6. The image acquisition method according to claim 5, wherein The display screen includes a first area corresponding to the camera and a second area surrounding the first area, and obtaining the actual display picture in the display screen of the electronic device includes: Obtain the actual display picture of the first area in the display screen of the electronic device.

7. The image acquisition method according to claim 5, characterized in that The training process of the image processing model includes: Obtaining different pictures presented by the display screen multiple times as multiple frames of training display pictures; Simulating multiple frames of interference images with the interference elements according to the pixel structure of the display screen, multiple frames of the training display pictures, and the directly captured images, where each frame of the interference image corresponds to one frame of the training display picture and one frame of the directly captured image, and the interference elements include one of horizontal and vertical stripes and blur; and Training the image processing model according to multiple frames of the training display pictures, multiple frames of the interference images, and the directly captured images.

8. The image acquisition method according to claim 7, wherein When the training display pictures include picture areas of different colors, the interference elements corresponding to the picture areas of different colors are different.

9. A non-volatile computer-readable storage medium storing a computer program, when the computer program is executed by one or more processors, the processors implement the image acquisition method according to any one of claims 5 to 8.

Images