Image processing method and apparatus, electronic device, and readable storage medium

Abstract

To provide an image processing method that is adopted to an electronic apparatus having an under screen camera installed under a display, the electronic apparatus, and a readable storage medium.SOLUTION: The method includes the steps of: acquiring an initial image photographed by an under screen camera at a plurality of different field angles; determining first image information where an influence of a diffraction phenomenon occurs in a plurality of initial images on the basis of difference information between the plurality of initial images; and performing complementary processing against the first image information and obtaining a target image on the basis of second image information in which the influence of the diffraction phenomenon of the plurality of initial images does not occur.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly to an image processing method and apparatus, an electronic device, and a readable storage medium.

Background Art

[0002] Cameras of electronic devices such as smartphones include a rear camera installed on the back side of the display for main shooting, and a front camera mainly installed on the same side as the display for selfies. Since the front camera is installed on the same side as the display, the installation location of the front camera needs to avoid the display, so it is difficult to realize a full-screen design with the entire surface of the electronic device being the display. To realize the full-screen design of the electronic device, a bezel-type design in which the front camera is installed in the bezel portion outside the display can be adopted, but there is a problem that it is difficult to operate depending on the position of the camera. There are also notch-type, dot-drop-type, and punch-hole-type designs in which a hole is opened in a part of the display to install the front camera, but a good full-screen design cannot be guaranteed. In addition, there is also a problem of poor reliability in a pop-up-type design in which the front camera is stored in the side wall of the electronic device and popped out at the time of shooting.

[0003] To further realize the full-screen design of the electronic device, a design in which the front camera is installed under the display is increasingly applied. The transmittance of the display directly above the front camera is increased to ensure the amount of light required for shooting, but increasing the transmittance of the display leads to a decrease in the display contrast of the display and a deterioration in the display quality, and there are problems such as low image quality due to the diffraction effect in the captured image of the under-screen camera.

Summary of the Invention

Problems to be Solved by the Invention

[0004] At least one embodiment of the present invention provides an image processing method and apparatus, an electronic device, and a readable storage medium that can improve the quality of captured images of an under-screen camera.

Means for Solving the Problems

[0005] In order to solve the above-described technical problems, the present invention is realized as follows.

[0006] As a first aspect, an embodiment of the present invention provides an image processing method applicable to an electronic device having an under-screen camera installed under a display. The method includes: obtaining initial images captured by the under-screen camera at a plurality of different shooting angles; determining first image information in which a diffraction phenomenon has occurred in the plurality of initial images based on difference information between the plurality of initial images; and performing a complement processing on the first image information based on second image information in which no diffraction phenomenon has occurred in the plurality of initial images to obtain a target image.

[0007] As a second aspect, an embodiment of the present invention provides an image processing apparatus applicable to an electronic device having an under-screen camera installed under a display. The apparatus includes: an acquisition module for obtaining initial images captured by the under-screen camera at a plurality of different shooting angles; a first processing module for determining first image information in which a diffraction phenomenon has occurred in the plurality of initial images based on difference information between the plurality of initial images; and a second processing module for performing a complement processing on the first image information based on second image information in which no diffraction phenomenon has occurred in the plurality of initial images to obtain a target image.

[0008] As a third aspect, an embodiment of the present invention provides an electronic device including a processor, a memory, and a program or instruction stored in the memory and executable by the processor. When the program or instruction is executed by the processor, it realizes the steps of the image processing method described in the first aspect above.

[0009] As a fourth aspect, an embodiment of the present invention provides a readable storage medium storing a program or instructions, and when the program or instructions are executed by a processor, the steps of the image processing method described in the above-mentioned first aspect are realized.

Advantages of the Invention

[0010] Compared with the prior art, in the case of an electronic device having an under-screen camera installed under a display, the image processing method, apparatus, electronic device, and readable storage medium according to the embodiments of the present invention acquire initial images taken by the under-screen camera at a plurality of different viewing angles, and based on the difference information between the plurality of initial images, determine first image information in which the influence of the diffraction phenomenon occurs in the plurality of initial images, and based on second image information in which the influence of the diffraction phenomenon does not occur in the plurality of initial images, perform a complement processing on the first image information to obtain a target image, thereby guaranteeing the shooting image quality of the under-screen camera, reducing the influence on the image quality caused by the diffraction phenomenon, and increasing the transmittance of the display to improve the shooting image quality of the under-screen camera, thereby avoiding the problem that the display effect of the display is reduced.

Brief Description of the Drawings

[0011]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Embodiments for Carrying Out the Invention

[0012] Hereinafter, with reference to the accompanying drawings in the embodiments of the present application, the technical means in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are part of, but not all of, the embodiments of the present application. All other embodiments that can be achieved by those skilled in the art without creative work based on the embodiments in the present application belong to the protection scope of the present application.

[0013] The terms "first", "second", etc. in the description and claims of this application are for distinguishing similar objects and do not necessarily explain a specific order or priority. It should be understood that the data used in this way may be appropriately interchanged so that the embodiments of this application can be implemented in an order other than the illustration or description here. Moreover, the objects distinguished by the terms "first", "second", etc. are usually of the same kind and do not limit the number of objects. For example, the first object may be one or a plurality. Note that "and / or" in the description and claims represents at least one of the connected objects. The character " / " generally indicates that the related objects before and after are in an "or" relationship.

[0014] In order to further realize the full-screen design of electronic devices, an under-screen camera has been proposed that installs a front camera under the display and takes selfies through the display. Regarding the research of the under-screen camera, if the transmittance of the display directly above the front camera can be increased to ensure the amount of light required for shooting, and at the same time, the display contrast of the display during use can be sufficiently obtained, there will be no problems such as cost, thickness, and reliability, and an electronic device with a full-screen design can be realized.

[0015] In order to take pictures through the display, it is necessary to increase the transmittance of the display at the part where the camera is installed. However, when the transmittance of the display is increased, the display contrast decreases and the display quality deteriorates. There is a problem that the uniformity with other parts is impaired, so there is a limit to increasing the transmittance of the display. Therefore, the demerits of the under-screen camera include the following problems of image quality degradation. (1) Sensitivity reduction due to a decrease in the transmitted light amount of the display. (2) Resolution degradation due to the diffraction effect of the display pixels. (3) Generation of flare and ghost due to light scattering in the display constituent layers. (4) Generation of color shift, artifacts, false colors, etc. due to the wavelength dependence of the display transmittance and the phase change in the constituent layers. (5) Image distortion due to the non-uniformity of the display constituent layers.

[0016] In view of these problems, embodiments of the present application provide an image processing method and apparatus, an electronic device, and a readable storage medium, so as to avoid the transmittance loss of the display and improve the captured image quality of the under-screen camera.

[0017] As shown in FIG. 1, embodiments of the present invention provide an image processing method applied to an electronic device having an under-screen camera installed under a display. The method specifically includes the following steps.

[0018] In step 11, obtain initial images captured by the under-screen camera at a plurality of different shooting angles.

[0019] Optionally, the under-screen camera is movable under the display of the electronic device. For example, the moving direction of the under-screen camera under the display includes one dimension (X or Y direction) or two dimensions (X, Y direction). Here, when the under-screen camera moves to different positions under the display, the shooting angle of its captured image is different. Therefore, the under-screen camera can obtain initial images captured at a plurality of different shooting angles.

[0020] In step 12, based on the difference information between the plurality of initial images, determine first image information in which the influence of the diffraction phenomenon occurs in the plurality of initial images.

[0021] Optionally, the difference information indicates the difference between the plurality of initial images, such as the difference information between the plurality of initial images obtained by, for example, alignment processing. Specifically, before step 12, align the plurality of initial images based on the feature point information of each initial image and the movement amount information between different initial images, and determine the difference information between the plurality of initial images based on the plurality of initial images after the alignment processing. Here, the movement amount information is the movement amount information of the under-screen camera when shooting different initial images.

[0022] Optionally, after obtaining the difference information between the plurality of initial images, it is further possible to detect the difference in the occurrence state of the diffraction phenomenon based on the extracted filter coefficients, and only extract the image information affected by the diffraction phenomenon. Specifically, step 12 includes determining first image information in which the diffraction phenomenon has an impact on the plurality of initial images based on the difference information and filter coefficients between the plurality of initial images. Here, the filter coefficients are determined based on the difference level associated with the exposure information of the underscreen camera. For example, the corresponding filter coefficients are extracted according to the difference level linked to the exposure information of the underscreen camera.

[0023] In step 13, based on the second image information in which the diffraction phenomenon has no impact on the plurality of initial images, complementary processing is performed on the first image information to obtain a target image.

[0024] Optionally, step 13 includes determining second image information in which the diffraction phenomenon has no impact corresponding to the target image region on the plurality of initial images based on the target image region corresponding to the first image information, and performing complementary processing on the first image information based on the second image information to obtain the target image.

[0025] In the above aspect, in the case of an electronic device having an under-screen camera installed under the display, the under-screen camera acquires initial images taken at a plurality of different shooting angles, and based on the difference information between the plurality of initial images, determines first image information in which the diffraction phenomenon affects the plurality of initial images, and based on second image information in which the diffraction phenomenon does not affect the plurality of initial images, performs a complement processing on the first image information to obtain a target image, thereby guaranteeing the shooting image quality of the under-screen camera, reducing the influence on the image quality caused by the diffraction phenomenon, and increasing the transmittance of the display to improve the shooting image quality of the under-screen camera, thereby avoiding the problem that the display effect of the display is reduced.

[0026] Optionally, acquiring the initial images taken by the under-screen camera at a plurality of different shooting angles includes acquiring the initial images respectively taken when the under-screen camera moves to a plurality of different positions along a first direction, or acquiring the initial images respectively taken with the highest contrast when the under-screen camera moves to a plurality of different positions along the first direction, or acquiring the initial images respectively taken when the under-screen camera moves to a plurality of different positions along a first direction and a second direction, or acquiring the initial images respectively taken with the highest contrast when the under-screen camera moves to a plurality of different positions along the first direction and the second direction.

[0027] Here, the first direction and the second direction are different directions parallel to the display. For example, the first direction and the second direction are directions perpendicular to each other such that the first direction is one of the X and Y directions and the second direction is the other of the X and Y directions.

[0028] Optionally, the fact that the under-screen camera has moved to a plurality of different positions includes that the under-screen camera has moved to a plurality of different positions with a fixed movement amount respectively, or includes that the under-screen camera has moved to a plurality of different positions with a movement amount determined by contrast respectively. For example, the under-screen camera moves with a fixed movement amount in a direction parallel to the display so as to acquire images with different viewing angles. Here, the fixed movement amount is set by the system or defined by the user himself, and in the embodiments of the present application, it is not limited thereto. Also, for example, the contrast component can be measured to obtain the difference in diffraction effects due to the position of the under-screen camera, and thereby, the movement amount of the under-screen camera can be set based on the contrast component to acquire images with different viewing angles. Embodiment 1

[0029] In the embodiments of the present application, taking the electronic device being a smartphone as an example, a front under-screen camera is installed under the display, and a plurality of cameras with different received light wavelengths are used. As shown in FIG. 2, a high-transmission region 210 with enhanced light transmittance is provided at a position above the under-screen camera 22 in the display 21. In this way, the camera is arranged under the high-transmission region of the display, and receives the light transmitted through the high-transmission region of the display to perform imaging.

[0030] Optionally, the under-screen camera is movable within the high-transmission region under the display. FIG. 3 shows an apparatus for changing the viewing angle of the under-screen camera. The under-screen camera 22 moves with respect to the planar space where the display exists (for example, in the horizontal direction X), and it is possible to acquire images with different viewing angles of the camera. For example, the arrow X0 is the horizontal change direction of the viewing angle.

[0031] Specifically, as one implementation method, it is as follows. FIG. 4 shows a driving device for an ultrasonic motor. An ultrasonic motor 41 for driving the movement of the under-screen camera 22 is arranged on one horizontal side in the moving direction of the under-screen camera 22, and a guide 42 for supporting the under-screen camera 22 and a position detection magnet 43 for detecting the position (movement amount) are arranged opposite to it. A magnetic sensor 44 is arranged on the same horizontal plane as the position detection magnet 43 of the under-screen camera 22. FIG. 5 shows a flowchart of the movement driving of the under-screen camera by the ultrasonic motor. Specifically, the position (movement amount) is calculated by feedback control by detecting the magnetic field of the position detection magnet 43 with the magnetic sensor 44, and the under-screen camera 22 is driven to move by the ultrasonic motor 41.

[0032] Another implementation method is as follows. FIG. 6 shows a driving device for a stepping motor. A driving stepping motor 61 is arranged on one horizontal side in the moving direction of the under-screen camera 22, and a guide 42 for supporting the under-screen camera 22 is arranged opposite to it. FIG. 7 shows a flowchart of the movement driving of the under-screen camera by the stepping motor. Specifically, the position (movement amount) is calculated by feedforward control by monitoring the pulses generated by the stepping motor 61 for driving, and the under-screen camera 22 is driven to move by the stepping motor 61.

[0033] FIG. 8 shows a diffraction effect removal and image fusion image processing flow using output images with different angular fields of view to solve the problem of resolution degradation caused by diffraction.

[0034] Specifically, it includes obtaining initial images respectively captured when the under-screen camera moves to a plurality of different positions along a first direction, determining first image information in which the influence of the diffraction phenomenon occurs in the plurality of initial images based on the difference information between the plurality of initial images, and performing a complement processing on the first image information based on second image information in which the influence of the diffraction phenomenon does not occur in the plurality of initial images to obtain a target image.

[0035] For example, the under-screen camera is moved by a certain amount to capture image data with different viewing angles, and alignment processing is performed based on the movement amount information for capturing different image data and the feature point information of each image data. The difference information is generated by comparing each aligned image data. Next, by setting the difference level extraction filter coefficient in conjunction with the exposure information of the under-screen camera, the difference in the occurrence state of the diffraction phenomenon can be accurately detected, and only the image information affected by diffraction can be extracted. Next, for the image information affected by diffraction obtained here, the part where the diffraction effect occurs is complemented and output from the image data where it does not occur, so as to obtain an image with the diffraction effect removed, thereby improving the shooting quality of the output image by the under-screen camera.

[0036] In this embodiment, by taking into account the exposure information of the under-screen camera, the influence of the diffraction effect corresponding to the magnitude of the diffraction effect due to the exposure amount is appropriately extracted. By complementing the part where the diffraction influence occurs from the image data where it does not occur, an image with the influence of diffraction removed can be obtained. By superimposing a plurality of images, the sensitivity can be improved. Embodiment 2

[0037] An embodiment of the present application takes an electronic device as a smartphone as an example, installs a front under-screen camera under the display, and uses a plurality of cameras with different light-receiving wavelengths. Continuing to refer to FIG. 2, a high-transmission region 210 with enhanced light transmittance is provided at a position above the under-screen camera 22 in the display 21. In this way, the camera is disposed under the high-transmission region of the display, and receives the light transmitted through the high-transmission region of the display to perform imaging.

[0038] Optionally, the under-screen camera is movable within the high-transmission region under the display. FIG. 3 shows an apparatus for changing the viewing angle of the under-screen camera. The under-screen camera 22 moves with respect to the planar space where the display exists (for example, in the horizontal direction X), and it is possible to acquire images with different viewing angles of the camera. For example, the arrow X0 indicates the direction of change of the viewing angle.

[0039] Specifically, as one implementation method, it is as follows. FIG. 4 shows a driving device of an ultrasonic motor. An ultrasonic motor 41 for driving the movement of the under-screen camera 22 is disposed on one side horizontal to the movement direction of the under-screen camera 22, and a guide 42 for supporting the under-screen camera 22 and a position detection magnet 43 for detecting the position (movement amount) are disposed opposite thereto. A magnetic sensor 44 is disposed on the same horizontal plane as the position detection magnet 43 of the under-screen camera 22. FIG. 5 shows a flowchart of driving the movement of the under-screen camera by the ultrasonic motor. Specifically, the position (movement amount) is calculated by feedback control by detecting the magnetic field of the position detection magnet 43 by the magnetic sensor 44, and the under-screen camera 22 is driven to move by the ultrasonic motor 41.

[0040] Another implementation method is as follows. FIG. 6 shows a driving device for a stepping motor. A driving stepping motor 61 is arranged on one horizontal side in the moving direction of the under-screen camera 22, and a guide 42 for supporting the under-screen camera 22 is arranged opposite thereto. FIG. 7 shows a flowchart of the movement driving of the under-screen camera by the stepping motor. Specifically, the position (moving amount) is calculated by feed-forward control by monitoring the pulses generated by the stepping motor 61 for driving, and the under-screen camera 22 is driven to move by the stepping motor 61.

[0041] FIG. 9 shows a diffraction effect removal and image fusion image processing flow using output images with different viewing angles to solve the problem of resolution degradation caused by diffraction.

[0042] Specifically, when the under-screen camera moves to a plurality of different positions along the first direction, acquiring initial images respectively taken with the highest contrast, determining first image information in which the influence of the diffraction phenomenon occurs in the plurality of initial images based on the difference information between the plurality of initial images, and performing a complement processing on the first image information based on second image information in which the influence of the diffraction phenomenon does not occur in the plurality of initial images to obtain a target image.

[0043] For example, when the under-screen camera moves to a plurality of different positions along the first direction, obtaining the initial images respectively captured with the highest contrast specifically includes obtaining an image optimal for the synthesis process based on the measurement of the difference in camera position and diffraction effect by different arbitrary measurement frames within the imaging region. Referring to FIG. 9, the difference in the diffraction effect due to the under-screen camera position can be obtained by measuring the contrast component. Referring to FIG. 10, the under-screen camera is moved to the position with the highest contrast in each of the different arbitrary measurement frames A, B, and C within the imaging region 101 and photographed to obtain images. The position and number of the measurement frames for measuring the contrast component at this time can be arbitrarily set. For example, as shown in FIG. 11, after moving the under-screen camera to the corresponding position according to the positions of the set plurality of measurement frames, the peak value of the contrast is measured according to the contrast of the corresponding measurement frame and photographed.

[0044] Then, after photographing an image for optimal image processing from the contrast component, the image data with different angles of view are subjected to alignment processing based on the movement amount information and the feature point information of each image data. The aligned image data are compared to generate difference information. Next, by setting the difference level extraction filter coefficient linked to the exposure information of the under-screen camera, the difference in the occurrence state of the diffraction phenomenon can be accurately detected, and only the information affected by diffraction can be extracted. Further, for the image information affected by diffraction obtained here, the part where the diffraction effect occurs is complemented and output from the image data where it does not occur, so as to obtain an image with the diffraction effect removed, thereby improving the shooting quality of the output image by the under-screen camera.

[0045] In this embodiment, by taking into account the exposure information of the under-screen camera, the influence of the diffraction effect according to the magnitude of the diffraction effect due to the exposure amount is appropriately extracted. By complementing the portion where the diffraction effect occurs from the image data where it does not occur, an image from which the influence of diffraction has been removed can be obtained. By superimposing a plurality of images, it is possible to improve the sensitivity. With contrast control, it is possible to obtain an image with an optimal diffraction effect for image processing, and high-performance resolution degradation correction, flare, and ghost removal can be realized. Example 3

[0046] In the embodiment of the present application, taking the case where the electronic device is a smartphone as an example, a front under-screen camera is installed under the display, and a plurality of cameras with different light reception wavelengths are used. Continuing to refer to FIG. 2, at a position above the under-screen camera 22 in the display 21, a high-transmission region 210 with an increased light transmittance is provided. In this way, the camera is disposed under the high-transmission region of the display and receives the light transmitted through the high-transmission region of the display to capture an image.

[0047] Optionally, the under-screen camera is movable within the high-transmission region under the display. FIG. 12 shows an apparatus for changing the angle of view of the under-screen camera. The under-screen camera moves with respect to the planar space where the display exists (for example, in the horizontal direction X and the vertical direction Y), and it is possible to obtain images with different angles of view of the under-screen camera. For example, the arrow X0 indicates the horizontal change direction of the angle of view, and the arrow Y0 indicates the vertical change direction of the angle of view.

[0048] FIG. 13 shows a diffraction effect removal and image fusion image processing flow using output images with different angles of view to solve the problem of resolution degradation caused by diffraction.

[0049] Specifically, it includes obtaining initial images respectively captured when the under-screen camera moves to a plurality of different positions along a first direction and a second direction; determining first image information in which the diffraction phenomenon has an impact on the plurality of initial images based on the difference information between the plurality of initial images; and performing a complement processing on the first image information based on second image information in which the diffraction phenomenon has no impact on the plurality of initial images to obtain a target image.

[0050] For example, the under-screen camera is moved by a certain amount along each of the X-axis and the Y-axis to capture image data with different angles of view. The image data with different angles of view are subjected to alignment processing based on the movement amount information and the feature point information of each image. The image data after alignment are compared to generate difference information. Next, by setting the difference level extraction filter coefficient in conjunction with the exposure information of the under-screen camera, the difference in the occurrence state of the diffraction phenomenon can be accurately detected, and only the image information affected by diffraction can be extracted. Next, for the image information affected by diffraction obtained here, the part where the diffraction effect occurs is complemented and output from the image data where it does not occur, so as to obtain an image with the diffraction effect removed, thereby improving the shooting quality of the output image by the under-screen camera.

[0051] In this embodiment, by taking into account the exposure information of the under-screen camera, the influence of the diffraction effect corresponding to the magnitude of the diffraction effect due to the exposure amount is appropriately extracted. By complementing the part where the diffraction effect occurs from the image data where it does not occur, an image with the diffraction effect removed can be obtained. By superimposing a plurality of images, the sensitivity can be improved. By obtaining images with changes in the angles of view in two directions of the X-axis and the Y-axis, accurate extraction and synthesis processing of the analysis phenomenon can be achieved. Example 4

[0052] An embodiment of the present application takes an electronic device as a smartphone as an example, installs a front under-screen camera under the display, and uses a plurality of cameras with different light-receiving wavelengths. Continuing to refer to FIG. 2, a high-transmission region 210 with enhanced light transmittance is provided at a position above the under-screen camera 22 in the display 21. In this way, the camera is disposed under the high-transmission region of the display, receives the light transmitted through the high-transmission region of the display, and captures an image.

[0053] Optionally, the under-screen camera is movable within the high-transmission region under the display. FIG. 12 shows an apparatus for changing the viewing angle of the under-screen camera. The camera moves with respect to the planar space where the display exists (for example, in the horizontal and vertical directions), and it is possible to acquire images with different viewing angles of the camera.

[0054] FIG. 14 shows a diffraction effect removal and image fusion image processing flow using output images with different viewing angles to solve the problem of resolution degradation caused by diffraction.

[0055] Specifically, when the under-screen camera moves to a plurality of different positions along the first direction and the second direction, acquiring initial images respectively captured with the highest contrast, determining first image information in which the diffraction phenomenon has an impact on the plurality of initial images based on the difference information between the plurality of initial images, and performing a complement processing on the first image information based on second image information in which the diffraction phenomenon has no impact on the plurality of initial images to obtain a target image.

[0056] For example, when the under-screen camera moves to a plurality of different positions along the first direction and the second direction, obtaining the initial images respectively captured with the highest contrast specifically includes obtaining an image optimal for the synthesis process based on the measurement of the differences in the camera position and the diffraction effect by any different measurement frames within the imaging region. Continuing to refer to FIG. 9, the difference in the diffraction effect due to the under-screen camera position can be obtained by measuring the contrast component. Continuing to refer to FIG. 10, the under-screen camera is moved to the position with the highest contrast within each of the different arbitrary measurement frames A, B, and C in the imaging region 101 for shooting to obtain images. The position (for example, different positions moved along the first direction and the second direction) and the number of the measurement frames for measuring the contrast component at this time can be arbitrarily set. For example, as shown in FIG. 11, after moving the under-screen camera to the corresponding positions according to the positions of the set plurality of measurement frames, the peak value of the contrast is measured according to the contrast of the corresponding measurement frame for shooting.

[0057] Then, the under-screen camera is moved along each of the X-axis and the Y-axis respectively. After shooting an image for optimal image processing from the contrast component, the plurality of image data with different viewing angles captured are subjected to alignment processing based on the movement amount information and the feature point information of each image data. The aligned image data are compared to generate difference information. Next, by setting the difference level extraction filter coefficient in conjunction with the exposure information of the under-screen camera, the difference in the occurrence state of the diffraction phenomenon can be accurately detected, and only the information affected by diffraction can be extracted. Further, for the image information affected by diffraction obtained here, the portion where the diffraction effect occurs is complemented and output from the image data where the diffraction effect does not occur, so as to obtain an image with the diffraction effect removed, thereby improving the shooting quality of the output image by the under-screen camera.

[0058] In this embodiment, by taking into account the exposure information of the camera, the influence of the diffraction effect according to the magnitude of the diffraction effect due to the exposure amount is appropriately extracted. By complementing the portion where the diffraction effect occurs from the image data where it does not occur, an image from which the influence of diffraction has been removed can be obtained. By superimposing a plurality of images, it is possible to improve the sensitivity. By acquiring an image with changes in the viewing angles in two directions of the X-axis and the Y-axis, it becomes possible to extract and synthesize an analysis phenomenon with high accuracy. By controlling the contrast, it becomes possible to acquire an image with an optimal diffraction effect for image processing, and high-performance resolution degradation correction, flare, and ghost removal can be realized.

[0059] As shown in FIG. 15, an embodiment of the present invention provides an image processing apparatus 1500. The electronic device has an under-screen camera installed under the display. The apparatus 1500 includes an acquisition module 1510 for acquiring initial images captured by the under-screen camera at a plurality of different viewing angles, and a first processing module 1520 for determining first image information in which the influence of the diffraction phenomenon has occurred in the plurality of initial images based on the difference information between the plurality of initial images, and a second processing module 1530 for performing a complementation process on the first image information based on second image information in which the influence of the diffraction phenomenon has not occurred in the plurality of initial images to obtain a target image.

[0060] Optionally, the acquisition module 1510 includes a first acquisition unit for acquiring initial images respectively captured when the under-screen camera moves to a plurality of different positions along a first direction, or includes a second acquisition unit for acquiring initial images respectively captured with the highest contrast when the under-screen camera moves to a plurality of different positions along the first direction, or includes a third acquisition unit for acquiring initial images respectively captured when the under-screen camera moves to a plurality of different positions along a first direction and a second direction, or includes a fourth acquisition unit for acquiring initial images respectively captured with the highest contrast when the under-screen camera moves to a plurality of different positions along the first direction and the second direction.

[0061] Optionally, the first direction and the second direction are different directions parallel to the display.

[0062] Optionally, the fact that the under-screen camera moves to a plurality of different positions includes that the under-screen camera moves to a plurality of different positions respectively with a fixed movement amount, or includes that the under-screen camera moves to a plurality of different positions respectively with a movement amount determined by contrast.

[0063] Optionally, the image processing apparatus 1500 further includes a third processing module for aligning a plurality of the initial images based on the feature point information for each of the initial images and the movement amount information between different initial images, and a fourth processing module for determining difference information between the plurality of the initial images based on the plurality of the initial images after the alignment processing, where the movement amount information is the movement amount information of the under-screen camera when capturing different initial images.

[0064] Optionally, the first processing module 1520 includes a first processing unit for determining first image information in which the diffraction phenomenon affects a plurality of the initial images based on difference information between the plurality of the initial images and filter coefficients, where the filter coefficients are determined based on a difference level associated with the exposure information of the under-screen camera.

[0065] Optionally, the second processing module 1530 includes a second processing unit for determining second image information in which the diffraction phenomenon does not affect a plurality of the initial images corresponding to a target image region based on the target image region corresponding to the first image information, and a third processing unit for performing a complement processing on the first image information based on the second image information to obtain the target image.

[0066] In the case of an electronic device having an under-screen camera installed under a display, the above device according to an embodiment of the present invention acquires initial images captured by the under-screen camera at a plurality of different shooting angles, determines first image information in which the diffraction phenomenon affects a plurality of the initial images based on difference information between the plurality of the initial images, and performs a complement processing on the first image information based on second image information in which the diffraction phenomenon does not affect a plurality of the initial images, so as to obtain a target image, thereby guaranteeing the shooting image quality of the under-screen camera, reducing the influence on the image quality caused by the diffraction phenomenon, and increasing the transmittance of the display to improve the shooting image quality of the under-screen camera, thereby avoiding the problem that the display effect of the display is reduced.

[0067] As shown in FIG. 16, an embodiment of the present invention further provides an electronic device 1600 including a processor 1601, a memory 1602, and a program or instruction stored in the memory 1602 and executable by the processor 1601. When the program or instruction is executed by the processor 1601, each process of the embodiment of the above image processing method is realized, and the same technical effect can be achieved. Therefore, for the sake of avoiding repetition, it is not described repeatedly here.

[0068] Note that the electronic devices in the embodiments of the present application include mobile electronic devices and non-mobile electronic devices.

[0069] Furthermore, the embodiments of the present invention provide a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, each process of the embodiments of the above-described image processing method can be realized, and the same technical effects can be achieved. Therefore, to avoid duplication, it will not be described repeatedly here.

[0070] As described above, the embodiments of the present invention have been described with reference to the drawings. However, the present invention is not limited to the above specific embodiments. The above specific embodiments are illustrative and not restrictive. Many forms that can be made by those skilled in the art without departing from the spirit and scope of the claims of the present invention under the inspiration of the present invention are all included in the protection scope of the present invention.

Claims

1. An image processing method applied to an electronic device having an under-screen camera installed under a display, comprising: obtaining initial images captured by the under-screen camera at a plurality of different viewing angles; determining first image information in which a diffraction phenomenon affects the plurality of initial images based on difference information between the plurality of initial images; performing a complement processing on the first image information based on second image information in which no diffraction phenomenon affects the plurality of initial images to obtain a target image; wherein the under-screen camera is movably installed, and the initial images captured at the plurality of different viewing angles are obtained by the movement of the under-screen camera.

2. The image processing method according to claim 1, wherein obtaining the initial images captured by the under-screen camera at a plurality of different viewing angles includes obtaining the initial images respectively captured when the under-screen camera moves to a plurality of different positions along a first direction; or obtaining the initial images respectively captured with the highest contrast when the under-screen camera moves to a plurality of different positions along the first direction; or obtaining the initial images respectively captured when the under-screen camera moves to a plurality of different positions along a first direction and a second direction; or obtaining the initial images respectively captured with the highest contrast when the under-screen camera moves to a plurality of different positions along the first direction and the second direction.

3. The image processing method according to claim 2, wherein It is characterized in that the first direction and the second direction are different directions parallel to the display.

4. The image processing method according to claim 2, The fact that the under-screen camera has moved to a plurality of different positions includes the fact that the under-screen camera has moved to a plurality of different positions respectively with a fixed movement amount, or It is characterized by including the fact that the under-screen camera has moved to a plurality of different positions respectively with a movement amount determined by contrast.

5. The image processing method according to claim 1, Before determining first image information in which a diffraction phenomenon has occurred in a plurality of the initial images based on difference information between the plurality of the initial images, performing alignment processing on the plurality of the initial images based on feature point information for each of the initial images and movement amount information between different initial images; determining difference information between the plurality of the initial images based on the plurality of the initial images after the alignment processing, where the movement amount information is movement amount information of the under-screen camera when taking different initial images.

6. The image processing method according to claim 1, Determining first image information in which a diffraction phenomenon has occurred in a plurality of the initial images based on difference information between the plurality of the initial images includes determining first image information in which a diffraction phenomenon has occurred in a plurality of the initial images based on the difference information between the plurality of the initial images and a filter coefficient, where the filter coefficient is determined based on a difference level associated with exposure information of the under-screen camera.

7. The image processing method according to claim 1, Based on the second image information in which the influence of the diffraction phenomenon does not occur in the plurality of initial images, performing a complement processing on the first image information to obtain a target image, which is determining, based on a target image region corresponding to the first image information, second image information in which the influence of the diffraction phenomenon does not occur corresponding to the target image region in the plurality of initial images; and performing a complement processing on the first image information based on the second image information to obtain the target image, characterized by including the above.

8. An image processing apparatus applied to an electronic device having an under-screen camera installed under a display, an acquisition module for acquiring initial images taken by the under-screen camera at a plurality of different shooting angles; a first processing module for determining first image information in which the influence of the diffraction phenomenon occurs in the plurality of initial images based on the difference information between the plurality of initial images; and a second processing module for performing a complement processing on the first image information based on second image information in which the influence of the diffraction phenomenon does not occur in the plurality of initial images to obtain a target image, wherein the under-screen camera is movably installed, and the initial images taken at the plurality of different shooting angles are obtained by the movement of the under-screen camera, characterized by the above.

9. An electronic device, including a processor, a memory, and a program or instruction stored in the memory and executable by the processor, wherein when the program or instruction is executed by the processor, it realizes the steps of the image processing method according to any one of Claims 1 to 7, characterized by the above.

10. A readable storage medium, wherein a program or instruction is stored in the readable storage medium, When the program or instruction is executed by a processor, it is characterized by realizing the steps of the image processing method according to any one of claims 1 to 7.

Images