Method, device, electronic equipment and medium for pre-correcting image aberration

Abstract

The present application discloses a method, device, electronic device and medium for pre-correcting image aberrations. In the present application, an aberration-free primitive image can be obtained; the aberration-free primitive image can be pre-corrected using a convolutional neural network model to obtain a plurality of first pre-corrected images, and the aberration-free primitive image can be pre-corrected using a Wiener filter to obtain a plurality of second pre-corrected images; the plurality of first pre-corrected images and the second pre-corrected images can be spliced ​​to obtain a plurality of corresponding joint pre-corrected primitive images; among the plurality of joint pre-corrected primitive images, the image with the highest image index value is selected as the target pre-corrected image. By applying the technical solution of the present application, the pre-trained convolutional neural network model training model and the Wiener filter can be used to achieve the pre-correction processing of the lens aberration in the system, so as to suppress the influence of various degrees of lens aberration on the light field display device without increasing the complexity of the system.

Description

Technical Field

[0001] The present application relates to image processing technology, and in particular to a method, device, electronic device and medium for pre-correcting image aberration. Background Art

[0002] In recent years, three-dimensional (3D) display technology has developed rapidly, and desktop 3D display technology has attracted widespread attention because it can simultaneously provide stereoscopic 3D scenes with correct depth information to multiple observers around the device.

[0003] Furthermore, it is still difficult to reconstruct a large true-color three-dimensional image in the existing three-dimensional display system due to the complex image acquisition, transmission and display processes. It is understandable that in an integrated imaging system, the viewing angle is determined by the size of the primitive image covered by a single lens and the distance between the lens and the display panel. Although integrated imaging technology can provide viewers with continuous parallax in both horizontal and vertical directions, it also has inherent defects of low display resolution and small viewing angle. Because, in an integrated imaging system, the display resolution and the number of viewpoints happen to be a pair of mutually constrained parameters. Generally speaking, directly increasing the number of viewpoints of a three-dimensional display device will lead to a decrease in display resolution, while increasing the display resolution will lead to a decrease in the number of viewpoints. There is a lack of a display method that satisfies both multiple viewpoints and high resolution. Summary of the invention

[0004] The embodiments of the present application provide a method, device, electronic device, and medium for pre-correcting image aberrations. According to one aspect of the embodiments of the present application, a method for pre-correcting image aberrations is provided, characterized in that it includes:

[0005] Acquire an aberration-free primitive image, wherein the aberration-free primitive image does not contain lens aberration;

[0006] Performing aberration pre-correction on the aberration-free primitive image using a convolutional neural network model to obtain a plurality of first pre-corrected images; and performing aberration pre-correction on the aberration-free primitive image using a Wiener filter to obtain a plurality of second pre-corrected images;

[0007] Performing image stitching on a plurality of the first pre-corrected images and the second pre-corrected images to obtain a corresponding plurality of joint pre-corrected primitive images;

[0008] Among the multiple joint pre-corrected primitive images, an image with the highest image index is selected as a target joint pre-corrected image.

[0009] Optionally, in another embodiment of the method of the present application, before acquiring the aberration-free primitive image, the method further includes:

[0010] Using a virtual camera array to shoot a three-dimensional scene, and acquiring scene information corresponding to the three-dimensional scene;

[0011] Generate a parallax image using the scene information;

[0012] The pixels are reversely mapped on the initial primitive image to obtain the primitive image without aberration.

[0013] Optionally, in another embodiment of the method of the present application, the aberration-free primitive image is subjected to aberration pre-correction using a convolutional neural network model to obtain a plurality of first pre-corrected images, including:

[0014] Inputting the aberration-free primitive image into the encoder of the convolutional neural network model to obtain image coding features, wherein the encoder includes 5 convolutional layers;

[0015] The image coding features are input into the decoder of the convolutional neural network model to obtain the first pre-corrected image after aberration pre-correction, wherein the decoder includes 5 deconvolution layers.

[0016] Optionally, in another embodiment of the method of the present application, the aberration-free primitive image is subjected to aberration pre-correction using a Wiener filter to obtain a plurality of second pre-corrected images, including:

[0017] The aberration-free primitive image is subjected to aberration pre-correction using a minimum mean square error filter to obtain the second pre-corrected image after aberration pre-correction.

[0018] Optionally, in another embodiment of the method of the present application, the step of performing image stitching on the plurality of first pre-corrected images and the second pre-corrected images to obtain a corresponding plurality of joint pre-corrected primitive images includes:

[0019] Selecting a first area image in the first pre-corrected image, where the diffuse spot root mean square radius value is greater than a root mean square radius threshold; and selecting a second area image in the second pre-corrected image, where the diffuse spot root mean square radius value is greater than the root mean square radius threshold;

[0020] The first region image and the second region image are stitched together to obtain the corresponding multiple joint pre-corrected primitive images.

[0021] Optionally, in another embodiment of the method of the present application, selecting an image with the highest image index among the multiple joint pre-corrected primitive images as a target joint pre-corrected image includes:

[0022] Taking the evaluation score as the ordinate and the root mean square radius of the diffuse spot as the abscissa, and calculating the evaluation score value of each target pre-corrected image, a correction efficiency curve is generated;

[0023] The target pre-corrected image corresponding to the maximum point in the correction efficiency curve is used as the target joint pre-corrected image.

[0024] Optionally, in another embodiment of the above method of the present application, after selecting the image with the highest image index as the target joint pre-correction image, the method further includes:

[0025] A liquid crystal display panel, a microlens array and the directional diffusion film are sequentially arranged on the optical path;

[0026] The liquid crystal display panel is used as an information input terminal to project the light field light of the target joint pre-corrected image to the microlens array;

[0027] The microlens array is used to refract and adjust the light of the light field passing through, and then project it onto the directional diffusion film to obtain a stereoscopic three-dimensional scene.

[0028] According to another aspect of the embodiments of the present application, a device for pre-correcting image aberration is provided, characterized in that it includes:

[0029] An acquisition module is configured to acquire an aberration-free primitive image, wherein the aberration-free primitive image does not contain lens aberration;

[0030] A pre-correction module is configured to perform aberration pre-correction on the aberration-free primitive image using a convolutional neural network model to obtain a plurality of first pre-corrected images; and perform aberration pre-correction on the aberration-free primitive image using a Wiener filter to obtain a plurality of second pre-corrected images;

[0031] a stitching module configured to stitch a plurality of the first pre-corrected images and the second pre-corrected images to obtain a corresponding plurality of joint pre-corrected primitive images;

[0032] The generating module is configured to select an image with the highest image index from the plurality of joint pre-corrected primitive images as a target joint pre-corrected image.

[0033] According to another aspect of the embodiments of the present application, an electronic device is provided, including:

[0034] A memory for storing executable instructions; and

[0035] A display is used to display with the memory to execute the executable instructions to complete the operation of any of the above-mentioned methods for pre-correcting image aberrations.

[0036] According to another aspect of the embodiments of the present application, a computer-readable storage medium is provided for storing computer-readable instructions, wherein the instructions, when executed, perform the operations of any of the above-mentioned methods for pre-correcting image aberrations.

[0037] In the present application, an aberration-free primitive image can be obtained, in which no lens aberration exists in the aberration-free primitive image; the aberration-free primitive image is pre-corrected by using a convolutional neural network model to obtain a plurality of first pre-corrected images; and the aberration-free primitive image is pre-corrected by using a Wiener filter to obtain a plurality of second pre-corrected images; the plurality of first pre-corrected images and the second pre-corrected images are image stitched to obtain a plurality of corresponding joint pre-corrected primitive images; among the plurality of joint pre-corrected primitive images, the image with the highest image index is selected as the target joint pre-corrected image. By applying the technical solution of the present application, the pre-correction processing of the lens aberration in the system can be realized by using the convolutional neural network model training model and the Wiener filter obtained by pre-training, so as to suppress the influence of various degrees of lens aberration on the light field display device without increasing the complexity of the system.

[0038] The technical solution of the present application is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The accompanying drawings, which constitute a part of the specification, illustrate embodiments of the present application and, together with the description, serve to explain the principles of the present application.

[0040] The present application can be more clearly understood from the following detailed description with reference to the accompanying drawings, in which:

[0041] Figure 1 A schematic diagram of a method for pre-correcting image aberrations proposed in this application;

[0042] Figure 2-Figure 4 A schematic diagram of a model system architecture for pre-correcting image aberrations proposed in this application;

[0043] Figure 5 A schematic diagram of the structure of an electronic device for pre-correcting image aberrations proposed in this application;

[0044] Figure 6 This is a schematic diagram of the structure of an electronic device for pre-correcting image aberrations proposed in this application. DETAILED DESCRIPTION

[0045] Various exemplary embodiments of the present application will now be described in detail with reference to the accompanying drawings. It should be noted that unless otherwise specifically stated, the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present application.

[0046] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0047] The following description of at least one exemplary embodiment is merely illustrative in nature and is not intended to limit the present application, its application, or uses.

[0048] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered as part of the specification.

[0049] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0050] In addition, the technical solutions between the various embodiments of the present application can be combined with each other, but it must be based on the fact that ordinary technicians in the field can implement it. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0051] It should be noted that all directional indications in the embodiments of the present application (such as up, down, left, right, front, back, etc.) are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0052] Combine the following Figure 1-Figure 4 The method for pre-correcting image aberration according to an exemplary embodiment of the present application is described. It should be noted that the following application scenarios are only shown to facilitate understanding of the spirit and principle of the present application, and the embodiments of the present application are not limited in this regard. On the contrary, the embodiments of the present application can be applied to any applicable scenario.

[0053] The present application also proposes a method, device, electronic equipment and medium for pre-correcting image aberration.

[0054] Figure 1 The flowchart of a method for pre-correcting image aberration according to an embodiment of the present application is schematically shown. Figure 1 As shown, the method includes:

[0055] S101, acquiring an aberration-free primitive image, wherein the aberration-free primitive image does not contain lens aberration.

[0056] In recent years, three-dimensional (3D) display technology has attracted widespread attention, and light field display technology is considered to be the most promising technology for realizing 3D display. Unlike 3D display based on binocular parallax, light field display can reconstruct the same 3D light field distribution as the real physical world in space, providing viewers with a real and natural 3D perception experience. This technology can easily achieve a full-color "true 3D" effect, and viewers do not need to wear any visual aids. Resolution is an important indicator that affects display quality in 3D display. An ideal 3D display technology can provide viewers with a visual sensory experience close to the real world. Generally, improving the display resolution can enhance the visual authenticity of the displayed stereoscopic content, so that it contains richer light field information, which helps to reconstruct the 3D scene more realistically.

[0057] Among the more mainstream three-dimensional light field display technologies, the integrated imaging technology based on microlens array and directional diffusion film has attracted particular attention. In integrated imaging, the primitive image covered by the unit circular lens undergoes spatial light modulation by the lens, so that the light emitted by the pixels arranged at different spatial positions on the focal plane is emitted in the direction of the line connecting the optical centers and re-converges on the directional diffusion film. The diffusion of the directional diffusion film forms a three-dimensional stereo image with full parallax. In the integrated imaging system, the viewing angle is determined by the size of the primitive image covered by a single lens and the distance between the lens and the display panel. Although the integrated imaging technology can provide viewers with continuous parallax in both horizontal and vertical directions, it also has inherent defects of low display resolution and small viewing angle. Because, in the integrated imaging system, the display resolution and the number of viewpoints happen to be a pair of mutually constrained parameters. Generally speaking, directly increasing the number of viewpoints of the three-dimensional display device will lead to a decrease in display resolution, while increasing the display resolution will lead to a decrease in the number of viewpoints. There is a lack of display methods that meet both multi-viewpoint and high resolution.

[0058] In the related technology, there are currently three main ideas to solve this problem: one is to use time division multiplexing to introduce modules and algorithms for refreshing the backlight so that multiple groups of content can be displayed in a short time; the second is to use space division multiplexing to introduce more two-dimensional display devices to enable the display of multiple groups of content; the third is to create a body pixel superposition and design special and complex optical devices to re-modulate the light distribution in space. The first two methods are to increase the number of pixels displayed per unit time to improve the visual resolution, and the last method is to increase the number of pixels that can be seen (visual pixels) to improve the visual resolution. Unfortunately, all three methods have significantly increased the complexity and manufacturing difficulty of the system without exception.

[0059] In view of the above problems, the present application adopts a convolutional neural network model training model obtained through pre-training and a Wiener filter to realize a processing method for pre-correcting lens aberrations in the system. First, it is necessary to obtain information collected by shooting a three-dimensional scene using a virtual camera, etc., and after obtaining a parallax image from the collected information, the pixels therein are reversely mapped to obtain a primitive image without aberrations.

[0060] S102, using a convolutional neural network model to perform aberration pre-correction on the aberration-free primitive image to obtain multiple first pre-corrected images; and using a Wiener filter to perform aberration pre-correction on the aberration-free primitive image to obtain multiple second pre-corrected images.

[0061] Furthermore, for obtaining the first pre-corrected image, the present application can use a preset convolutional neural network to perform aberration pre-correction processing on the composite image, so that it can have better performance when dealing with different degrees of aberration without increasing the number of system hardware devices.

[0062] Among them, in order to minimize the influence of lens aberration on the light field display quality, the present application needs to use a convolutional neural network generated by a five-layer convolution and deconvolution structure to re-encode the aberration-free primitive image, thereby obtaining a first pre-corrected image after aberration pre-correction.

[0063] Specifically, the deep learning convolutional neural network used in this application consists of two parts: an encoder and a decoder. The encoder contains five convolutional layers. When passing through each convolutional layer, the size of the feature is reduced to 0.5 times the original feature size, and the number of features is increased to 2 times the original number of features. The decoder contains five deconvolution layers, which use deconvolution operations to increase the resolution of features layer by layer, and finally obtain the first pre-corrected image. It can be understood that the above-mentioned convolutional neural network structure only gives an example that can meet the requirements of this application, and it is not only this specific network structure that can realize this application.

[0064] Furthermore, for obtaining the second pre-corrected image, the Wiener filter proposed in the present application may be a linear filter with the least square (statistical criterion) as the optimal criterion, also known as a minimum mean square error filter.

[0065] where, under the assumption that the noise and image are uncorrelated, one or the other has zero mean, and the gray level in the estimate is a linear function of the gray level in the degraded image, the minimum value of the mean square error function is given by the following expression in the frequency domain:

[0066]

[0067] Among them, H(u,v) is the degradation function; G(u,v) is the Fourier transform of the degraded image; S η (u,v) is the noise power spectrum, S f (u, v) is the power spectrum of the undegraded image. In practical applications, the appropriate K value can be selected according to the processing effect.

[0068] S103: stitching the multiple first pre-corrected images and the second pre-corrected images to obtain a corresponding multiple joint pre-corrected primitive images.

[0069] S104, selecting an image with the highest image index from among multiple joint pre-corrected primitive images as a target joint pre-corrected image.

[0070] Furthermore, for the aberration-free primitive images, if there is only one of them, a suitable aberration pre-correction method can be selected according to the distribution of aberration severity in the system, that is, the aberration pre-correction of a part of the area in an aberration-free primitive image is completed by a convolutional neural network, and the aberration pre-correction of another part of the area is completed by a Wiener filter.

[0071] For primitive images without aberration, if there are multiple primitive images without aberration, it is necessary to perform aberration pre-correction for each primitive image without aberration, and select a suitable joint pre-corrected primitive image from them.

[0072] like Figure 3 As shown, the present application may include the following steps to obtain a target pre-corrected image, so that the target pre-corrected image can be subsequently loaded on a liquid crystal display panel:

[0073] Step 1: If Figure 4 As shown, the present application can use a convolutional neural network and a Wiener filter to correct the aberration of the aberration-free primitive image to obtain a first pre-corrected image and a second pre-corrected image respectively, and splice the two pre-corrected images with the size of the root mean square radius of the diffuse spot as a variable to obtain a joint pre-corrected primitive image.

[0074] Step 2: The corrected stitched primitive image obtained in the first step (i.e., the joint pre-corrected primitive image) is sequentially sent to the image quality evaluation algorithm, wherein the quality evaluation algorithm draws a correction efficiency curve with the evaluation score as the vertical coordinate and the root mean square radius of the diffuse spot as the horizontal coordinate.

[0075] Among them, for the evaluation score, one way is to obtain it from the information fidelity (IFC) of the image. Among them, IFC is obtained by statistically analyzing natural images using a multi-scale Gaussian mixture model, measuring the degree of information loss, mainly targeting the waveform characteristics of high-frequency information, and has no response to the structural information of the image. Of course, this application can also use other parameters of the image as evaluation scores, such as peak signal-to-noise ratio, structural similarity, etc.

[0076] As for aberrations, since the severity of aberrations can be represented by the root mean square radius of the diffuse spot, it can be concluded that the larger the root mean square radius of the diffuse spot, the more severe the aberration in the area, and the smaller the radius, the milder the aberration.

[0077] Step 3: Find the maximum point of the correction efficiency curve in step 2, and use the joint pre-corrected primitive image corresponding to the point as the final target correction image.

[0078] In the present application, an aberration-free primitive image can be obtained; the aberration-free primitive image is pre-corrected using a convolutional neural network model to obtain a plurality of first pre-corrected images, and the aberration-free primitive image is pre-corrected using a Wiener filter to obtain a plurality of second pre-corrected images; the plurality of first pre-corrected images and the second pre-corrected images are image stitched to obtain a plurality of corresponding joint pre-corrected primitive images; among the plurality of joint pre-corrected primitive images, the image with the highest image index is selected as the target joint pre-corrected image. By applying the technical solution of the present application, the pre-trained convolutional neural network model training model and the Wiener filter can be used to implement the pre-correction of the lens aberration in the system, thereby suppressing the influence of various degrees of lens aberration on the light field display device without increasing the complexity of the system.

[0079] Optionally, in another embodiment of the method of the present application, before acquiring the aberration-free primitive image, the method further includes:

[0080] Using a virtual camera array to shoot a three-dimensional scene, and acquiring scene information corresponding to the three-dimensional scene;

[0081] Generate a parallax image using the scene information;

[0082] The pixels are reversely mapped on the initial primitive image to obtain the primitive image without aberration.

[0083] Optionally, in another embodiment of the method of the present application, the aberration-free primitive image is subjected to aberration pre-correction using a convolutional neural network model to obtain a plurality of first pre-corrected images, including:

[0084] Inputting the aberration-free primitive image into the encoder of the convolutional neural network model to obtain image coding features, wherein the encoder includes 5 convolutional layers;

[0085] The image coding features are input into the decoder of the convolutional neural network model to obtain the first pre-corrected image after aberration pre-correction, wherein the decoder includes 5 deconvolution layers.

[0086] Optionally, in another embodiment of the method of the present application, the aberration-free primitive image is subjected to aberration pre-correction using a Wiener filter to obtain a plurality of second pre-corrected images, including:

[0087] The aberration-free primitive image is subjected to aberration pre-correction using a minimum mean square error filter to obtain the second pre-corrected image after aberration pre-correction.

[0088] Optionally, in another embodiment of the method of the present application, the step of performing image stitching on the plurality of first pre-corrected images and the second pre-corrected images to obtain a corresponding plurality of joint pre-corrected primitive images includes:

[0089] Selecting a first area image in the first pre-corrected image, where the diffuse spot root mean square radius value is greater than a root mean square radius threshold; and selecting a second area image in the second pre-corrected image, where the diffuse spot root mean square radius value is less than the root mean square radius threshold;

[0090] The first region image and the second region image are stitched together to obtain the corresponding multiple joint pre-corrected primitive images.

[0091] It can be understood that since the method of pre-correction using only Wiener filtering in the related art will produce a ringing effect in areas with severe aberrations, and the method of pre-correction using only a deep learning network is not as effective as the filtering method in areas with slight aberrations, the present application can choose to select the first area image with a larger aberration value (greater than the aberration threshold) in the first pre-correction image (the image obtained after correction by the neural network model), and select the second area image with a smaller aberration value (less than the aberration threshold) in the second pre-correction image (the image obtained after filter correction), so that the images of the two areas can be subsequently spliced ​​to obtain the corresponding joint pre-correction primitive image.

[0092] Optionally, in another embodiment of the method of the present application, selecting an image with the highest image index among the multiple joint pre-corrected primitive images as a target joint pre-corrected image includes:

[0093] Taking the evaluation score as the ordinate and the root mean square radius of the diffuse spot as the abscissa, and calculating the evaluation score value of each target pre-corrected image, a correction efficiency curve is generated;

[0094] The joint pre-corrected image corresponding to the maximum point in the correction efficiency curve is used as the target pre-corrected image.

[0095] Optionally, in another embodiment of the above method of the present application, after selecting the image with the highest image index as the target joint pre-correction image, the method further includes:

[0096] A liquid crystal display panel, a microlens array and the directional diffusion film are sequentially arranged on the optical path;

[0097] The liquid crystal display panel is used as an information input terminal to project the light field light of the target pre-corrected image toward the microlens array;

[0098] The microlens array is used to refract and adjust the light of the light field passing through, and then project it onto the directional diffusion film to obtain a stereoscopic three-dimensional scene.

[0099] Furthermore, after obtaining the target pre-corrected image, the present application can subsequently project the target pre-corrected image array onto a directional diffusion film through a liquid crystal display panel and a lens array to obtain a final stereoscopic three-dimensional scene.

[0100] In one embodiment, the lens units of the lens array in the present application may be arranged in a 53*30 matrix, and the center distance between each adjacent lens is 13 mm.

[0101] Optionally, in another embodiment of the present application, Figure 5 As shown, the present application also provides a device for pre-correcting image aberrations. The device includes:

[0102] An acquisition module 201 is configured to acquire an aberration-free primitive image, wherein the aberration-free primitive image does not contain lens aberration;

[0103] The pre-correction module 202 is configured to perform aberration pre-correction on the aberration-free primitive image using a convolutional neural network model to obtain a plurality of first pre-corrected images; and perform aberration pre-correction on the aberration-free primitive image using a Wiener filter to obtain a plurality of second pre-corrected images;

[0104] A stitching module 203 is configured to stitch a plurality of the first pre-corrected images and the second pre-corrected images to obtain a corresponding plurality of joint pre-corrected primitive images;

[0105] The generating module 204 is configured to select an image with the highest image index from the plurality of joint pre-corrected primitive images as a target pre-corrected image.

[0106] In the present application, an aberration-free primitive image can be obtained; the aberration-free primitive image is pre-corrected using a convolutional neural network model to obtain a plurality of first pre-corrected images, and the aberration-free primitive image is pre-corrected using a Wiener filter to obtain a plurality of second pre-corrected images; the plurality of first pre-corrected images and the second pre-corrected images are image stitched to obtain a plurality of corresponding joint pre-corrected primitive images; among the plurality of joint pre-corrected primitive images, the image with the highest image index is selected as the target pre-corrected image. By applying the technical solution of the present application, the pre-trained convolutional neural network model training model and the Wiener filter can be used to implement the pre-correction processing of the lens aberration in the system, thereby suppressing the influence of various degrees of lens aberration on the light field display device without increasing the complexity of the system.

[0107] In another embodiment of the present application, the acquisition module 201 further includes:

[0108] The acquisition module 201 is configured to capture the three-dimensional scene using a virtual camera array to obtain scene information corresponding to the three-dimensional scene;

[0109] An acquisition module 201 is configured to generate a parallax image from the scene information;

[0110] The acquisition module 201 is configured to perform reverse mapping processing on pixels on the initial primitive image to obtain the primitive image without aberration.

[0111] In another embodiment of the present application, the pre-correction module 202 further includes:

[0112] A pre-correction module 202 is configured to input the aberration-free primitive image into an encoder of the convolutional neural network model to obtain image coding features, wherein the encoder includes 5 convolutional layers;

[0113] The pre-correction module 202 is configured to input the image coding features into the decoder of the convolutional neural network model to obtain the first pre-corrected image after aberration pre-correction, wherein the decoder includes 5 deconvolution layers.

[0114] In another embodiment of the present application, the pre-correction module 202 further includes:

[0115] The pre-correction module 202 is configured to perform aberration pre-correction on the aberration-free primitive image using a Wiener filter to obtain the second pre-corrected image after aberration pre-correction.

[0116] In another embodiment of the present application, the splicing module 203 further includes:

[0117] The stitching module 203 is configured to select a first region image in the first pre-corrected image, where the diffuse spot root mean square radius value is greater than a root mean square radius threshold; and select a second region image in the second pre-corrected image, where the diffuse spot root mean square radius value is less than the root mean square radius threshold;

[0118] The stitching module 203 is configured to stitch the first region image and the second region image to obtain the corresponding multiple joint pre-corrected primitive images.

[0119] In another embodiment of the present application, the generating module 204 further includes:

[0120] The generating module 204 is configured to use the evaluation score as the ordinate and the diffuse spot root mean square radius as the abscissa, and calculate the evaluation score value of each target joint pre-corrected image to generate a correction efficiency curve;

[0121] The generating module 204 is configured to use the joint pre-corrected image corresponding to the maximum point in the correction efficiency curve as the target joint pre-corrected image.

[0122] In another embodiment of the present application, the generating module 204 further includes:

[0123] A generating module 204 is configured to sequentially arrange a liquid crystal display panel, a micro lens array and the directional diffusion film on an optical path;

[0124] The generation module 204 is configured to use the liquid crystal display panel as an information input terminal to project the light field light of the target pre-corrected image to the microlens array; the microlens array is used to refract and adjust the passing light field light and then project it onto the directional diffusion film to obtain a stereoscopic three-dimensional scene.

[0125] Figure 6 3 is a logical structure block diagram of an electronic device according to an exemplary embodiment. For example, the electronic device 300 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0126] In an exemplary embodiment, a non-temporary computer-readable storage medium including instructions is also provided, such as a memory including instructions, the instructions can be executed by a processor of an electronic device to complete the method of pre-correcting image aberrations, the method comprising: obtaining an aberration-free primitive image, wherein there is no lens aberration in the aberration-free primitive image; performing aberration pre-correction on the aberration-free primitive image using a convolutional neural network model to obtain a plurality of first pre-corrected images; and performing aberration pre-correction on the aberration-free primitive image using a Wiener filter to obtain a plurality of second pre-corrected images; performing image stitching on the plurality of first pre-corrected images and the second pre-corrected images to obtain a corresponding plurality of joint pre-corrected primitive images; and selecting the image with the highest image index from the plurality of joint pre-corrected primitive images as the target pre-corrected image. Optionally, the instructions can also be executed by a processor of an electronic device to complete other steps involved in the exemplary embodiment. For example, the non-temporary computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

[0127] In an exemplary embodiment, an application / computer program product is also provided, comprising one or more instructions, which can be executed by a processor of an electronic device to complete the above-mentioned method of pre-correcting image aberrations, the method comprising: obtaining an aberration-free primitive image, wherein the aberration-free primitive image does not have lens aberrations; performing aberration pre-correction on the aberration-free primitive image using a convolutional neural network model to obtain a plurality of first pre-corrected images; and performing aberration pre-correction on the aberration-free primitive image using a Wiener filter to obtain a plurality of second pre-corrected images; performing image stitching on the plurality of first pre-corrected images and the second pre-corrected images to obtain a corresponding plurality of joint pre-corrected primitive images; and selecting the image with the highest image index from the plurality of joint pre-corrected primitive images as the target pre-corrected image. Optionally, the above-mentioned instructions can also be executed by a processor of an electronic device to complete other steps involved in the above-mentioned exemplary embodiment.

[0128] Figure 6 FIG. 3 is an example diagram of a computer device 30. A person skilled in the art will understand that Figure 6 It is only an example of the computer device 30 and does not constitute a limitation of the computer device 30. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the computer device 30 may also include input and output devices, network access devices, buses, etc.

[0129] The processor 302 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor 302 may also be any conventional processor, etc. The processor 302 is the control center of the computer device 30, and uses various interfaces and lines to connect various parts of the entire computer device 30.

[0130] The memory 301 can be used to store computer-readable instructions 303. The processor 302 implements various functions of the computer device 30 by running or executing the computer-readable instructions or modules stored in the memory 301 and calling the data stored in the memory 301. The memory 301 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the computer device 30, etc. In addition, the memory 301 can include a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM) or other non-volatile / volatile storage devices.

[0131] If the module integrated in the computer device 30 is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also instruct the relevant hardware to complete it through computer-readable instructions. The computer-readable instructions can be stored in a computer-readable storage medium. When the computer-readable instructions are executed by the processor, the steps of the above-mentioned various method embodiments can be implemented.

[0132] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0133] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A method for pre-correcting image aberrations, characterized in that: include: Acquire an aberration-free primitive image, wherein the aberration-free primitive image does not contain lens aberration; Performing aberration pre-correction on the aberration-free primitive image using a convolutional neural network model to obtain a plurality of first pre-corrected images; and performing aberration pre-correction on the aberration-free primitive image using a Wiener filter to obtain a plurality of second pre-corrected images; Performing image stitching on a plurality of the first pre-corrected images and the second pre-corrected images to obtain a corresponding plurality of joint pre-corrected primitive images; Among the multiple joint pre-corrected primitive images, an image with the highest image index is selected as a target pre-corrected image.

2. The method according to claim 1, characterized in that Before acquiring the aberration-free primitive image, the method further includes: Using a virtual camera array to shoot a three-dimensional scene, and acquiring scene information corresponding to the three-dimensional scene; Generate a parallax image using the scene information; The pixels are reversely mapped on the initial primitive image to obtain the primitive image without aberration.

3. The method according to claim 1, characterized in that The method of using a convolutional neural network model to pre-correct the aberration of the primitive image without aberration to obtain a plurality of first pre-corrected images includes: Inputting the aberration-free primitive image into the encoder of the convolutional neural network model to obtain image coding features, wherein the encoder includes 5 convolutional layers; The image coding features are input into the decoder of the convolutional neural network model to obtain the first pre-corrected image after aberration pre-correction, wherein the decoder includes 5 deconvolution layers.

4. The method according to claim 1 or 3, characterized in that The method of using the Wiener filter to pre-correct the aberration of the aberration-free primitive image to obtain a plurality of second pre-corrected images includes: The aberration-free primitive image is subjected to aberration pre-correction by using a Wiener filter to obtain the second pre-corrected image after aberration pre-correction.

5. The method according to claim 1, characterized in that The step of performing image stitching on the plurality of the first pre-corrected images and the second pre-corrected images to obtain a corresponding plurality of joint pre-corrected primitive images includes: Selecting a first area image in the first pre-corrected image, where the diffuse spot root mean square radius value is greater than a root mean square radius threshold; and selecting a second area image in the second pre-corrected image, where the diffuse spot root mean square radius value is less than the root mean square radius threshold; The first region image and the second region image are stitched together to obtain the corresponding multiple joint pre-corrected primitive images.

6. The method according to claim 1, characterized in that The step of selecting an image with the highest image index from among the plurality of joint pre-corrected primitive images as a target pre-corrected image comprises: Taking the evaluation score as the ordinate and the root mean square radius of the diffuse spot as the abscissa, and calculating the evaluation score value of each target pre-corrected image, a correction efficiency curve is generated; The target corrected image corresponding to the maximum point in the correction efficiency curve is used as the target joint pre-corrected image.

7. The method according to claim 1, characterized in that After selecting the image with the highest image index as the target joint pre-correction image, the method further includes: A liquid crystal display panel, a microlens array and the directional diffusion film are sequentially arranged on the optical path; The liquid crystal display panel is used as an information input terminal to project the light field light of the target joint pre-positive image to the microlens array; The microlens array is used to refract and adjust the light of the light field passing through, and then project it onto the directional diffusion film to obtain a stereoscopic three-dimensional scene.

8. A device for pre-correcting image aberrations, characterized in that: include: An acquisition module is configured to acquire an aberration-free primitive image, wherein the aberration-free primitive image does not contain lens aberration; A pre-correction module is configured to perform aberration pre-correction on the aberration-free primitive image using a convolutional neural network model to obtain a plurality of first pre-corrected images; and perform aberration pre-correction on the aberration-free primitive image using a Wiener filter to obtain a plurality of second pre-corrected images; a stitching module configured to stitch a plurality of the first pre-corrected images and the second pre-corrected images to obtain a corresponding plurality of joint pre-corrected primitive images; The generating module is configured to select an image with the highest image index from the plurality of joint pre-corrected primitive images as a target joint pre-corrected image.

9. An electronic device, characterized in that: include: A memory for storing executable instructions; as well as, A processor is used to display with the memory to execute the executable instructions to complete the operation of the method for pre-correcting image aberrations in any one of claims 1-7.

10. A computer-readable storage medium for storing computer-readable instructions, characterized in that: When the instructions are executed, the method for pre-correcting image aberrations according to any one of claims 1 to 7 is performed.

Images