High-resolution integrated imaging global correction method for accurately eliminating voxel diffusion

Abstract

The invention provides a high-resolution integrated imaging global correction method capable of accurately eliminating voxel diffusion, which comprises four processes of accurate representation of voxel diffusion, pixel rearrangement preprocessing, global optimization of voxel diffusion and accurate three-dimensional reconstruction, and comprises the following steps: taking a lens array with multiple rotation errors as a whole; the method comprises the following steps: calculating a generalization equation of reconstructed light, obtaining an accurate voxel diffusion size by splitting a voxel diffusion polygon, obtaining a pre-corrected micro-image array by using a pixel rearrangement preprocessing method, taking the pre-corrected micro-image array as the input of a discriminator in a pixel-to-pixel conditional generative adversarial network, and comparing the input with the output of a generator to obtain a reconstructed image. And finally, performing three-dimensional reconstruction on the corrected micro-image array by means of an integrated imaging 3D display system, thereby effectively simplifying the correction process of voxel diffusion, and giving consideration to the recovery of detail features while improving the overall resolution.

Description

1. Technical Field

[0001] The present invention relates to the field of image processing for computer vision tasks, and more particularly to a high-resolution integrated imaging global correction method for accurately eliminating voxel diffusion. 2. Background Technology

[0002] Integrated imaging 3D display has the advantages of compact structure, no need for coherent light source, and can display true 3D images with full parallax, true color and no visual fatigue. It conforms to the visual physiological habits of the human eye and has good compatibility and inheritance with modern flat panel displays. It is considered to be one of the most commercially valuable 3D display technologies and has broad application prospects in the fields of electronic sandbox, leisure and entertainment, medical practice, and education and teaching. As a key core component in the integrated imaging optical architecture, the lens array affects the 3D resolution in terms of aberration and rotation error. For a designed system, the aberration is fixed, while the rotation error is difficult to avoid during the movement and assembly of the system. However, the randomness of the rotation error makes the accurate positioning and correction of voxel diffusion very challenging. Therefore, how to establish the corresponding relationship between rotation error and voxel diffusion is a key issue.

[0003] In the related research on lens array rotation error calibration and correction, the traditional method cannot accurately characterize the light field reconstruction process because it only analyzes the special case of translation error and only targets a single lens element. By comparing the overlap between the test voxel and the reference voxel on the reference depth plane, the correction of voxels on different depth planes can be achieved. However, due to the lack of analysis of the interaction of multiple rotation errors at the same time, the correction of voxel diffusion is not accurate enough, which affects the improvement of the resolution of the reconstructed 3D image. In addition, the calculation process of the existing rotation error correction method is relatively cumbersome, which limits the promotion of practical applications. III. Summary of the invention

[0004] The present invention proposes a global correction method for high-resolution integrated imaging that accurately eliminates voxel diffusion. The core of the method is to effectively integrate the information of the reconstructed light field to achieve accurate characterization of voxel diffusion and adjust the pixel position to adapt to the change of the light field. The method first calculates the influence of various rotation errors of the lens array on the reconstructed light to obtain the precise size of the voxel diffusion. Subsequently, the pixel points with the same name of the ideal micro-image array are traversed and searched until the corresponding voxel is basically consistent with the ideal voxel size. The micro-image array after pixel rearrangement and the ideal micro-image array are used to train a batch generative adversarial network to generate a corrected micro-image array, realize global optimization of voxel diffusion, and thus improve the resolution of integrated imaging 3D display.

[0005] The method includes four steps: accurate characterization of voxel diffusion, pixel rearrangement preprocessing, global optimization of voxel diffusion and accurate three-dimensional reconstruction. The specific process is shown in the attached figure.Figure 1 shown.

[0006] The accurate characterization process of voxel diffusion divides the rotation error into left-hand error and right-hand error, where the translation error is a special case of the rotation error, as shown in the attached figure. Figure 2 shown.

[0007] According to the attached Figure 2 The geometric relationship in the figure, the influence of rotation error on the misalignment of the reconstructed light can be expressed as:

[0008]

[0009] in Δx represents the distance from the pixel A on the LCD to the main optical axis, z is the distance from the ideal lens array to the optical diffuser, and g is the distance from the ideal lens array to the LCD plane. n Represents the axial translation error, which is greater than 0 when located on the left side of the ideal lens array plane and less than 0 when located on the right side of the ideal lens array plane. n Represents the lateral translation error, which is greater than 0 when located above the principal optical axis of the ideal lens and less than 0 when located below the principal optical axis of the ideal lens.

[0010] Taking the lens array with multiple rotation errors as a whole, the generalized equation of the reconstructed light is calculated to obtain the intersection point of the reconstructed light, as shown in the attached figure. Figure 3 shown.

[0011] Voxel diffusion is composed of the intersection lines of the reconstruction rays, which are generally polygons. The voxel diffusion is divided into convex polygons and concave polygons by calculating the cross product of the edge vectors of the polygon. Use dotted lines to mark the positions where the cross product of the edge vectors is of opposite signs, extend the opposite-sign edges for cutting, and transform the concave polygon into a convex polygon. Then split the convex polygon to obtain multiple triangles, and sum up the areas of the triangles to get the size of the voxel diffusion, as shown in the figure below. Figure 4 shown.

[0012] The pixel rearrangement preprocessing process is as follows Figure 5 As shown. The position of the ideal homonymous point on the micro-image array of size M×N is marked. According to the mapping relationship between the homonymous point and the 3D voxel, the position of the corresponding voxel diffusion is obtained, and then the size of the voxel diffusion is calculated. The pixels near each ideal homonymous point are searched with a step size of one pixel, and the size of the corresponding voxel diffusion is calculated and compared with the ideal voxel until the sizes of the two are infinitely close, and the pre-corrected micro-image array is obtained.

[0013] The global optimization process of voxel diffusion is shown in the attached Figure 6As shown. A pixel-to-pixel conditional generative adversarial network is used. The generator uses the U-Net network architecture, which consists of an encoder and a decoder with jump connections. The input is an ideal EIA with a resolution of M×N and random noise z, and the output is a corrected micro-image array. The discriminator uses the PatchGAN structure, with the input being a pre-corrected micro-image array, and the output being a probability value that determines the degree of matching between the correction and the pre-corrected micro-image array. The loss function is calculated, and the back-propagation algorithm is used to update the parameters of the generator. Finally, a corrected micro-image array that matches the light field with rotation errors is obtained, thereby achieving global optimization of voxel diffusion.

[0014] The precise three-dimensional reconstruction process optically reconstructs the corrected micro-image array obtained by the global optimization process of voxel diffusion with the help of an integrated imaging 3D display system to obtain a high-resolution 3D display effect.

[0015] The present invention proposes a global correction method for high-resolution integrated imaging that accurately eliminates voxel diffusion. The influence of multiple rotation errors on voxel reconstruction when they exist simultaneously is comprehensively considered, and the influence mechanism of lens array rotation error on the reconstructed light field is fully explored. Taking the lens array as a whole, the corresponding relationship between the reconstructed light and the size of voxel diffusion is found, and the accurate characterization of voxel diffusion is achieved. The combination of pixel rearrangement preprocessing and the global optimization process of voxel diffusion can not only effectively simplify the correction process of voxel diffusion, but also take into account the restoration of detailed features while improving the overall resolution. IV. Description of the drawings

[0016] Attached Figure 1 This is a flow chart of a high-resolution integrated imaging global correction method for accurately eliminating voxel diffusion proposed by the present invention.

[0017] Attached Figure 2 This is a schematic diagram of rotation error.

[0018] Attached Figure 3 Schematic diagram for overall analysis of lens array rotation error.

[0019] Attached Figure 4 Flowchart for calculating voxel diffusion size.

[0020] Attached Figure 5 Flowchart for pixel realignment preprocessing.

[0021] Attached Figure 6 Flowchart for global optimization of voxel diffusion.

[0022] It should be understood that the above drawings are only schematic and are not drawn to scale. V. Specific implementation methods

[0023] A typical embodiment of a high-resolution integrated imaging global correction method for accurately eliminating voxel diffusion proposed by the present invention is described in detail below, and the present invention is further described in detail. It is necessary to point out here that the following embodiments are only used to further illustrate the present invention and cannot be understood as limiting the scope of protection of the present invention. Persons skilled in the art in this field may make some non-essential improvements and adjustments to the present invention based on the above content of the present invention, which still fall within the scope of protection of the present invention.

[0024] The present invention proposes a global correction method for high-resolution integrated imaging that accurately eliminates voxel diffusion, which specifically includes four processes: accurate characterization of voxel diffusion, pixel rearrangement preprocessing, global optimization of voxel diffusion, and accurate three-dimensional reconstruction. The specific process is shown in the attached figure. Figure 1 shown.

[0025] The accurate characterization process of voxel diffusion divides the rotation error into left-hand error and right-hand error, where the translation error is a special case of the rotation error, as shown in the attached figure. Figure 2 shown.

[0026] According to the attached Figure 2 The geometric relationship in the figure, the influence of rotation error on the misalignment of the reconstructed light can be expressed as:

[0027]

[0028] in Δx represents the distance from the pixel A on the LCD to the main optical axis, z is the distance from the ideal lens array to the optical diffuser, and g is the distance from the ideal lens array to the LCD plane. n Represents the axial translation error, which is greater than 0 when located on the left side of the ideal lens array plane and less than 0 when located on the right side of the ideal lens array plane. n Represents the lateral translation error, which is greater than 0 when located above the principal optical axis of the ideal lens and less than 0 when located below the principal optical axis of the ideal lens.

[0029] Taking the lens array with multiple rotation errors as a whole, the generalized equation of the reconstructed light is calculated to obtain the intersection point of the reconstructed light, as shown in the attached figure. Figure 3 shown.

[0030] Voxel diffusion is composed of the intersection lines of the reconstruction rays, which are generally polygons. The voxel diffusion is divided into convex polygons and concave polygons by calculating the cross product of the edge vectors of the polygon. Use dotted lines to mark the positions where the cross product of the edge vectors is of opposite signs, extend the opposite-sign edges for cutting, and transform the concave polygon into a convex polygon. Then split the convex polygon to obtain multiple triangles, and sum up the areas of the triangles to get the size of the voxel diffusion, as shown in the figure below. Figure 4 shown.

[0031] The pixel rearrangement preprocessing process is as shown in the attached Figure 5As shown. The position of the ideal homonymous point on the micro-image array with a size of 3840×2160 is marked. According to the mapping relationship between the homonymous point and the 3D voxel, the position of the corresponding voxel diffusion is obtained, and then the size of the voxel diffusion is calculated. The pixels near each ideal homonymous point are searched with a step size of one pixel, and the size of the corresponding voxel diffusion is calculated and compared with the ideal voxel until the sizes of the two are infinitely close, and the pre-corrected micro-image array is obtained.

[0032] The global optimization process of voxel diffusion is shown in the attached Figure 6 As shown. A pixel-to-pixel conditional generative adversarial network is used. The generator uses the U-Net network architecture, which consists of an encoder and a decoder with jump connections. The input is the ideal EIA with a resolution of 3840×2160 and random noise z, and the output is a corrected micro-image array. The discriminator uses the PatchGAN structure, with the input being the pre-corrected micro-image array, and the output being a probability value that determines the degree of matching between the correction and the pre-corrected micro-image array. The loss function is calculated, and the back-propagation algorithm is used to update the parameters of the generator. Finally, a corrected micro-image array that matches the light field with rotation errors is obtained, thereby achieving global optimization of voxel diffusion.

[0033] The precise three-dimensional reconstruction process optically reconstructs the corrected micro-image array obtained by the global optimization process of voxel diffusion with the help of an integrated imaging 3D display system to obtain a high-resolution 3D display effect.

[0034] The present invention proposes a global correction method for high-resolution integrated imaging that accurately eliminates voxel diffusion. The influence of multiple rotation errors on voxel reconstruction when they exist simultaneously is comprehensively considered, and the influence mechanism of lens array rotation error on the reconstructed light field is fully explored. Taking the lens array as a whole, the corresponding relationship between the reconstructed light and the size of voxel diffusion is found, and the accurate characterization of voxel diffusion is achieved. The combination of pixel rearrangement preprocessing and the global optimization process of voxel diffusion can not only effectively simplify the correction process of voxel diffusion, but also take into account the restoration of detailed features while improving the overall resolution.

Claims

1. A high-resolution integrated imaging global correction method for accurately eliminating voxel diffusion, characterized in that: The method comprises four processes: accurate characterization of voxel diffusion, pixel rearrangement preprocessing, global optimization of voxel diffusion and accurate three-dimensional reconstruction. In the accurate characterization of voxel diffusion, the influence of different rotation errors of the lens array on the reconstructed light is calculated, and then the lens array with multiple rotation errors is taken as a whole to calculate the generalized equation of the reconstructed light, and the intersection of the reconstructed light is obtained. The area of ​​the voxel diffusion polygon formed by the intersection is calculated to obtain the accurate size of the voxel diffusion. In the pixel rearrangement preprocessing, the position of the ideal same-name point on the micro-image array is marked, the pixels near the ideal same-name point are traversed and searched, and the size of the corresponding voxel is compared with the ideal voxel until the two are infinitely close to obtain a pre-corrected micro-image array. In the global optimization of voxel diffusion, a pixel-to-pixel conditional generative adversarial network is used to generate a corrected micro-image array that matches the pre-corrected micro-image array. In the accurate three-dimensional reconstruction, the integrated imaging 3D display system is used to optically reconstruct the corrected micro-image array to achieve a high-resolution 3D display effect.

2. A high-resolution integrated imaging global correction method for accurately eliminating voxel diffusion according to claim 1, characterized in that: The rotation error is divided into left-hand error and right-hand error, among which the translation error is a special case of the rotation error. The influence on the misalignment of the reconstructed light is calculated respectively, and the generalized equation of the reconstructed light of the lens array after various rotation errors is obtained, and then the position of the voxel diffusion is determined; the shape of the voxel diffusion is generally a polygon, and the voxel diffusion is divided into convex polygons and concave polygons by calculating the cross product of the edge vectors. The positions where the cross product of the edge vectors is of opposite signs are marked with dotted lines, and the opposite-sign edges are extended for cutting to transform the concave polygon into a convex polygon. The convex polygon is then divided into multiple triangles, and the areas of the triangles are summed to obtain the size of the voxel diffusion.

3. The high-resolution integrated imaging global correction method for accurately eliminating voxel diffusion according to claim 1, characterized in that: The positions of ideal homonymous points on a micro-image array of M×N size are marked, and the positions of corresponding voxel diffusions are obtained according to the mapping relationship between homonymous points and 3D voxels, and then the size of voxel diffusion is calculated; the pixels near each ideal homonymous point are searched with a step size of one pixel, and the size of corresponding voxel diffusion is calculated and compared with the ideal voxel until the sizes of the two are infinitely close, thus obtaining a pre-corrected micro-image array.

4. The high-resolution integrated imaging global correction method for accurately eliminating voxel diffusion according to claim 1, characterized in that: A pixel-to-pixel conditional generative adversarial network is adopted. The generator uses the U-Net network architecture, which is composed of an encoder and a decoder with jump connections. The input is the ideal EIA with a resolution of M×N and random noise z, and the output is a corrected micro-image array. The discriminator adopts the PatchGAN structure, with the input being the pre-corrected micro-image array and the output being a probability value that determines the degree of matching between the correction and the pre-corrected micro-image array. The loss function is calculated, and the back-propagation algorithm is used to update the parameters of the generator. Finally, a corrected micro-image array that matches the light field with rotation errors is obtained, thereby achieving global optimization of voxel diffusion.

5. The high-resolution integrated imaging global correction method for accurately eliminating voxel diffusion according to claim 1, characterized in that: The corrected micro-image array obtained by the global optimization process of voxel diffusion is optically reconstructed with the help of an integrated imaging 3D display system to obtain a high-resolution 3D display effect.

Images