A high-resolution integrated imaging global correction method for accurate elimination of voxel diffusion

Abstract

The application provides a high-resolution integrated imaging global correction method for accurately eliminating voxel diffusion, which comprises four processes of accurate characterization of voxel diffusion, pixel rearrangement preprocessing, global optimization of voxel diffusion and accurate three-dimensional reconstruction, the method takes a lens array with multiple rotation errors as a whole, calculates a general equation of a reconstructed light line, obtains an accurate size of voxel diffusion by splitting a voxel diffusion polygon, uses a pixel rearrangement preprocessing method to obtain a pre-corrected micro image array, uses the pre-corrected micro image array as an input of a discriminator in a pixel-to-pixel conditional generative adversarial network, compares the input with an output of a generator until the input and the output match to obtain a corrected micro image array, and finally performs three-dimensional reconstruction on the corrected micro image array by means of an integrated imaging 3D display system, so that the correction process of the voxel diffusion is effectively simplified, the overall resolution is improved, and the recovery of detail features is considered.

Description

I. TECHNICAL FIELD

[0001] The present application relates to the field of image processing of computer vision tasks, and more particularly, to a high-resolution integrated imaging global correction method for accurately eliminating voxel diffusion. II. BACKGROUND

[0002] Integrated imaging 3D display has the advantages of compact structure, no need for coherent light source, display of full parallax, true color and true 3D images without visual fatigue, in line with the visual physiological habits of human eyes, and good compatibility and inheritance with modern flat panel displays, and is considered as one of the 3D display technologies with the most commercial application value, and has broad application prospects in the fields of electronic sand table, leisure and entertainment, medical practice and education and teaching. As a key core element in the optical architecture of integrated imaging, the lens array has an impact on 3D resolution, including aberration and rotation error. For a well-designed system, the aberration is fixed, while the rotation error is difficult to avoid in the system movement and assembly process. However, the randomness of the rotation error makes it very challenging to accurately locate and correct the voxel diffusion. Therefore, how to establish the corresponding relationship between the rotation error and the voxel diffusion is a key problem.

[0003] In the related research on the calibration and correction of the rotation error of the lens array, the traditional method only analyzes the special case of the translation error and only targets a single lens element, and thus cannot accurately characterize the process of light field reconstruction. By comparing the overlap of the test voxel on the reference depth plane with the reference voxel, the correction of the voxel on different depth planes can be realized, but due to the lack of analysis of the interaction of multiple rotation errors existing at the same time, the correction of the voxel diffusion is not accurate enough, thereby affecting the improvement of the resolution of the reconstructed 3D image. In addition, the existing rotation error correction method has a relatively complicated calculation process, which limits the popularization of practical application. III. SUMMARY

[0004] The present application provides a high-resolution integrated imaging global correction method for accurately eliminating voxel diffusion, the core of which is to effectively integrate the information of the reconstructed light field to realize accurate characterization of the voxel diffusion and adjust the pixel position to adapt to the light field change. The method first calculates the influence of multiple rotation errors of the lens array on the reconstructed light, and obtains the accurate size of the voxel diffusion. Subsequently, the same-named pixel points of the ideal micro image array are searched iteratively until the corresponding voxel and the ideal voxel are basically the same in size. The micro image array after pixel rearrangement and the ideal micro image array are used to train the batch-generated adversarial network to generate a corrected micro image array, realize global optimization of the voxel diffusion, and thereby improve the resolution of the integrated imaging 3D display.

[0005] The method includes four processes of accurate characterization of voxel diffusion, pixel rearrangement preprocessing, global optimization of voxel diffusion and accurate three-dimensional reconstruction, and the specific process is shown in the accompanyingFigure 1 shown.

[0006] The precise characterization process of voxel diffusion divides the rotation error into left-hand error and right-hand error, where the translation error is a special case of the rotation error, as shown in the attached figure. Figure 2 shown.

[0007] According to the attached Figure 2 The geometric relationship in , the effect of rotation error on the size of the reconstructed light misalignment can be expressed as:

[0008]

[0009] in Δx represents the distance from the pixel A on the LCD to the principal optical axis, z is the distance from the ideal lens array to the optical diffuser, and g is the distance from the ideal lens array to the LCD plane. n Represents the axial translation error, which is greater than 0 when located on the left side of the ideal lens array plane and less than 0 when located on the right side of the ideal lens array plane. n Represents the lateral translation error, which is greater than 0 when located above the principal optical axis of the ideal lens and less than 0 when located below the principal optical axis of the ideal lens.

[0010] Taking the lens array with multiple rotation errors as a whole, the generalized equation of the reconstructed light is calculated to obtain the intersection of the reconstructed light, as shown in the attached figure. Figure 3 shown.

[0011] The voxel diffusion is formed by connecting the intersection points of the reconstruction rays, which is generally a polygon. The voxel diffusion is divided into convex polygons and concave polygons by calculating the cross product of the edge vectors of the polygon. The locations where the cross product of the edge vectors is opposite are marked with dotted lines. The opposite-sign edges are extended and cut to convert the concave polygon into a convex polygon. The convex polygon is then divided into multiple triangles. The area of ​​the triangles is summed to obtain the size of the voxel diffusion, as shown in the figure. Figure 4 shown.

[0012] The pixel rearrangement preprocessing process is as shown in the attached Figure 5 As shown in the figure, the ideal key points on the M×N micro-image array are marked. Based on the mapping relationship between the key points and 3D voxels, the corresponding voxel diffusion position is obtained, and then the voxel diffusion size is calculated. The pixels near each ideal key point are searched with a step size of one pixel. The corresponding voxel diffusion size is calculated and compared with the ideal voxel until the two sizes are infinitely close, resulting in the pre-corrected micro-image array.

[0013] The global optimization process of voxel diffusion is shown in the attached Figure 6The pixel-to-pixel conditional generative adversarial network is used, the generator uses the U-Net network architecture, which is composed of a jump-connected encoder and a decoder. The input is the ideal EIA with a resolution of MxN and random noise z, and the output is the corrected micro image array. The discriminator adopts the PatchGAN structure, the input is the pre-corrected micro image array, and the output is a probability value for judging the matching degree of the corrected and pre-corrected micro image array. The loss function is calculated, and the parameters of the generator are updated using the back propagation algorithm. Finally, the corrected micro image array matched with the light field with rotation error is obtained, so that the global optimization of voxel diffusion is realized.

[0014] The accurate three-dimensional reconstruction process uses the corrected micro image array obtained by the global optimization of voxel diffusion to perform optical reconstruction by means of the integrated imaging 3D display system, and obtains a high-resolution 3D display effect.

[0015] The application provides a high-resolution integrated imaging global correction method for accurately eliminating voxel diffusion. The influence of the simultaneous existence of various rotation errors on voxel reconstruction is comprehensively considered, and the influence mechanism of the lens array rotation error on the reconstructed light field is fully tapped. The lens array is taken as a whole, the corresponding relationship between the reconstructed light and the voxel diffusion size is found, and the accurate characterization of the voxel diffusion is realized. The combination of the pixel rearrangement preprocessing and the global optimization process of the voxel diffusion can not only effectively simplify the correction process of the voxel diffusion, but also improve the overall resolution while considering the recovery of the detail features. IV. BRIEF DESCRIPTION OF DRAWINGS

[0016] FIG. 1 is a schematic diagram of a rotation error. Figure 1 FIG. 2 is a flowchart of a high-resolution integrated imaging global correction method for accurately eliminating voxel diffusion.

[0017] FIG. 3 is a schematic diagram of an overall analysis of lens array rotation error. Figure 2 FIG. 4 is a flowchart for calculating the size of voxel diffusion.

[0018] Figure 3 FIG. 5 is a flowchart of pixel rearrangement preprocessing.

[0019] FIG. 6 is a global optimization flowchart of voxel diffusion. Figure 4 It should be understood that the above-mentioned drawings are only schematic and are not drawn to scale.

[0020] V. DETAILED DESCRIPTION Figure 5

[0021] Figure 6

[0022] It should be understood that the above-mentioned drawings are only schematic and are not drawn to scale. V. DETAILED DESCRIPTION

[0023] ​​​​The following describes in detail a typical embodiment of the global correction method for high-resolution integrated imaging that accurately eliminates voxel spread, further illustrating the present invention. It is important to note that the following embodiment is intended only to further illustrate the present invention and is not to be construed as limiting the scope of protection of the present invention. Non-essential improvements and adjustments made by persons skilled in the art based on the above disclosure are still within the scope of protection of the present invention.

[0024] The present invention proposes a global correction method for high-resolution integrated imaging that accurately eliminates voxel diffusion. Specifically, it includes four processes: accurate characterization of voxel diffusion, pixel rearrangement preprocessing, global optimization of voxel diffusion, and accurate three-dimensional reconstruction. The specific process is shown in the attached figure. Figure 1 shown.

[0025] The precise characterization process of voxel diffusion divides the rotation error into left-hand error and right-hand error, where the translation error is a special case of the rotation error, as shown in the attached figure. Figure 2 shown.

[0026] According to the attached Figure 2 The geometric relationship in , the effect of rotation error on the size of the reconstructed light misalignment can be expressed as:

[0027]

[0028] in Δx represents the distance from the pixel A on the LCD to the principal optical axis, z is the distance from the ideal lens array to the optical diffuser, and g is the distance from the ideal lens array to the LCD plane. n Represents the axial translation error, which is greater than 0 when located on the left side of the ideal lens array plane and less than 0 when located on the right side of the ideal lens array plane. n Represents the lateral translation error, which is greater than 0 when located above the principal optical axis of the ideal lens and less than 0 when located below the principal optical axis of the ideal lens.

[0029] Taking the lens array with multiple rotation errors as a whole, the generalized equation of the reconstructed light is calculated to obtain the intersection of the reconstructed light, as shown in the attached figure. Figure 3 shown.

[0030] The voxel diffusion is formed by connecting the intersection points of the reconstruction rays, which is generally a polygon. The voxel diffusion is divided into convex polygons and concave polygons by calculating the cross product of the edge vectors of the polygon. The locations where the cross product of the edge vectors is opposite are marked with dotted lines. The opposite-sign edges are extended and cut to convert the concave polygon into a convex polygon. The convex polygon is then divided into multiple triangles. The area of ​​the triangles is summed to obtain the size of the voxel diffusion, as shown in the figure. Figure 4 shown.

[0031] The pixel rearrangement preprocessing process is as shown in the attached Figure 5The position of the ideal homonymic point on the micro image array with the size of 3840*2160 is marked, the position of the corresponding voxel diffusion is obtained according to the mapping relationship between the homonymic point and the 3D voxel, and then the size of the voxel diffusion is calculated. The pixels near each ideal homonymic point are searched step by step with one pixel as a step, the size of the corresponding voxel diffusion is calculated, compared with the ideal voxel, until the sizes of the two are infinitely close, and the pre-corrected micro image array is obtained.

[0032] The global optimization process of the voxel diffusion is as shown in the accompanying drawings Figure 6 A pixel-to-pixel conditional generative adversarial network is used, the generator uses a U-Net network architecture, and is composed of a jump-connected encoder and a decoder. The input is the ideal EIA and random noise z with a resolution of 3840*2160, and the output is the corrected micro image array. The discriminator adopts a PatchGAN structure, the input is the pre-corrected micro image array, and the output is a probability value for judging the matching degree of the corrected and pre-corrected micro image arrays. The loss function is calculated, and the parameters of the generator are updated using the back propagation algorithm. Finally, the corrected micro image array matched with the light field with rotation error is obtained, so that the global optimization of the voxel diffusion is realized.

[0033] The accurate three-dimensional reconstruction process is as follows: the corrected micro image array obtained by the global optimization process of the voxel diffusion is used for optical reconstruction by means of the integrated imaging 3D display system, and a high-resolution 3D display effect is obtained.

[0034] The present application provides a high-resolution integrated imaging global correction method for accurately eliminating voxel diffusion. The influence of various rotation errors on voxel reconstruction is comprehensively considered, and the influence mechanism of lens array rotation error on reconstructed light field is fully explored. The lens array is taken as a whole, and the corresponding relationship between the reconstructed light and the size of the voxel diffusion is found, so that the accurate characterization of the voxel diffusion is realized. The combination of pixel rearrangement preprocessing and the global optimization process of the voxel diffusion not only can effectively simplify the correction process of the voxel diffusion, but also can improve the overall resolution while considering the recovery of detail features.

Claims

1. A global correction method for high-resolution integrated imaging that accurately eliminates voxel diffusion, characterized in that: The method includes four processes: precise characterization of voxel diffusion, pixel rearrangement preprocessing, global optimization of voxel diffusion, and precise three-dimensional reconstruction. In the precise characterization of voxel diffusion, the influence of different rotation errors of the lens array on the reconstructed light is calculated. Then, the lens array with multiple rotation errors is taken as a whole, the equation of the reconstructed light is calculated, the intersection of the reconstructed light is obtained, and the area of ​​the voxel diffusion polygon formed by the intersection is calculated to obtain the precise size of the voxel diffusion. In the pixel rearrangement preprocessing, the position of the ideal homonymous point on the micro-image array is marked, the pixels near the ideal homonymous point are traversed and searched, and the size of the corresponding voxel is compared with the ideal voxel until the two are infinitely close to obtain a pre-corrected micro-image array. In the global optimization of voxel diffusion, a pixel-to-pixel conditional generative adversarial network is used to generate a corrected micro-image array that matches the pre-corrected micro-image array. In the precise three-dimensional reconstruction, the corrected micro-image array is optically reconstructed using an integrated imaging 3D display system to achieve a high-resolution 3D display effect. Rotational errors are divided into left-handed and right-handed errors, with translational errors being a special case of rotational errors. Their effects on the size of the reconstructed light's misalignment are calculated separately, resulting in the equations for the reconstructed light of the lens array after various rotational errors. This allows the position of the voxel diffusion to be determined. The voxel diffusion is polygonal in shape and is divided into convex and concave polygons by calculating the cross product of edge vectors. Positions where the cross product of edge vectors is of opposite sign are marked with dashed lines. The concave polygon is then converted to a convex polygon by extending the opposite-sign edges for cutting. The convex polygon is then segmented to obtain multiple triangles, and the areas of the triangles are summed to obtain the size of the voxel diffusion. The positions of ideal homonymous points on the M×N microimage array are marked. Based on the mapping relationship between homonymous points and 3D voxels, the positions of the corresponding voxel diffusions are obtained, and the size of the voxel diffusions is then calculated. Pixels near each ideal homonymous point are searched with a step size of one pixel, and the size of the corresponding voxel diffusion is calculated and compared with the ideal voxel diffusion until the two sizes are infinitely close, resulting in a pre-corrected microimage array.

2. The high-resolution integrated imaging global correction method for accurately eliminating voxel diffusion according to claim 1, characterized in that: A pixel-to-pixel conditional generative adversarial network is adopted. The generator uses a U-Net network architecture, which consists of an encoder and a decoder with jump connections. The input is an ideal EIA with a resolution of M×N and random noise z, and the output is a corrected micro-image array. The discriminator adopts a PatchGAN structure, with an input of a pre-corrected micro-image array and an output as a probability value that determines the degree of matching between the correction and pre-corrected micro-image arrays. The loss function is calculated, and the back-propagation algorithm is used to update the parameters of the generator. Finally, a corrected micro-image array that matches the light field with rotation error is obtained, thereby achieving global optimization of voxel diffusion.

3. The high-resolution integrated imaging global correction method for accurately eliminating voxel diffusion according to claim 1, characterized in that: The corrected micro-image array obtained by the global optimization process of voxel diffusion is optically reconstructed with the help of an integrated imaging 3D display system to obtain a high-resolution 3D display effect.

Images