Integrated imaging element image array generation method for optimizing depth of field based on adaptive depth hierarchical fusion

Abstract

The invention discloses an integrated imaging element image array generation method based on adaptive depth layering fusion optimization depth of field under monocular vision. The method mainly comprises three parts of adaptive depth layering, layering generation of an element image array and fusion generation of a final element image array. According to the method, firstly, depth information in a depth map and display equipment parameters are utilized to jointly build an adaptive hierarchical model, and a three-dimensional scene is divided into a plurality of depth layers; next, encoding each layer of information to generate a corresponding element image array; and finally, fusing the element image arrays of different depth layers by referring to a human visual system in combination with a depth clue and adopting a Gaussian fusion coding method with depth difference to generate a final element image array. According to the element image array generated by the invention, the high-quality display field depth range can be expanded in a set field depth boundary, and the three-dimensional visual display quality of the integrated imaging system is improved.

Description

Technical Field

[0001] The present invention relates to the field of three-dimensional image processing, in particular to the field of integrated imaging display content generation, and in particular provides a method for generating an integrated imaging element image array based on adaptive depth layered fusion and optimized depth of field. Background Art

[0002] Integrated imaging light field display technology has attracted widespread attention from scholars and companies at home and abroad due to its simple system structure, the absence of auxiliary equipment, and its full-color, continuous viewing angle. The generation of elemental image arrays, as a key technical link, is particularly important for the stereoscopic display of three-dimensional scenes. At present, the generation of elemental image arrays based on the original light field color image and its registered depth map under monocular vision has become a mainstream generation method, but there are problems such as the limitation of the clear imaging depth of field range. Based on this, the present invention "A method for generating integrated imaging elemental image arrays based on adaptive depth layered fusion to optimize depth of field" is proposed. Summary of the Invention

[0003] The purpose of the present invention is to provide a method for generating an integrated imaging element image array based on adaptive depth layered fusion to optimize depth of field. The core of the method is to use the depth information in the depth map and the display device parameters to jointly build an adaptive layered model to divide the three-dimensional scene into multiple depth layers; each depth layer is encoded separately to generate an element image array; and finally, referring to the human visual system combined with depth cues, a Gaussian fusion coding method with depth differences is used to fuse the element image arrays of different depth layers to generate the final element image array. The present invention can expand the depth of field range of high-quality display within the established depth of field boundaries.

[0004] The technical solutions of the present invention are as follows:

[0005] The first step is to build an adaptive hierarchical model: omit the Y axis and build the following Figure 1 The coordinate system shown is based on the X-axis. eia ,g) emits a light path, passes through the optical center of the lens and intersects the two adjacent depth layers Z1 and Z2 at A(x oe1 ,Z1)、A(x oe2 , Z2), let the pixel size in the original scene be P I , then we must ensure that Δx=|x oe1 -x oe2 |>=P I , assuming the distance between the two depth planes is ΔZ, then the layered distance is This is the key constraint condition for adaptive stratification, and the formula is as follows:

[0006]

[0007] Where g is the distance between the display lens array and the element image array.

[0008] Based on formula (1), the maximum number of layers that can be layered when the display device parameters are fixed can be calculated, as shown in formula (2):

[0009]

[0010] Where D is the display depth range of the display device, as expressed in formula (3):

[0011]

[0012] Among them, d c is the center depth value, as expressed in formula (4):

[0013]

[0014] Where f is the focal length of the lens.

[0015] Next, we can filter the layer threshold by combining the local maximum value in the histogram of the depth map with the constraint conditions, merge the adjacent layers whose spacing is less than the constraint distance calculated by formula (1), and finally determine the effective layer threshold {Z1, Z2, ..., Z N}, where Z1 = Z min , Z N =Z max .

[0016] The second step is to generate the element image array of each layer: according to the depth mask of each layer established in the first step, the information in the color image is divided into each depth layer in turn, and the corresponding element image array is generated for each layer. n , the element image array of a single element image size is L*L, pixel A(x eia ,y eia ) and the original scene image A(x oe ,y oe ) is shown in formula (5), where L0 represents the optical center of the lens.

[0017]

[0018] The third step is to fuse and generate the final element image array: First, the element image arrays of each layer generated in the second step are scaled based on the depth distance. The scaling formula is as follows:

[0019]

[0020] P D Indicates the physical size of the display pixel of the display device, P DnRepresents the element image array in Z n The physical size of the pixel at reconstruction.

[0021] Next, the scaled element image array is fused using Gaussian weights based on depth differences, as shown in Equation (7):

[0022]

[0023] in, To ensure that the sum of the weights of all layers is 1, the weights are normalized. n is the standard deviation of the Gaussian kernel of the nth layer, which controls the weight distribution range of the current depth layer during fusion, as expressed in formula (8):

[0024]

[0025] σ0 is the initial standard deviation at the central depth plane and is a reference parameter for weight distribution. In the present invention, σ0≈0.28D.

[0026] The weights calculated according to formula (7) are brought into formula (9) to perform weighted fusion on the element images of each layer to obtain the final element image array:

[0027]

[0028] The beneficial effects of the present invention are: based on improving the display quality of each layer of element image array and following the visual perception characteristics of the human eye, the element image arrays of different depth layers are fused in a reasonable and natural way, the areas with large depth differences are moderately fused, and the areas with similar depths are more closely fused, ensuring the continuity and integrity of the fused image, thereby effectively expanding the high-quality display depth of field range of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 This is the principle diagram of adaptive layering based on depth information in the present invention

[0030] Figure 2 The two-dimensional color image and depth image of the original scene used in the specific implementation of the present invention are

[0031] Figure 3 This is the adaptive layered effect diagram generated in the specific implementation of the present invention

[0032] Figure 4 In the specific implementation of the present invention, the element image array generated

[0033] Figure 5The element image array generated in the specific implementation of the present invention and the element image array generated by the comparison method are displayed after being reconstructed by computer at different depths.

[0034] The comparison method is the method disclosed in the invention patent "Wang Yu, Wang Lu. (2025). A method for generating an integrated imaging display image array based on local depth template matching. CN118982562B." DETAILED DESCRIPTION

[0035] The following is a detailed description of the implementation method of the present invention "A method for generating an integrated imaging element image array based on adaptive depth layered fusion to optimize depth of field" in conjunction with the accompanying drawings.

[0036] In the present invention, the two-dimensional color image and depth image of the original scene are shown in the attached figure. Figure 2 The integrated imaging display system consists of 100×100 focal length f=3mm, width P L =1mm square lenses are closely arranged to form a lens array. The distance from the lens array to the display image array is g = 3.5mm. The display pixel size is P D =0.06mm.

[0037] Based on the above parameters, the central depth d of the system can be calculated c , depth range D and original scene pixels P I size:

[0038]

[0039] The maximum display depth and minimum display depth are:

[0040] Z max =d c +D / 2=28.56mm

[0041] Z min =d c -D / 2=13.44mm

[0042] The minimum layer spacing can be calculated as:

[0043]

[0044] The maximum number of layers is:

[0045]

[0046] In this example we use the Figure 2 The color image and depth image are used as the original scene information. Combined with the histogram of the depth data and the above constraints, the color image in this example is divided into three layers, as shown in the attached figure. Figure 3shown.

[0047] Next, the three layers of information are encoded according to formula (5) to generate element image arrays, and then scaled according to formula (6).

[0048] Finally, the element image array is Gaussian fused with depth differences according to equations (7), (8), and (9) to generate the final element image array.

[0049] Attachment Figure 4 The element image array generated in the specific implementation of the present invention is attached Figure 5 The following are the display effects of the element image array generated by the present invention and the element image array generated by the contrast algorithm after computer reconstruction at different depth planes, as well as the objective technical indicators. This shows the superiority of the present invention in high-quality display depth of field.

[0050] Matters not covered by the present invention are known technologies.

[0051] The above embodiments are intended only to illustrate the technical concepts and features of the present invention. Their purpose is to enable those skilled in the art to understand the contents of the present invention and implement them accordingly. They are not intended to limit the scope of protection of the present invention. Any equivalent changes or modifications made in accordance with the spirit of the present invention are intended to be covered by the scope of protection of the present invention.

Claims

1. A method for generating an integrated imaging element image array based on adaptive depth layered fusion to optimize depth of field, characterized by: The method includes the following steps: S1, build an adaptive depth layering model: according to the depth information in the original scene depth map and the display device parameters, including but not limited to the physical distance P between the optical centers of adjacent lenses L , the distance between the element image array and the display lens array is g, and the center display depth is d c , jointly build an adaptive layering model to divide the 3D scene into N layers. During the layering process, the maximum number of layers N that match the display device parameters max , which is expressed as follows: Among them, Z max and Z min is the maximum display depth and minimum display depth that the display device can present, D is the display depth range of the display device, is the minimum depth difference between two adjacent depth layers, which is an important constraint in the layering process. As shown in formula (2), P I is the physical size of a pixel in the original scene; S2, generate each layer of element image array: according to the depth mask of each layer established in S1, divide the information in the color image into each depth layer in turn, and generate the corresponding element image array EIA for each layer of layer encoding n (x eia ,y eia ); S3, fusion generates the final element image array: with reference to the human visual system, the Gaussian fusion coding method with depth difference is used to fuse the element image arrays of different depth layers to generate the final EIA. The weight is calculated according to the distance between the depth of each layer and the central depth plane. The closer the depth value is to the central depth plane, the higher the weight is obtained. After the weight is normalized, the element images of each layer are weighted fused to form the final element image array EIA. final (x eia ,y eia ), which is expressed as follows: Among them, w n (x eia ,y eia ) represents the weight obtained when the element image array of each layer is fused, which is expressed as follows: in, To ensure that the sum of the weights of all layers is 1, the weights are normalized. n is the standard deviation of the Gaussian kernel of the nth layer, which controls the weight distribution range of the current depth layer during fusion.

Images