Image processing method and device for surround image

Abstract

The present invention discloses a method and device for processing an ambient image. The processing method of the present invention converts an ambient image into a cube map format. The depth information map of the ambient image is calculated in the cube map format, and the depth information map is seamlessly processed. The depth information map is converted from the cube map format back to the original format to obtain a depth information map corresponding to the ambient image.

Description

Technical Field

[0001] The invention relates to an image processing method and device for surround images. Background Art

[0002] In the past, a panoramic image usually refers to an image that covers 180 degrees in the horizontal direction. With the advancement of computer graphics technology, a panoramic image can now refer to an image that covers 360 degrees in the horizontal direction, or an image that covers 360 degrees in the horizontal direction and 360 degrees in the vertical direction (also called a spherical panorama). The difficulty of obtaining depth information of a panoramic image is often affected by whether the scene is indoors or outdoors. Generally speaking, it is more difficult to obtain depth information of a panoramic image of an outdoor environment than to obtain depth information of a panoramic image of an indoor environment. In addition, the methods of obtaining depth information of a panoramic image in the prior art are not suitable for application to outdoor scenes. In view of this, it is necessary to propose a method for obtaining depth information of a panoramic image that can be used for both indoor and outdoor scenes. Summary of the invention

[0003] The embodiment of the present invention discloses a device for processing panoramic images, including a processing unit and a storage unit. The storage unit is coupled to the processing unit for storing a computer-readable medium. When the computer-readable medium is executed by the processing unit, the processing unit is caused to execute: obtaining a first image; converting the first image from a first format into a second image having a cube map format; copying two vertical viewing blocks of the second image to generate three copy blocks respectively, and connecting a plurality of horizontal viewing blocks of the second image that are not connected to the two vertical viewing blocks with the copy blocks by corresponding edges between each of the horizontal viewing blocks and each of the copy blocks to generate a third image; taking out two images of 3x3 blocks from the third image to generate a fourth image and a fifth image; calculating the depth information of the fourth image and the fifth image to generate a first depth information map corresponding to the fourth image and a second depth information map corresponding to the fifth image;

[0004] The method further comprises: performing a first hybrid fusion process on the overlapping areas of the first depth information map and the overlapping areas of the second depth information map according to the two overlapping areas of the two block groups corresponding to the fourth image of the first depth information map and the two overlapping areas of the two block groups corresponding to the fifth image of the second depth information map; taking out a plurality of parts of the blocks corresponding to the second image from the first depth information map and the second depth information map to generate a plurality of depth information blocks; for each of the depth information blocks corresponding to the vertical viewing angle block, respectively mixing and merging the depth information blocks corresponding to the horizontal viewing angle blocks with the depth information blocks corresponding to the interval between the horizontal viewing angle blocks and the vertical viewing angle blocks; One side of the corresponding side is connected to four sides of the depth information block corresponding to the vertical viewing block; a second mixing process is performed on a plurality of connection points between the depth information blocks corresponding to the vertical viewing blocks and the depth information blocks corresponding to the horizontal viewing blocks to generate a third depth information map and a fourth depth information map; the depth information blocks corresponding to the blocks of the second image are taken out from the third depth information map and the fourth depth information map, and the depth information blocks are spliced ​​into a fifth depth information map having the cube map format; and the fifth depth information map is converted from the cube map format into a sixth depth information map having the first format.

[0005] Another embodiment of the present invention discloses a method for processing an ambient image, comprising: obtaining a first image; converting the first image from a first format into a second image having a cubemap format; copying two vertical view blocks of the second image to generate three copy blocks respectively, and connecting a plurality of horizontal view blocks of the second image that are not connected to the two vertical view blocks with the copy blocks by corresponding edges between each of the horizontal view blocks and each of the copy blocks to generate a third image; extracting two images of 3x3 blocks from the third image to generate a fourth image and a fifth image; calculating depth information of the fourth image and the fifth image to generate a first depth information map corresponding to the fourth image and a second depth information map corresponding to the fifth image;

[0006] According to the two overlapping areas of the two block groups of the first depth information map corresponding to the fourth image and the two overlapping areas of the two block groups of the second depth information map corresponding to the fifth image, a first hybrid fusion process is performed on the overlapping areas of the first depth information map and the overlapping areas of the second depth information map; multiple parts of the blocks corresponding to the second image are taken out from the first depth information map and the second depth information map to generate multiple depth information blocks; for each of the depth information blocks corresponding to the vertical viewing block, the depth information blocks corresponding to the horizontal viewing blocks are respectively mixed and fused with the depth information blocks corresponding to the horizontal viewing blocks and the vertical viewing blocks. One side of the corresponding side is connected to four sides of the depth information block corresponding to the vertical viewing block; a second mixing process is performed on a plurality of connection points between the depth information blocks corresponding to the vertical viewing blocks and the depth information blocks corresponding to the horizontal viewing blocks to generate a third depth information map and a fourth depth information map; the depth information blocks corresponding to the blocks of the second image are taken out from the third depth information map and the fourth depth information map, and the depth information blocks are spliced ​​into a fifth depth information map having the cube map format; and the fifth depth information map is converted from the cube map format into a sixth depth information map having the first format. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 A block diagram of a device for processing surround images according to an embodiment of the present invention is shown;

[0008] Figure 2A , 2B A flowchart of a method for processing surround images according to an embodiment of the present invention is shown;

[0009] Figures 3A to 3H A flowchart of a method for processing surround images according to an embodiment of the present invention is shown;

[0010] Figure 4 Another embodiment of the present invention is shown in FIG. Figure 2A , 2B A flowchart of a method for processing a surrounding image; DETAILED DESCRIPTION

[0011] Please refer to Figure 1 , Figure 1A block diagram of a device for processing surround images according to an embodiment of the present invention is shown. The processing device 10 includes a processing unit 102 and a storage unit 104. The processing unit 102 may be a general-purpose processor or a special-purpose processing chip. The storage unit 104 may be a non-volatile memory, or a combination of a non-volatile memory and a volatile memory. The storage unit 104 stores a computer-readable medium. When the computer-readable medium is executed by the processing unit 102, the processing unit 102 executes the method for processing surround images. The details of the method for processing surround images will be described in detail in conjunction with the accompanying drawings. Figure 2A and Figure 2B Describe it in detail.

[0012] Please refer to Figure 2A , 2B , Figure 2A , 2B A flowchart of a method for processing surround images according to an embodiment of the present invention is shown. Please also refer to the operation diagrams of the method for processing surround images in FIGS. A to 3H for better understanding.

[0013] In step S201, a first image is obtained. This first image IMG1 is a panoramic image with a first format. The panoramic image is, for example, an image covering 360 degrees in the horizontal direction and 360 degrees in the vertical direction (also known as a spherical panorama). The first format may be, for example, an equirectangular projection and a little planet projection, but the present invention is not limited thereto. In one embodiment, the first image IMG1 may be provided by a panoramic camera. In another embodiment, the first image IMG1 may be an image data stored in the storage device 104. In other words, the present invention does not limit the source and acquisition method of the first image IMG1.

[0014] In step S203, the first image IMG1 is converted from a first format into a second image IMG2 having a cubemap format, such as Figure 3A As shown. In one embodiment, the conversion method between the first format and the cube map format can be used in a manner well known to those skilled in the art. The second image IMG2 includes six blocks b#1~b#6, wherein the six blocks b#1~b#6 can be combined into a cube, blocks b#1~b#4 correspond to the front, back, left and right directions parallel to the plane P where the observation point C is located, and blocks b#5~b#6 correspond to the upper and lower directions perpendicular to the plane P where the observation point C is located, as shown in FIG. Figure 3BAs shown. When blocks b#1 to b#6 are combined into a cube, any block will be adjacent to four of the remaining five blocks and will not be adjacent to another of the remaining five blocks. For example, block b#5 will be adjacent to blocks b#1 to b#4 and will not be adjacent to block b#6, wherein the four edges of block b#5 will be connected to one of blocks b#1 to b#4 respectively. For the convenience of understanding and explanation, the present invention defines the relationship between blocks in the second image IMG2. Blocks b#1 to b#4 corresponding to the front, back, left, and right directions parallel to the plane P where the observation point C is located are defined as "horizontal viewing blocks", and blocks b#5 to b#6 corresponding to the upper and lower directions perpendicular to the plane P where the observation point C is located are defined as "vertical viewing blocks". The "relative block" of block b#i is defined as the block that is not adjacent to block b#i when blocks b#1 to b#6 are combined into a cube. The "adjacent blocks" of block b#i are defined as the four blocks adjacent to block b#i when blocks b#1 to b#6 are combined into a cube. The "adjacent blocks corresponding to the jth edge" of block b#i are defined as the blocks connected to the jth edge of block b#i when blocks b#1 to b#6 are combined into a cube, and the edge connecting any two blocks is defined as the "connecting edge" of the two blocks, where i and j are positive integers, and i=1~6, j=1~4. According to the above definition, for example, for block b#5, the relative block is block b#6, the adjacent blocks are b#1~b#4, the adjacent block corresponding to the first edge (the lower edge) is b#1, the adjacent block corresponding to the second edge (the right edge) is b#2, the adjacent block corresponding to the third edge (the upper edge) is b#3, and the adjacent block corresponding to the fourth edge (the left edge) is b#4. For another example, the corresponding edges of block b#5 and block b#2 are the right edge of block b#5 and the top edge of block b#2, and so on.

[0015] In step S205, one of the vertical perspective blocks b#5 is copied and rotated to generate three copy blocks b#5-1 to b#5-3, and each horizontal perspective block b#2 to b#4 that is not connected to the vertical perspective block b#5 is connected to one of the copy blocks b#5-1 to b#5-3 with a corresponding edge, and another vertical perspective block b#6 is copied and rotated to generate three copy blocks b#6-1 to b#6-3, and each horizontal perspective block b#2 to b#4 that is not connected to the vertical perspective block b#6 is connected to one of the copy blocks b#6-1 to b#6-3 with a corresponding edge to generate a third image IMG3. As shown in Figure 3C, the copy block b#5-1 is block b#5 rotated 90 degrees clockwise, and connected to the upper edge of block b#2 with the edge equivalent to the right side of block b#5; the copy block b#5-2 is block b#5 rotated 180 degrees clockwise, and connected to the upper edge of block b#3 with the edge equivalent to the upper edge of block b#5; the copy block b#5-3 is block b#5 rotated 270 degrees clockwise, and connected to the upper edge of block b#4 with the edge equivalent to the left side of block b#5. Edge connection; the copied block b#6-1 is rotated 90 degrees counterclockwise for block b#6, and connected to the lower edge of block b#2 with the edge equivalent to the right side of block b#6; the copied block b#6-2 is rotated 180 degrees counterclockwise for block b#6, and connected to the lower edge of block b#3 with the edge equivalent to the lower edge of block b#6; the copied block b#6-3 is rotated 270 degrees counterclockwise for block b#6, and connected to the lower edge of block b#4 with the edge equivalent to the left side of block b#6.

[0016] In step S207, two images of 3x3 blocks are taken from the third image IMG3 to generate a fourth image IMG4 and a fifth image IMG5, wherein the fourth image IMG4 and the fifth image IMG5 include a combination of three non-repeating blocks and duplicate blocks and a combination of six repeating blocks and duplicate blocks, and the combination of the six repeating blocks and duplicate blocks includes two block groups, each block group includes a combination of three connected blocks and duplicate blocks, and the two block groups are not connected. Figure 3D As shown, the fourth image IMG4 includes blocks b#1, b#2, b#3, b#5, b#6 and duplicate blocks b#5-1, b#5-2, b#6-1, b#6-2, and the fifth image IMG5 includes blocks b#1, b#3, b#4, b#5, b#6 and duplicate blocks b#5-2, b#5-3, b#6-2, b#6-3. That is, for the fourth image IMG4 and the fifth image IMG5, the repeated block groups are [b#1, b#5, b#6] and [b#3, b#5-2, b#6-2], and these two block groups are not connected to each other.

[0017] In step S209, the depth information of the fourth image IMG4 and the fifth image IMG5 is calculated to generate a first depth information map IMGD1 corresponding to the fourth image IMG4 and a second depth information map IMGD2 corresponding to the fifth image IMG5. Figure 3E In one embodiment, the fourth image IMG4 and the fifth image IMG5 are sent to a neural network to calculate depth information to generate a first depth information map IMGD1 and a second depth information map IMGD2.

[0018] In step S211, a first mixing process is performed on the two overlapping areas a#1, a#2 of the two block groups in the fourth image IMG4 corresponding to the first depth information map IMGD1 and the two overlapping areas a#3, a#4 of the two block groups in the fifth image IMG5 corresponding to the two overlapping areas a#1, a#2 of the two block groups in the fourth image IMG4 corresponding to the first depth information map IMGD1 and the two overlapping areas a#1, a#2 of the two block groups in the fifth image IMG5 corresponding to the second depth information map IMGD2 according to the first overlapping areas a#1, a#2 of the two block groups in the fourth image IMG4 corresponding to the first depth information map IMGD1 and the two overlapping areas a#3, a#4 of the two block groups in the fifth image IMG5. For example, the first mixing process is to average the depth value of each pixel in the overlapping area a#1 corresponding to blocks b#1, b#5, and b#6 in the first depth information map with the depth value of the pixel corresponding to the overlapping area a#4 corresponding to blocks b#1, b#5, and b#6 in the second depth information map according to the inverse proportion of the horizontal distance from the pixel to the block boundary as the depth value of the corresponding pixel of the first depth image map and the second depth image map, and to average the depth value of each pixel in the overlapping area a#2 corresponding to blocks b#3, b#5-2, and b#6-2 in the first depth information map with the depth value of the pixel corresponding to the overlapping area a#3 corresponding to blocks b#3, b#5-2, and b#6-2 in the second depth information map according to the inverse proportion of the horizontal distance from the pixel to the block boundary as the depth value of the corresponding pixel of the first depth image map and the second depth image map. It should be noted that the algorithm used in the first blending process in the above example is the inverse distance average, but the present invention is not limited thereto. In alternative embodiments, an average value may be calculated or Poisson blending may be used.

[0019] In step S213, six parts corresponding to the six blocks b#1-b#6 in the second image IMG2 are extracted from the first depth information map IMGD1 and the second depth information map IMGD2 to generate six depth information blocks db#1-db#6. Figure 3F shown.

[0020] In step S215, for each depth information block db#5, db#6 corresponding to the vertical viewing blocks b#5, b#6, the four depth information blocks db#1-db#4 corresponding to the horizontal viewing blocks b#1-b#4 are connected to the four edges of the depth information blocks db#5, db#6 corresponding to the vertical viewing blocks b#5, b#6 with the edges corresponding to the corresponding edges between the horizontal viewing blocks b#1-b#4 and the vertical viewing blocks b#5, b#6, as shown in FIG. Figure 3G shown.

[0021] In step S217, a second blending process is performed on a plurality of connections between the depth information blocks db#5 and db#6 corresponding to the vertical viewing blocks b#5 and b#6 and the depth information blocks db#1 to db#4 corresponding to the horizontal viewing blocks b#1 to b#4 to generate a third depth information map IMGD3 and a fourth depth information map IMGD4, as shown in FIG. Figure 3G The second blending process is, for example, Poisson blending, or blurring the boundaries between blocks b#5, b#6 and horizontal viewing blocks b#1 to b#4.

[0022] In step S219, a plurality of depth information blocks db#1 to db#6 corresponding to the blocks b#1 to b#6 of the second image IMG2 are extracted from the third depth information map IMGD3 and the fourth depth information map IMGD4, and the depth information blocks db#1 to db#6 are spliced ​​into a fifth depth information map IMGD5 having a cubemap format according to the positional relationship between the blocks b#1 to b#6 of the second image IMG2, as shown in FIG. Figure 3H shown.

[0023] In step S221, the fifth depth information map IMGD5 is converted from the cubemap format into a sixth depth information map IMGD6 having the first format, such as Figure 3H shown.

[0024] In this way, a sixth depth information map IMGD6 having the depth information of the first image IMG1 can be obtained. The sixth depth information map IMGD6 can be used in combination with the first image IMG1 for various applications. In one embodiment, the sixth depth information map IMGD6 can be used in combination with the first image IMG1 for a virtual reality device. In one embodiment, the processing device 10 is a virtual reality device, and further includes a display unit and a posture detection unit coupled to the processing unit. The posture detection unit is used to obtain a rotation angle of the processing device. The method for processing the surround image may further include Figure 4 Steps shown.

[0025] In step S401, a resolution and a radius are determined according to the sixth depth information map IMGD6, and a spherical mesh is generated according to the resolution and the radius. In actual application, the processing unit will further determine the resolution and the radius according to the performance of the display unit of the virtual reality and the actual application configuration.

[0026] In step S403, for each pixel of the sixth depth information map IMGD6, the coordinates mapped to the spherical grid are calculated, and the distance of the coordinates from the center of the spherical grid is adjusted according to the depth value of the pixel.

[0027] In step S405 , for each pixel of the first image IMG1 , the coordinates mapped to the spherical grid are calculated, and the coordinates are filled with colors according to the color information of the pixel.

[0028] The coordinate conversion between the pixels in the first format and the spherical grid can be calculated according to a method well known to those skilled in the art, and the present invention is not limited thereto.

[0029] In step S407, the rotation angle of the virtual reality device is obtained. In one embodiment, the posture detection unit may include an inertial measurement module (IMU) and / or a spatial positioning module (lighthouse, SLAM tracking), and the rotation angle is obtained therefrom.

[0030] In step S409, the user's observation point is set at the center of the sphere, the sight line direction is set according to the rotation angle, and the observation point is moved a specific distance in the opposite direction of the sight line direction.

[0031] In step S411, an image on a spherical grid within a field of view is displayed according to the position of the observation point and the direction of sight.

[0032] By moving the observation point farther away, the displayed image can have a sense of distance due to the depth information, thereby enhancing the user's visual perception of the distance of the scene in the displayed image.

[0033] The specific embodiments described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A device for processing surround images, comprising: Processing unit; A storage unit, coupled to the processing unit, for storing a computer-readable medium, which, when executed by the processing unit, causes the processing unit to execute: Obtaining the first image; Converting the first image from a first format into a second image having a cubemap format, wherein the second image includes four horizontal viewing blocks and two vertical viewing blocks to form a cube, and four connecting edges of each vertical viewing block are connected to the horizontal viewing blocks; Copying two vertical viewing blocks of the second image, wherein a first vertical viewing block among the vertical viewing blocks is copied three times to generate three first copy blocks, and a second vertical viewing block among the vertical viewing blocks is copied three times to generate three second copy blocks, wherein the second image includes three unconnected horizontal viewing blocks that are not connected to the two vertical viewing blocks, and each of the unconnected horizontal viewing blocks is connected to a lower edge of each of the first copy blocks with an upper edge, and is connected to an upper edge of each of the second copy blocks with a lower edge, so as to generate a third image; Extract two images of 3×3 blocks from the third image to generate a fourth image and a fifth image, wherein the two repeated block groups of the fourth image are projected onto the first depth information map with a first set of overlapping areas, and the two repeated block groups of the fifth image are projected onto the second depth information map with a second set of overlapping areas; calculating depth information of the fourth image and the fifth image to generate a first depth information map corresponding to the fourth image and a second depth information map corresponding to the fifth image; Performing a first mixing process on a first group of overlapping areas of the first depth information map and a second group of overlapping areas of the second depth information map; Extracting a plurality of portions of the blocks corresponding to the second image from the first depth information map and the second depth information map to generate a plurality of depth information blocks; For each of the depth information blocks corresponding to the vertical viewing block, the four depth information blocks corresponding to the four horizontal viewing blocks are respectively connected to four sides of the depth information block corresponding to the vertical viewing block through four connecting edges of one of the two vertical viewing blocks; performing a second blending process on a plurality of connection points between the depth information blocks corresponding to the vertical viewing blocks and the depth information blocks corresponding to the horizontal viewing blocks to generate a third depth information map and a fourth depth information map; extracting the depth information blocks corresponding to the blocks of the second image from the third depth information map and the fourth depth information map, and splicing the depth information blocks into a fifth depth information map having the cube map format; and The fifth depth information map is converted from the cube map format into a sixth depth information map having the first format.

2. The processing device as described in claim 1 further includes a display unit and a posture detection unit, which are respectively coupled to the processing unit, and the posture detection unit is used to obtain a rotation angle, wherein the processing unit images the display unit according to the first image, the sixth depth information map and the rotation angle.

3. The processing device of claim 1 , wherein when the computer readable medium is executed by the processing unit, the processing unit further causes the processing unit to execute: Calculating a resolution and a radius according to the sixth depth information map, and generating a spherical grid according to the resolution and the radius; For each pixel of the sixth depth information map, calculate the coordinates mapped to the spherical grid, and adjust the distance of the coordinates from the center of the spherical grid according to the depth value of the pixel; For each pixel of the first image, calculating the coordinates mapped to the spherical grid, and filling the coordinates with colors according to the color information of the pixel; The user's observation point is set at the center of the sphere, a sight line direction is set according to the rotation angle, and the observation point is moved a specific distance in the opposite direction of the sight line direction; as well as The image on the spherical grid within the field of view is displayed according to the position of the observation point and the sight direction. The processing device as claimed in claim 1 , wherein the second mixing process is a Boisson mixing process.

5. The processing device as described in claim 1, wherein when calculating the depth information of the fourth image and the fifth image to generate a first depth information map corresponding to the fourth image and a second depth information map corresponding to the fifth image, the fourth image and the fifth image are sent to a type of neural network for calculation to obtain the first depth information map and the second depth information map.

6. A method for processing an ambient image, comprising: Acquire a first image; Converting the first image from a first format into a second image having a cubemap format, wherein the second image includes four horizontal viewing blocks and two vertical viewing blocks to form a cube, and four connecting edges of each vertical viewing block are connected to the horizontal viewing blocks; Copying two vertical viewing blocks of the second image, wherein a first vertical viewing block among the vertical viewing blocks is copied three times to generate three first copy blocks, and a second vertical viewing block among the vertical viewing blocks is copied three times to generate three second copy blocks, wherein the second image includes three unconnected horizontal viewing blocks that are not connected to the two vertical viewing blocks, and each of the unconnected horizontal viewing blocks is connected to a lower edge of each of the first copy blocks with an upper edge, and is connected to an upper edge of each of the second copy blocks with a lower edge, so as to generate a third image; Extract two images of 3×3 blocks from the third image to generate a fourth image and a fifth image, wherein the two repeated block groups of the fourth image are projected onto the first depth information map with a first set of overlapping areas, and the two repeated block groups of the fifth image are projected onto the second depth information map with a second set of overlapping areas; calculating depth information of the fourth image and the fifth image to generate a first depth information map corresponding to the fourth image and a second depth information map corresponding to the fifth image; Performing a first mixing process on a first group of overlapping areas of the first depth information map and a second group of overlapping areas of the second depth information map; Extracting a plurality of portions of the blocks corresponding to the second image from the first depth information map and the second depth information map to generate a plurality of depth information blocks; For each of the depth information blocks corresponding to the vertical viewing block, the four depth information blocks corresponding to the four horizontal viewing blocks are respectively connected to four sides of the depth information block corresponding to the vertical viewing block through four connecting edges of one of the two vertical viewing blocks; performing a second blending process on a plurality of connection points between the depth information blocks corresponding to the vertical viewing blocks and the depth information blocks corresponding to the horizontal viewing blocks to generate a third depth information map and a fourth depth information map; extracting the depth information blocks corresponding to the blocks of the second image from the third depth information map and the fourth depth information map, and splicing the depth information blocks into a fifth depth information map having the cube map format; and The fifth depth information map is converted from the cube map format into a sixth depth information map having the first format. 7 . The processing method as claimed in claim 6 , further comprising obtaining a rotation angle by a posture detection unit, and imaging on a display unit according to the first image, the sixth depth information map and the rotation angle.

8. The processing method according to claim 6, further comprising: Calculating a resolution and a radius according to the sixth depth information map, and generating a spherical grid according to the resolution and the radius; For each pixel of the sixth depth information map, calculate the coordinates mapped to the spherical grid, and adjust the distance of the coordinates from the center of the spherical grid according to the depth value of the pixel; For each pixel of the first image, calculating the coordinates mapped to the spherical grid, and filling the coordinates with colors according to the color information of the pixel; The user's observation point is set at the center of the sphere, the sight line direction is set according to the rotation angle, and the observation point is moved a specific distance in the opposite direction of the sight line direction; as well as The image on the spherical grid within the field of view is displayed according to the position of the observation point and the sight direction.

9. The processing method of claim 6, wherein the second mixing process is a Boisson mixing process.

10. The processing method as described in claim 6, wherein when calculating the depth information of the fourth image and the fifth image to generate a first depth information map corresponding to the fourth image and a second depth information map corresponding to the fifth image, the fourth image and the fifth image are sent to a type of neural network for calculation to obtain the first depth information map and the second depth information map.

Images