Quick processing method and system for naked-eye 3D figure photo

Abstract

The invention discloses a quick processing method and system for a naked-eye 3D figure photo. The method comprises the following steps: extracting depth information of a viewpoint image and generating a three-dimensional point cloud; and interpolating viewpoints between the viewpoints of the two viewpoint images, and generating interpolated viewpoint images corresponding to the interpolated viewpoints. And carrying out pixel extraction and splicing on each viewpoint image to form a sub-view image corresponding to the naked eye 3D figure photo. According to the method, the interpolation viewpoint image is generated through automatic calculation between the left viewpoint image and the right viewpoint image, the viewpoint images are spliced alternately to form the split-view image, and by means of the resolution parameter of the printer and the grating line number of the lenticular grating plate, the interpolation viewpoint image is formed on the grating alignment lines on the two sides of the split-view image. And a worker can conveniently and quickly obtain a 3D naked eye figure photo on the coverage lenticular grating plate of the automatically generated split view image. According to the invention, the time for generating an intact sub-view image is about 60 seconds, and the requirement of shooting and taking at once in tourist industry is met.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision and image processing, and particularly to a method and system for quickly processing naked-eye 3D human photos. Background Art

[0002] In the tourism industry, naked-eye 3D photos usually adopt the method of covering a multiplexed image with a lenticular grating. To generate a naked-eye 3D photo, first, it is necessary to generate quantitative interpolation viewpoint images as a sequence of images, and then mix the sequence of images into a multiplexed image. Mixing the sequence of images is a process of extracting appropriate pixels from the quantitative interpolation viewpoint images, using the width of each pixel as a step, and then splicing them into a new image in a certain order. The multiplexed image contains multi-viewpoint information of the original image.

[0003] Currently, among the methods for generating naked-eye 3D photos in the tourism industry, the most common method is to generate N interpolation viewpoint images based on 2D photos by manual rotation, assign depth information to the spliced image according to personal experience, and then manually select the sequence of images and use tools such as to manually generate the spliced image. This method relies on

[0004] manually generating viewpoint images, which takes as long as 2 - 5 hours, and the angle error rate of the viewing angle transformation between adjacent interpolation viewpoint images is relatively large. In addition, the generation of depth information relies on manual experience annotation, resulting in a distortion of the three-dimensional sense and making it difficult to meet the requirements of high-quality 3D photos. At the same time, the calculation of the splicing step also relies on manual experience, which is prone to image ghosting and further reduces the visual effect of the naked-eye 3D photo. These defects severely limit the application of traditional methods in the tourism industry, and there is an urgent need for an efficient and accurate automated solution. Summary of the Invention

[0005] The purpose of the present invention is to provide a method and system for generating naked-eye 3D human photos with a fast processing speed, high photo quality, and suitable for tourist photography scenarios for 3D naked-eye human photos.

[0006] In the first aspect, the present invention provides a method for quickly processing naked-eye 3D human photos, and the process is as follows:

[0007] Extract the depth information of at least two viewpoint images and generate a three-dimensional point cloud.

[0008] Establish an observation arc line between the viewpoints of the initial at least two viewpoint images, rotate and interpolate viewpoints on the observation arc line, and generate interpolated viewpoint images corresponding to the interpolated viewpoints.

[0009] Pixel extraction and stitching are performed on each viewpoint image to form a parallax image corresponding to the naked-eye 3D portrait photo.

[0010] Preferably, the process of generating the three-dimensional point cloud is as follows:

[0011] Pixel-level matching is performed on two viewpoint images through a stereo matching algorithm to generate a disparity map.

[0012] Color mapping is performed on the disparity map to generate a color disparity image; the color disparity image is converted into a three-dimensional point cloud; the three-dimensional depth information of each point in the three-dimensional point cloud is extracted.

[0013] Preferably, the process of interpolating the rotational viewpoint is as follows: A rotational reference point is selected in the three-dimensional point cloud, and an observation arc line is established; the observation arc line takes the rotational reference point as the center and passes through the viewpoints of the initial two viewpoint images. One or more viewpoints are interpolated on the observation arc line.

[0014] Preferably, the process of generating the interpolated viewpoint image is as follows: According to the position of the interpolated viewpoint on the observation arc line, an interpolation quaternion q corresponding to the interpolated viewpoint is generated t ; According to the interpolation quaternion q t The coordinates p of each point in the three-dimensional point cloud n are rotated to obtain the coordinates p' of each point in the three-dimensional point cloud under the perspective of the interpolated viewpoint n . The three-dimensional point cloud under the perspective of the interpolated viewpoint is projected onto a pixel grid and adjusted to obtain a primary interpolated viewpoint image.

[0015] Preferably, for the initial viewpoint image and the interpolated viewpoint images interpolated and generated on the observation arc line, one or more secondary interpolated viewpoint images are generated between the viewpoint images corresponding to adjacent viewpoints through a large motion frame interpolation method.

[0016] Preferably, the process of pixel extraction and stitching for the viewpoint image is as follows: The resolution parameter of the printer is divided by the number of grating lines of the cylindrical lens grating plate to obtain the optimal step size ; According to the arrangement order of each viewpoint image, different pixel column intervals are respectively extracted on different viewpoint images; the pixel width of the pixel column interval is equal to the optimal step size . The pixel column intervals extracted on each viewpoint image are stitched in sequence to form a parallax image

[0017] Preferably, grating alignment lines are generated on both sides of the parallax image; the distance between the grating alignment line and the edge of the parallax image is an integer multiple of the optimal step size . According to the position of the grating alignment line, a cylindrical lens grating plate is covered and fixed on the parallax image to obtain the naked-eye 3D portrait photo

[0018] Preferably, the rotation reference point is the position of the center point of the human face in the viewpoint image in the three-dimensional point cloud.

[0019] Preferably, the viewpoint image is preprocessed. The preprocessing includes binarization, dilation, and Gaussian blur processing.

[0020] Preferably, the initial at least two viewpoint images are obtained by shooting with a binocular camera.

[0021] Preferably, the rotation reference point is the position of the focus of the initial viewpoint image in the three-dimensional point cloud.

[0022] In a second aspect, the present invention provides a fast processing system for nude 3D human photos, which is used to execute the foregoing method; the system includes a binocular camera, a point cloud generation module, an interpolation module, a stitching module, and a printing output module. The binocular camera is used to shoot the initial viewpoint image; the point cloud generation module is used to extract the depth information of the viewpoint image and generate a three-dimensional point cloud. The interpolation module is used to generate an interpolated viewpoint image; the stitching module is used to perform pixel extraction and stitching on each viewpoint image; the printing output module is used to print the stereoscopic images.

[0023] In a third aspect, the present invention provides a computer device, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor, and is characterized in that: the memory stores the computer program; the processor executes the foregoing method for fast processing of nude 3D human photos.

[0024] The present invention has the following beneficial effects.

[0025] 1. The present invention automatically calculates and generates an interpolated viewpoint image between the left and right two viewpoint images, and uses multiple viewpoint images to intersperse and stitch to form a stereoscopic image, and uses the printer resolution parameter and the grating line number of the lenticular grating plate to align the grating on both sides of the stereoscopic image, which is convenient for the staff to quickly cover the lenticular grating plate on the automatically generated stereoscopic image to obtain a 3D nude human photo. The time for the present invention to generate a complete stereoscopic image is about 60 seconds, which meets the on-site instant photo-taking demand of the tourism industry.

[0026] 2. The present invention extracts depth information based on the left and right two viewpoint images, generates a three-dimensional point cloud, and further obtains an observation arc line for generating an interpolated image; the viewpoint images generated along the observation arc line can simulate the binocular parallax motion characteristics of the human eye, and can make the stereoscopic image synthesized by multiple viewpoint images more three-dimensional.

[0027] 3. The 3D nude photos generated by the present invention have the advantages of strong 3D effect and fast processing speed, which meet the high-quality demand of tourists for photos and the demand for quickly obtaining photos. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The present invention will be further described below in conjunction with the accompanying drawings and embodiments. In the accompanying drawings:

[0029] Figure 1 is the overall flowchart of the embodiment of the present invention;

[0030] Figure 2 is the original photo obtained in step S1 of the embodiment of the present invention;

[0031] Figure 3 is the partial interpolated view point image obtained in step S4 of the embodiment of the present invention;

[0032] Figure 4 is the enlarged view of the perspective difference between different interpolated view point images in the embodiment of the present invention;

[0033] Figure 5 is the split view image obtained in step S63 of the embodiment of the present invention. Specific Embodiments

[0034] In order to better understand the principle and steps of the present invention, the specific embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings.

[0035] The embodiment is as Figure 1 shown. A method for generating a naked-eye 3D portrait photo includes the following steps:

[0036] S1. Use a binocular camera to collect images of the same object to obtain the original photo as Figure 2 shown; the original photo includes the left-eye portrait photo and the right-eye portrait photo taken by the binocular camera, and uses them as the leftmost view point image and the rightmost view point image respectively.

[0037] S2. Extract the 2D coordinates of the focus when the binocular camera takes pictures to facilitate finding the rotation reference point for rotational interpolation (combining the quaternion spherical linear interpolation algorithm and the FILM frame filling model). In this embodiment, the three-dimensional coordinates of the focus are used as the rotation reference point.

[0038] S3. Use a binocular stereo matching model to extract the depth information of the leftmost view point image and the rightmost view point image respectively and generate a three-dimensional point cloud, specifically as follows:

[0039] S31. Based on the stereo matching algorithm, perform pixel-level matching on the leftmost view point image and the rightmost view point image to generate a disparity map. In this embodiment, the stereo matching algorithm uses the SGM (semi-global matching) algorithm.

[0040] S32. Perform color mapping on the disparity map, perform pseudo-color coding on the disparity data based on a preset color space conversion model, generate a color disparity image with a visual enhancement effect, convert the color-mapped disparity map into a three-dimensional point cloud, and obtain the three-dimensional depth information of each pixel point in the three-dimensional point cloud;

[0041] In this embodiment, the ratio of the pixel distance x to the actual distance X between any two points in the three-dimensional point cloud is 1:16; the expression of the Euclidean distance d between any two points in the three-dimensional point cloud is as follows:

[0042]

[0043] Where are the horizontal coordinates of the two points respectively; are the vertical coordinates of the two points respectively; are the vertical coordinates of the two points respectively.

[0044] S4. Generate N interpolated viewpoint images based on the rotation interpolation method, and use the interpolated viewpoint images as a sequence of images, specifically as follows:

[0045] S41. According to the two focal point 2d coordinates obtained in step S2 and the three-dimensional depth information of the point cloud obtained in step S3, obtain the three-dimensional coordinates of the focal points in the point cloud as the rotation reference points.

[0046] S42. Generate interpolated viewpoint images based on an arc path. Determine one or more interpolated viewpoint coordinates in the point cloud space. The interpolated viewpoint coordinates are on the observation arc line. The observation arc line takes the rotation reference point obtained in step S41 as the center of the circle and passes through the positions of the two lenses of the binocular camera. For each interpolated viewpoint coordinate, generate a first-level interpolated viewpoint image; the first-level interpolated viewpoint image is a photo taken with the interpolated viewpoint coordinate as the shooting point.

[0047] In this embodiment, determine an interpolated viewpoint coordinate, denoted as the middle viewpoint coordinate; the middle viewpoint coordinate is located at the center point of the two lens positions (i.e., the left viewpoint and the right viewpoint) on the observation arc line.

[0048] The process of generating the first-level interpolated viewpoint image for the middle viewpoint coordinate is as follows:

[0049] (1) Determine the interpolation ratio according to the position of the set interpolated viewpoint coordinate between the two rear viewpoints ; the interpolation ratio represents the ratio of the viewing angle difference from the interpolated viewpoint to the left viewpoint to the viewing angle difference from the left viewpoint to the right viewpoint; in this embodiment, since the interpolated viewpoint is located at the center point of the two lens positions on the observation arc line, so ; in some other embodiments, the interpolation ratio One or more other values can also be taken, such as 1 / 4, 1 / 3, 2 / 3, 3 / 4 or other reasonable numerical values.

[0050] (2) Establish the direction vector of the rotation axis in three-dimensional space ; The rotation axis is a straight line passing through the rotation reference point and perpendicular to the plane where the observed circular arc line is located. In this embodiment, taking the focus as the rotation reference point and the three-dimensional coordinate origin, the rotation axis is the Z-axis. .

[0051] (3) Establish the spherical linear interpolation quaternion q of the interpolation view point t as follows:

[0052]

[0053] Among them, and are the view point quaternions corresponding to the leftmost view point image and the rightmost view point image respectively, representing the initial rotation states of the leftmost view point image and the rightmost view point image. The quaternion is calculated by the following formula: ; ; is the viewing angle of the view point; satisfying ; is the viewing angle difference between the leftmost view point image and the rightmost view point image. In this embodiment, it is stipulated that the viewing angle of the leftmost view point image is 0, and the viewing angle of the rightmost view point image is .

[0054] (4) Use the spherical linear interpolation quaternion q t to perform position rotation adjustment on each point in the original three-dimensional point cloud respectively; The specific rotation method is: represent the coordinates of each point in the original three-dimensional point cloud as , where n = 1, 2,..., N, and N is the total number of points in the three-dimensional point cloud;

[0055] Represent the quaternion q t as ; Among them, w t is the real part, x t , y t , z t are the imaginary parts, and i, j, k are three imaginary units; Convert each point in the three-dimensional point cloud into the form of a pure imaginary quaternion ; Calculate the pure imaginary quaternion corresponding to the coordinates of each point in the three-dimensional point cloud under the viewing angle of the interpolation view point according to the following formula

[0056]

[0057] Among them, , Extract the pure imaginary quaternion and obtain the coordinates of each point in the three-dimensional point cloud from the imaginary part under the new perspective. .

[0058] (5) Project the three-dimensional point cloud obtained in step (4) under the new perspective onto the pixel grid corresponding to the interpolation view point, use the bicubic interpolation method to supplement the sawteeth or holes, and use the depth buffer (Z-Buffer) method to solve the visibility conflict when multiple three-dimensional points are projected onto the same pixel. Generate the first-level interpolation view point image.

[0059] In this step, since the established view point change curve is arc-shaped, it can simulate the binocular parallax motion characteristics of the human eye, and can make the anaglyph image synthesized from multiple view point images more three-dimensional.

[0060] S43. After step S41, more than three view point images are generated. Further generate the second-level interpolation view point images between adjacent two view point images.

[0061] In this embodiment, through the large motion frame interpolation method (FILM, Frame Interpolation for LargeMotion), one or more second-level interpolation view point images are generated between adjacent two view point images. This step can further encrypt the view point images and reduce the viewing angle difference between adjacent view point images, so that the transition of different positions in the finally generated anaglyph image is smoother.

[0062] In this step, by generating the interpolation view point images, the large motion occlusion area can be repaired, the observer's field of view can be expanded, and the realism and clarity of the frame can be improved.

[0063] The efficiency of generating view point images by the large motion frame interpolation method is relatively high; and, since the viewing angle difference between adjacent view point images has been reduced after generating the first-level interpolation view point images in step S42, the view point images directly generated by the large motion frame interpolation method are still close to the observation arc line, and can still provide good three-dimensionality for the anaglyph image.

[0064] S44. Arrange all view point images in a sequence diagram. The number N of images in the sequence diagram is determined according to the smoothness requirement of the anaglyph image; the larger the number N of images, the higher the smoothness of the anaglyph image, but the higher the required image processing computing power, and the longer the time required to generate the anaglyph image.

[0065] Some of the interpolation view point images shown in step S4 are as Figure 3 and 4 shown; as can be seen from Figure 4 , the human figure occlusion relationships of different interpolation view point images are different, which helps to improve the three-dimensionality of the final 3D photo.

[0066] S5. Optimize the sequence diagram obtained in step S44 through binarization, dilation, and Gaussian blur processing. The specific process is as follows:

[0067] S51. Perform binarization processing on the sequence diagram to convert the image into a black-and-white binary image.

[0068] S52. Perform dilation processing on the sequence diagram. Use a 3×3 rectangular kernel to perform morphological dilation on the binary image to enhance the edge continuity.

[0069] S53. Perform Gaussian blur processing on the sequence diagram. Apply a Gaussian filter to smooth the noise, retain the main body contour, and finally enhance the image quality and three-dimensional sense.

[0070] S6. Calculate the optimal step size for stitching the obtained stereoscopic image, stitch the sequence diagram processed in step S5 to obtain the stereoscopic image, and generate a raster alignment line, which is then uploaded to the printer. The specific process is as follows:

[0071] S61. Automatically optimize and adjust the size (including length and width) of the sequence diagram processed in step S5 to the target size according to the target size of the printed photo.

[0072] S62. Calculate the optimal step size for stitching according to the printer resolution parameter to ensure that the visual effect is natural after the stitched image covers the lenticular grating plate;

[0073] Optimal step size Is the number of horizontal pixels for each step of stitching, and its expression is as follows:

[0074]

[0075] Where, Is the printer resolution parameter (specifically, the number of physical ink dots that can be printed per unit length); Is the grating line number of the lenticular grating plate (specifically, the number of grating columns per unit length).

[0076] S63. Stitch the sequence diagram according to the optimal step size , and obtain the stereoscopic image as follows:

[0077] Extract the reserved area on each viewpoint image as follows:

[0078] S i =[ k · N·γ+( i-1)· γ+1, k · N·γ+ i·γ ]

[0079] where k = 0, 1, 2,... ; i = 1, 2,... N; P is the total number of pixel columns of the target photo; N is the number of viewpoint images; is the serial number of the viewpoint image; is the set of column serial numbers of the reserved area of the viewpoint image i.

[0080] The reserved areas of each viewpoint image are spliced in sequence to obtain a multi-view image. The multi-view image is as Figure 5 shown. The 3D effect of the multi-view image cannot be directly observed, and its 3D effect appears after covering with a lenticular grating sheet. The multi-view image occupies a large amount of memory, and the memory occupied by the multi-view image required to print a six-inch photo is between 120 Mb and 160 Mb.

[0081] After completing the step S6, the multi-view image is printed, covered with a lenticular grating sheet and framed, and thus the complete finished product of the present invention is obtained.

[0082] S64. Generate grating alignment lines on both sides of the multi-view image; the grating alignment lines are at a distance of .

[0083] S65. Use a printer to print out the multi-view image.

[0084] S66. Stick a lenticular grating sheet on the multi-view image, a naked-eye 3D portrait photo.

[0085] In some embodiments, instead of using the focus as the rotation reference point, the center point of one of the faces in the leftmost viewpoint image and the rightmost viewpoint image is used as the rotation reference point. The position of the face in the image is obtained by identifying through an application network.

Claims

1. A method for fast processing naked eye 3D character photos, characterized in that: The method is as follows: Extracting depth information of at least two viewpoint images and generating a three-dimensional point cloud; Establishing an observation arc line between viewpoints of at least two initial viewpoint images, rotating the interpolated viewpoints on the observation arc line, and generating interpolated viewpoint images corresponding to each interpolated viewpoint; Pixels of each viewpoint image are extracted and spliced ​​to form a dichroic image corresponding to the naked-eye 3D character photo.

2. The method for rapidly processing naked eye 3D character photos according to claim 1, characterized in that: The process of generating a 3D point cloud is as follows: The two viewpoint images are matched at the pixel level through the stereo matching algorithm to generate a disparity map; Perform color mapping on the disparity map to generate a color disparity image; convert the color disparity image into a three-dimensional point cloud; and extract the three-dimensional depth information of each point in the three-dimensional point cloud.

3. The method for rapidly processing naked eye 3D character photos according to claim 1, characterized in that: The process of the rotation interpolation viewpoint is as follows: selecting a rotation reference point in the three-dimensional point cloud and establishing an observation arc line; the observation arc line takes the rotation reference point as the center of the circle and passes through the viewpoints of the initial two viewpoint images; Interpolate one or more viewpoints on the observation arc.

4. The method for rapidly processing naked eye 3D character photos according to claim 3, characterized in that: The process of generating the interpolated viewpoint image is as follows: according to the position of the interpolated viewpoint on the observation arc line, the interpolated quaternion corresponding to the interpolated viewpoint is generated. q t ; Based on the interpolated quaternion q t The coordinates of each point in the 3D point cloud p n Rotate to obtain the coordinates of each point in the 3D point cloud from the perspective of the interpolation viewpoint p ' n ; Project the three-dimensional point cloud from the perspective of the interpolation viewpoint into the pixel grid and adjust it to obtain a first-level interpolation viewpoint image.

5. The method for rapidly processing naked-eye 3D character photos according to claim 3, characterized in that: For the initial viewpoint image and the primary interpolated viewpoint image generated by interpolation on the observation arc line, one or more secondary interpolated viewpoint images are generated between the viewpoint images corresponding to two adjacent viewpoints through the large motion frame interpolation method.

6. The method for rapidly processing naked eye 3D character photos according to claim 1, characterized in that: The process of pixel extraction and splicing of viewpoint images is as follows: the resolution parameter of the printer is divided by the number of grating lines of the cylindrical lens grating plate to obtain the optimal step length γ; different pixel column intervals are extracted from different viewpoint images according to the arrangement order of each viewpoint image; the pixel width of the pixel column interval is equal to the optimal step length γ; the pixel column intervals extracted from each viewpoint image are sequentially spliced ​​to form a divergent view image.

7. The method for rapidly processing naked eye 3D character photos according to claim 1, characterized in that: Grating alignment lines are generated on both sides of the diffraction image; the distance between the grating alignment lines and the edge of the diffraction image is a positive integer multiple of the optimal step length γ; according to the position of the grating alignment lines, a cylindrical grating plate is covered and fixed on the diffraction image to obtain a naked-eye 3D character photo.

8. The method for rapidly processing naked-eye 3D character photos according to claim 1, characterized in that: The rotation reference point is the position of the face center point in the viewpoint image in the three-dimensional point cloud.

9. A naked-eye 3D character photo fast processing system, characterized by: Used to execute the method as claimed in claim 1; the system includes a binocular camera, a point cloud generation module, an interpolation module, a stitching module and a print output module; the binocular camera is used to capture an initial viewpoint image; the point cloud generation module is used to extract depth information of the viewpoint image and generate a three-dimensional point cloud; the interpolation module is used to generate an interpolated viewpoint image; the stitching module is used to extract and stitch pixels of each viewpoint image; the print output module is used to print a diffraction image.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: The memory stores computer programs; the processor executes a method for quickly processing naked-eye 3D character photos as described in any one of claims 1-8.

Images