Three-dimensional imaging method, storage medium and electronic device

Abstract

The embodiment of the invention provides a three-dimensional imaging method, a storage medium and an electronic device. The method comprises the following steps: acquiring a first image of a first camera and a second image of a second camera; under the condition that the parallax between the first image and the second image is larger than a preset threshold value, the first image and / or the second image are / is controlled to move, so that the overlapping area of the first image and the overlapping area of the second image meet the preset requirement; supplementing the non-overlapping area of the first image and the non-overlapping area of the second image to form a third image and a fourth image; and synthesizing the third image and the fourth image into a three-dimensional image. According to the invention, the problem that the imaging effect is poor during short-distance photographing is solved, and the effect of improving the imaging quality is further achieved.

Description

Technical Field

[0001] The embodiments of the present invention relate to the technical field of image processing, and in particular, to a three-dimensional imaging method, a storage medium and an electronic device. Background Art

[0002] Traditional photography technology can only capture two-dimensional images of objects, and cannot present the depth and three-dimensional sense of objects. In order to solve this problem, people began to study the use of dual cameras to simulate the stereoscopic vision effect of the human eye. The principle of dual cameras to shoot three-dimensional pictures is: use two cameras to take pictures at the same time to get two images. These two images need to leave parallax to form left and right eye images. Then the left and right eye images are synthesized into one image using a three-dimensional algorithm. With the help of a naked-eye three-dimensional screen, users can see the effect of three-dimensional imaging. However, as the distance between the camera and the object being photographed changes, the parallax between the two images will also change in real time, which can easily make the three-dimensional imaging effect poor. Summary of the invention

[0003] The embodiments of the present invention provide a three-dimensional imaging method, a storage medium and an electronic device to at least solve the problem of poor imaging effect when taking photos at a close distance in the related art.

[0004] According to an embodiment of the present invention, there is provided a three-dimensional imaging method, comprising:

[0005] Acquire a first image from the first camera and a second image from the second camera;

[0006] When the parallax between the first image and the second image is greater than a preset threshold, controlling the first image and / or the second image to move so that an overlapping area of ​​the first image and an overlapping area of ​​the second image meet a preset requirement;

[0007] Performing a padding operation on a non-overlapping area of ​​the first image and a non-overlapping area of ​​the second image to form a third image and a fourth image;

[0008] A three-dimensional image is synthesized based on the third imaging and the fourth imaging.

[0009] According to yet another embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the steps of any one of the above method embodiments when run.

[0010] According to another embodiment of the present invention, an electronic device is provided, which may include a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0011] Through the present invention, by detecting and adjusting the parallax, the first image and / or the second image is moved so that the overlapping area between the two images meets the preset requirements, and the non-overlapping area is filled, and finally a three-dimensional image with good stereoscopic effect is synthesized. Therefore, the problem of poor imaging effect when taking photos at a close distance can be solved, thereby achieving the effect of improving the imaging quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 It is a schematic diagram of a hardware structure block diagram of a mobile terminal of a three-dimensional imaging method according to an embodiment of the present invention;

[0013] Figure 2 It is a schematic diagram of two images formed by two cameras simultaneously photographing the same object;

[0014] Figure 3 It is a schematic diagram of two images with appropriate parallax;

[0015] Figure 4 It is a schematic diagram of two images with inappropriate parallax;

[0016] Figure 5 is a schematic flow chart of a three-dimensional imaging method according to an embodiment of the present invention;

[0017] Figure 6 is a schematic diagram of converting two images with large parallax into two images with appropriate parallax according to an embodiment of the present invention;

[0018] Figure 7 is a schematic diagram of a process of detecting a parallax between two images according to an embodiment of the present invention;

[0019] Figure 8 It is a schematic diagram of the position of feature pixels, the position of feature pixels in two images, and the positions of two cameras;

[0020] Fig. 9 is a schematic diagram of a process of matching characteristic pixel points of two images according to an embodiment of the present invention;

[0021] Fig.10 is a schematic diagram of the positional relationship between the first camera, the second camera, the first imaging area, the second imaging area, the first characteristic pixel point, and the second characteristic pixel point;

[0022] Fig.11 It is a schematic diagram of a structural block diagram of a three-dimensional imaging device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0023] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings and in combination with the embodiments.

[0024] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0025] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 FIG. 1 is a hardware structure block diagram of a mobile terminal of a three-dimensional imaging method according to an embodiment of the present invention. Figure 1 As shown, the mobile terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is for illustration only and does not limit the structure of the mobile terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0026] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as a computer program corresponding to a three-dimensional imaging method in an embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the mobile terminal via a network. Examples of the above network may include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0027] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 may include a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 may be a Radio Frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0028] Figure 2 It is a schematic diagram of two images formed by two cameras simultaneously shooting the same object, such as Figure 2 As shown, the left dotted box is the imaging of one camera, and the right dotted box is the imaging of another camera. There will be a displacement difference between the left imaging and the right imaging in the horizontal direction, and the displacement difference is parallax.

[0029] Figure 3 is a schematic diagram of two images with inappropriate parallax, such as Figure 3 The parallax shown is relatively large, and the three-dimensional effect of the synthesized image is poor. For example, there is an obvious ghosting problem in the middle area of ​​the synthesized image, and there is a perspective distortion problem in the edge area.

[0030] Figure 4 is a schematic diagram of two images with appropriate parallax, such as Figure 4 The parallax shown can satisfy the parallax of left and right eye imaging, and the synthesized image has a better three-dimensional effect.

[0031] The embodiments of the present application can be run in devices with image acquisition functions, such as mobile phones, tablet computers, cameras, monitoring, vehicle-mounted imaging equipment, medical imaging equipment, etc.

[0032] In this embodiment, a three-dimensional imaging method is provided.

[0033] Figure 5 is a schematic flow chart of a three-dimensional imaging method according to an embodiment of the present invention, Figure 6 It is a schematic diagram of converting two images with large parallax into two images with appropriate parallax according to an embodiment of the present invention.

[0034] like Figure 5 As shown, the method comprises the following steps:

[0035] Step S501, obtaining a first image of a first camera and a second image of a second camera;

[0036] In some exemplary embodiments, camera A and camera B of the terminal are started, so that the terminal obtains two images of the photographed object through camera A and camera B.

[0037] Step S502, when the parallax between the first image and the second image is greater than a preset threshold, controlling the first image and / or the second image to move so that an overlapping area of ​​the first image and an overlapping area of ​​the second image meet a preset requirement;

[0038] When the parallax is greater than a preset threshold, controlling the first imaging to move or the second imaging to move, or the first imaging and the second imaging to move, so that the overlapping area of ​​the two imagings meets the preset requirements, and performing a padding operation on the non-overlapping areas of the two imagings to form two new imagings respectively;

[0039] In some exemplary embodiments, Figure 6 As shown, when the parallax is greater than a preset threshold, the first imaging movement or the second imaging movement, or the first imaging and the second imaging movement, is controlled. The purpose of the movement is to make the overlapping area of ​​the two images meet the preset requirements, that is, the objects in the two images are relatively stable in the overlapping area, and there are enough feature pixels that meet the parallax requirements in the overlapping area. Through the padding operation, it is convenient to include more feature pixels in the new imaging to improve the imaging effect. And after the feature pixels in the non-overlapping area after padding are moved, they meet the parallax requirements, which is convenient to solve the problem of perspective distortion in the edge area of ​​the synthetic image.

[0040] Step S503: synthesize a three-dimensional image based on the third image and the fourth image.

[0041] Through the above steps, the problem of poor 3D imaging caused by parallax changes in 3D pictures taken by dual cameras is solved. By detecting and adjusting the parallax, the overlapping area between the two images meets the preset requirements, and the non-overlapping area is filled, and finally a 3D image with good stereoscopic effect is synthesized. This can improve the realism and stereoscopic effect of the image, allowing users to observe and experience more realistic shooting objects through the generated 3D image.

[0042] In one embodiment, the filling operation may include: stitching the overlapping area of ​​the first image and the non-overlapping area of ​​the second image into a third image, and stitching the non-overlapping area of ​​the first image and the overlapping area of ​​the second image into a fourth image.

[0043] By adopting the above method, all characteristic pixels in the synthesized three-dimensional image are formed by movement, so as to make as many characteristic pixels as possible meet the parallax requirement, thereby making the imaging effect of the synthesized three-dimensional image better.

[0044] Figure 7is a flow chart of detecting the parallax between two images according to an embodiment of the present invention.

[0045] In one embodiment, after the step of acquiring the first image of the first camera and the second image of the second camera, the method further includes: detecting the parallax between the first image and the second image.

[0046] In one embodiment, Figure 7 As shown, detecting the parallax between the first image and the second image includes the following steps:

[0047] Step S701, calibrating the first image and the second image so that characteristic pixels of the first image and the second image are aligned;

[0048] In some exemplary embodiments, taking a camera as an example, the two images are corrected by the calibration result of the camera. The calibration result of the camera refers to the result describing the internal and external parameters of the camera obtained through a series of calibration processes. Camera calibration is the process of determining the internal parameters (such as focal length, principal point position) and external parameters (such as the position and orientation of the camera) of the camera, aiming to correspond the pixel coordinates in the image to the physical coordinates in the real world.

[0049] Camera calibration usually requires taking multiple images using a known 3D object or a special calibration plate. In these images, feature points are extracted and mapped to known physical coordinates. By analyzing and calculating these images and feature points, the camera's intrinsic and extrinsic parameters can be estimated.

[0050] The internal parameters of the camera may include focal length, principal point position, distortion parameters, etc. These parameters describe the optical characteristics and imaging properties of the camera, and are very important for geometric correction and 3D reconstruction of the image.

[0051] The camera's external parameters can include the camera's position and orientation. The external parameters describe the camera's position and orientation in three-dimensional space and can be used to map pixel coordinates in an image to a world coordinate system.

[0052] The camera calibration results can be presented in many forms, including camera matrix, distortion coefficient, rotation matrix, translation vector, etc. These results provide the camera's geometric information and imaging model, which can be used to correct image distortion, perform 3D reconstruction, and perform camera pose estimation.

[0053] Camera calibration results are very important for computer vision, augmented reality, robot navigation and other fields. Accurate camera calibration results can improve the accuracy and stability of computer vision algorithms and provide accurate spatial references for camera-based applications.

[0054] In an exemplary embodiment, the present application preferably uses a homography matrix transformation method or a binocular correction method to correct the two images.

[0055] The implementation of the homography matrix transformation method is as follows:

[0056] First, obtain the corresponding points between the two images. Select some corresponding pixels in the two images that have the same physical position in the two images. For example, you can select corner points or feature points in the two images as corresponding points. Second, calculate the homography matrix through the corresponding points. Based on the selected corresponding points, use the least squares method or the direct solution of the linear equations to calculate the homography matrix. The homography matrix is ​​a 3*3 matrix that can map the pixel coordinates in one image to the pixel coordinates in another image.

[0057] Finally, the imaging is corrected. Use the calculated homography matrix to correct one of the imaging. For each pixel, use the homography matrix to map it to the other imaging, and assign the pixel value of the corresponding position in the other imaging to the pixel. In this way, the purpose of correcting the two imaging is achieved.

[0058] The binocular correction method is implemented as follows:

[0059] Camera calibration: Calibrate the two cameras to obtain the camera's intrinsic and extrinsic parameter matrices.

[0060] Construct the basic matrix: Use the camera's intrinsic and extrinsic matrices to calculate the basic matrix between the two cameras.

[0061] Construct correction mapping: According to the basic matrix, the correction mapping matrix of the left and right cameras is calculated to correct the images of the left and right cameras to the same plane.

[0062] Image correction: Use the correction mapping matrix to correct the images of the left and right cameras so that the two images are located in the same plane and parallel to each other.

[0063] Pixel coordinate alignment: Based on the rectified image, align the pixel coordinates of the same point in the two images to the same row.

[0064] Through the binocular correction method, the original images captured by the two cameras can be corrected so that the corrected images are located in the same plane and parallel to each other, and the pixel coordinates of the same point in the two images are aligned to the same row.

[0065] Step S702, matching characteristic pixel points of the first image and the second image;

[0066] Step S703 , calculating the disparity of characteristic pixels of the first image and the second image, and generating a disparity map to detect the disparity between the first image and the second image.

[0067] In one embodiment, after generating the disparity map, the disparity of all feature pixels in the disparity map may be compared. When the maximum value of the disparity of the feature pixel in the disparity map is less than the maximum preset threshold, the disparity map is converted into a depth map to obtain information and depth of the feature pixel;

[0068] Synthesize 3D images based on depth maps.

[0069] In some exemplary embodiments, when the maximum value of the disparity of the characteristic pixel points in the disparity map is less than the maximum preset threshold, it indicates that the disparity between the two images is less than or equal to the preset threshold, and at this time, there is no need to move the two images, and the disparity map generated by the two images can be converted into a depth map, and a three-dimensional image can be synthesized based on the depth map. Therefore, the purpose of synthesizing the two new images into a three-dimensional image can be achieved.

[0070] Figure 8 It is a schematic diagram of the position of feature pixels, the position of feature pixels in two images, and the positions of two cameras.

[0071] like Figure 8 As shown, taking the characteristic pixel point P in the photographed object as an example, under the imaging effect of cameras C1 and C2, the imaging points are P1 and P2 respectively. Detect the main axis direction and imaging plane of the camera photography. For example, the two upward arrow lines in the figure represent the main axis direction of the camera photography, and the two horizontal lines along the horizontal direction represent the imaging plane, and the two imaging planes have the same length in the horizontal direction. Taking the left edge of the imaging plane as the starting point, the horizontal coordinates of P1 and P2 are X1 and X2 respectively, and the parallax S is X1-X2. Through the parallax S, using the characteristics of similar triangles, calculate the depth information of point P from the camera. Let the distance between cameras C1 and C2 be B, the distance between P1 and P2 be b, the depth be D, and the focal length be f, then the following relationship can be obtained:

[0072]

[0073] Based on the above formula, the relationship between B, b, X1, and X2 is summarized: b is the distance between P1 and P2, which is equal to B minus the bases of the two triangles on the left and right sides; the base on the left is X1-L / 2; the base on the right is L / 2-X2; L is the horizontal length of the imaging plane; therefore, b=B-(X1-L / 2)-(L / 2-X2)=B-X1+X2=B-(X1-X2)=BS. Substituting b=BS into the above formula, we get:

[0074]

[0075] Finally, we conclude:

[0076]

[0077] In summary, we can conclude that the farther the object distance is, the smaller the parallax is; the closer the object distance is, the larger the parallax is. When the parallax is large, the 3D image will have obvious ghosting in the overlapping area, and perspective distortion will occur in the non-overlapping area. The 3D effect is far less than when the object distance is large and the parallax is small.

[0078] Fig. 9 The flowchart of matching characteristic pixel points of two images according to an embodiment of the present invention.

[0079] Fig.10 It is a position relationship diagram of the first camera, the second camera, the first imaging area, the second imaging area, the first characteristic pixel point, and the second characteristic pixel point.

[0080] In one embodiment, Fig. 9 and Fig.10 As shown, matching the characteristic pixel points of the first image and the second image includes:

[0081] Step S901, detecting a first characteristic pixel point in a first imaging area by a first camera;

[0082] Step S902, identifying the plane formed by the optical center of the first camera, the optical center of the second camera, and the first characteristic pixel point;

[0083] Step S903, identifying a straight line where the plane intersects the first imaging area and the second imaging area of ​​the second camera;

[0084] Step S904: searching on a straight line for a first characteristic pixel point corresponding to a second characteristic pixel point in a second imaging area, so that the characteristic pixel points of the two images match.

[0085] For example: Fig.10 As shown, when the same spatial point P is imaged in the first imaging area and the second imaging area respectively, it is known that the image in the first imaging area on the left is point P1, and point P and the points C1 and C2 at the centers of the two cameras form a plane PC1C2 in the three-dimensional space. The plane PC1C2 intersects the first imaging area on the left and the second imaging area on the right in a straight line (the bold straight line in the figure). Then, the second imaging point P2 of point P in the second imaging area can be found on this straight line, so that the first pixel point and the second pixel point can be matched.

[0086] In one implementation, the parallax being greater than a preset threshold may include:

[0087] The proportion of feature pixel points in the disparity map whose disparity is greater than or equal to the maximum preset threshold is higher than the proportion of feature pixel points whose disparity is less than the maximum preset threshold.

[0088] In some exemplary embodiments, whether the parallax between two images is greater than a preset threshold is reflected by the parallax between characteristic pixels in the images. min , N max 】. There are multiple feature pixels in the disparity map, and the disparity of each feature pixel is compared with the maximum preset threshold within the preset threshold. The disparity of some feature pixels is greater than or equal to the maximum preset threshold N max , and the proportion of this part of characteristic pixels is 80%; the disparity of another part of characteristic pixels is less than the maximum preset threshold N max , and the proportion of this part of feature pixels is 20%, which indicates that there are more feature pixels that do not meet the requirements, and it is determined that the disparity between the two images is greater than the preset threshold. Conversely, when the proportion of feature pixel disparities in the disparity map that are greater than or equal to the maximum preset threshold is lower than the proportion of feature pixel disparities that are less than the maximum preset threshold, it indicates that there are more feature pixel points that meet the requirements, and it is determined that the disparity between the two images is less than or equal to the preset threshold.

[0089] In an exemplary embodiment, when the maximum value of the disparity values ​​of the feature pixels in the disparity map is less than the maximum preset threshold value N max , it can be confirmed that in the disparity map, the disparity of all feature pixels is less than the maximum preset threshold N max , the disparity between the two images is less than or equal to a preset threshold. At this time, the disparity map can be converted into a depth map to obtain the information and depth of the feature pixel points, thereby facilitating the synthesis of a three-dimensional image based on the depth map.

[0090] In one embodiment, when the parallax is greater than a preset threshold, controlling the first image and / or the second image to move so that an overlapping area of ​​the first image and the second image meets a preset requirement includes:

[0091] Get the coordinate value of the feature pixel point (X k ,Y k ) and the disparity value H k ;

[0092] Based on the coordinate value (X k ,Y k ) and the disparity value H k , we get the disparity set C = {H1, H2, H3..., H k};

[0093] Determine the total distance moved to be P units;

[0094] The minimum value of the total distance P is the difference between the minimum value in the disparity set C and the maximum preset threshold, the maximum value of the total distance P is the difference between the maximum value in the disparity set C and the minimum preset threshold, and k is the number of characteristic pixels in the imaging.

[0095] In some exemplary embodiments, it is necessary to make the element H in the disparity set C k Satisfy the constraints: N min <=H k -P<=N max , then -N max <=PH k <=-N min , H k -N max <=P<=H k -N min , therefore, the value range of the total distance P can be obtained as:

[0096] P min =C min -N max <=H k -N max <=P<=H k -N min <=C max -N min =P max .

[0097] In one implementation, the total distance P may also be determined by using a dichotomy method:

[0098] Step S11, set P0 = P min , P1=P max , then Pleft=P0+(P1-P0) / 4; Pright=P1-(P1-P0) / 4;

[0099] Step S22, calculate the set: S1 = {k|N min <=H k -Pleft<=N max}; S2 = {k | N min <=H k -Preight<=N max};

[0100] Step S33, comparing the number of elements in S1 and S2;

[0101] If the number of elements in S1 is greater than the number of elements in S2, let P1'=P0+(P1-P0) / 2, let P1=P1';

[0102] If the number of elements in S1 is less than or equal to the number of elements in S2, let P0'=P0+(P1-P0) / 2, let P0=P0';

[0103] Among them, k is the number of characteristic pixels in the image, N min is the minimum preset threshold, N max is the maximum preset threshold.

[0104] Iterate the above steps S22 and S33 until the set S1 or S2 is equal to the set {1, 2, ..., k}, or the maximum number of iterations is reached, then terminate the iteration process and determine the total distance P outputted at last.

[0105] In one embodiment, the method may further include:

[0106] The first image and the second image are both moved by P / 2 units in a direction approaching each other, or the first image is moved by P units in a direction approaching the second image, or the second image is moved by P units in a direction approaching the first image, so as to form a composite image;

[0107] Calculate the length of the composite image;

[0108] Calculating the length of the area that needs to be supplemented between the first new image and the second new image, wherein the length of the area that needs to be supplemented is equal to the length of the composite image minus the length of the overlapping portion of the first new image and the second new image;

[0109] Based on the length of the area to be supplemented, determining a first newly imaged non-overlapping area in the synthesized image as a first area, and determining a second newly imaged non-overlapping area in the synthesized image as a second area;

[0110] The first area and the second newly imaged area of ​​the overlapping area in the synthetic image are synthesized into a third image, and the second area and the first newly imaged area of ​​the overlapping area in the synthetic image are synthesized into a fourth image;

[0111] A disparity map is generated based on the third imaging and the fourth imaging.

[0112] Reference Figure 6For example, the left image is the image before the two images are moved. In the left image, the left dotted line box represents the first image, and the right solid line box represents the second image. Move the first image and the second image by P / 2 units in the direction close to each other, and then form a composite image after the right side moves. Among them, the composite image can include four parts: the overlapping area of ​​the first new image, the non-overlapping area of ​​the first new image, the overlapping area of ​​the second new image, and the non-overlapping area of ​​the second new image. Calculate the distance between the leftmost dotted line and the rightmost solid line in the composite image, which is the length F of the composite image. Calculate the distance between the leftmost solid line and the rightmost dotted line in the composite image, which is the length G of the overlapping area. Then, the lengths of the non-overlapping area of ​​the first new image (first area) and the non-overlapping area of ​​the second new image (second area) are both M, then M=(FG) / 2. After the above-mentioned moving operation, the first area is synthesized with the area of ​​the second new image in the overlapping area of ​​the composite image into a third image, and the second area is synthesized with the area of ​​the first new image in the overlapping area of ​​the composite image into a fourth image, and a disparity map can be generated based on the third image and the fourth image.

[0113] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and can include a number of instructions to enable a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0114] Fig.11 It is a schematic diagram of a structural block diagram of a three-dimensional imaging device according to an embodiment of the present invention.

[0115] In this embodiment, a three-dimensional imaging device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable. Fig.11 As shown, the device comprises:

[0116] An acquisition module 111 is used to acquire a first image of a first camera and a second image of a second camera;

[0117] The processing module 112 is used to control the first image and / or the second image to move when the parallax between the first image and the second image is greater than a preset threshold value, so that the overlapping area of ​​the first image and the overlapping area of ​​the second image meet the preset requirements, and,

[0118] Performing a padding operation on the non-overlapping area of ​​the first image and the non-overlapping area of ​​the second image to form a third image and a fourth image; and,

[0119] synthesizing a three-dimensional image based on the third imaging and the fourth imaging;

[0120] The output module 113 is used to output the three-dimensional image.

[0121] In one embodiment, the device is further used to: stitch the overlapping area of ​​the first image and the non-overlapping area of ​​the second image into a third image, and stitch the non-overlapping area of ​​the first image and the overlapping area of ​​the second image into a fourth image.

[0122] In one embodiment, the device is also used to:

[0123] Correcting the first image and the second image so that characteristic pixel points of the first image and the second image are aligned;

[0124] Matching characteristic pixel points of the first image and the second image;

[0125] Calculate the disparity of the characteristic pixel points of the first image and the second image, and generate a disparity map to detect the disparity between the first image and the second image. In one embodiment, the device is also used for:

[0126] When the maximum value of the disparity of the feature pixel in the disparity map is less than a maximum preset threshold, the disparity map is converted into a depth map to obtain information and depth of the feature pixel;

[0127] Synthesize 3D images based on depth maps.

[0128] In one embodiment, the device is also used to:

[0129] Detecting a first characteristic pixel point within a first imaging area by a first camera;

[0130] Identify the plane formed by the optical center of the first camera, the optical center of the second camera, and the first characteristic pixel point;

[0131] A straight line where the recognition plane intersects the first imaging area and the second imaging area of ​​the second camera;

[0132] The first characteristic pixel point is searched on the straight line for a second characteristic pixel point in the second imaging area, so that the characteristic pixel points of the first imaging and the second imaging match.

[0133] In one embodiment, the device is further configured to: the ratio of feature pixel points in the disparity map having disparities greater than or equal to a maximum preset threshold is higher than the ratio of feature pixel points having disparities less than the maximum preset threshold.

[0134] In one embodiment, the device is also used to:

[0135] Get the coordinate value of the feature pixel point (X k ,Y k ) and the disparity value H k ;

[0136] Based on the coordinate value (X k ,Y k ) and the disparity value H k , we get the disparity set C = {H1, H2, H3..., H k};

[0137] Determine that the total distance of movement is P units; wherein the minimum value of the total distance P is the difference between the minimum value in the disparity set C and the maximum preset threshold, the maximum value of the total distance P is the difference between the maximum value in the disparity set C and the minimum preset threshold, and k is the number of characteristic pixels in the imaging.

[0138] In one embodiment, the device is further used to: determine the total distance P by using a dichotomy method:

[0139] Set P0 = P min , P1=P max , then Pleft=P0+(P1-P0) / 4; Pright=P1-(P1-P0) / 4;

[0140] Calculation set: S1 = {k | N min <=H k -Pleft<=N max}; S2 = {k | N min <=H k -Preight<=N max};

[0141] Compare the number of elements in S1 and S2;

[0142] If the number of elements in S1 is greater than the number of elements in S2, let P1'=P0+(P1-P0) / 2, let P1=P1';

[0143] If the number of elements in S1 is less than or equal to the number of elements in S2, let P0'=P0+(P1-P0) / 2, let P0=P0';

[0144] Among them, k is the number of characteristic pixels in the image, Nmin is the minimum preset threshold, N max is the maximum preset threshold.

[0145] In one embodiment, the device is also used to:

[0146] The first image and the second image are both moved by P / 2 units in a direction approaching each other, or the first image is moved by P units in a direction approaching the second image, or the second image is moved by P units in a direction approaching the first image, so as to form a composite image;

[0147] Calculate the length of the composite image;

[0148] Calculating the length of the area that needs to be supplemented between the first new image and the second new image, wherein the length of the area that needs to be supplemented is equal to the length of the composite image minus the length of the overlapping portion of the first new image and the second new image;

[0149] Based on the length of the area to be supplemented, determining a first newly imaged non-overlapping area in the synthesized image as a first area, and determining a second newly imaged non-overlapping area in the synthesized image as a second area;

[0150] The first area and the second newly imaged area of ​​the overlapping area in the synthetic image are synthesized into a third image, and the second area and the first newly imaged area of ​​the overlapping area in the synthetic image are synthesized into a fourth image;

[0151] A disparity map is generated based on the third imaging and the fourth imaging.

[0152] It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0153] An embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.

[0154] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0155] An embodiment of the present invention further provides an electronic device, which may include a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0156] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0157] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.

[0158] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0159] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A three-dimensional imaging method, characterized in that: include: Acquire a first image from the first camera and a second image from the second camera; When the parallax between the first image and the second image is greater than a preset threshold, controlling the first image and / or the second image to move so that an overlapping area of ​​the first image and an overlapping area of ​​the second image meet a preset requirement; Performing a padding operation on a non-overlapping area of ​​the first image and a non-overlapping area of ​​the second image to form a third image and a fourth image; A three-dimensional image is synthesized based on the third imaging and the fourth imaging.

2. The method according to claim 1, characterized in that The padding operation includes: The overlapping area of ​​the first imaging and the non-overlapping area of ​​the second imaging are spliced ​​into a third imaging, and the non-overlapping area of ​​the first imaging and the overlapping area of ​​the second imaging are spliced ​​into a fourth imaging.

3. The method according to claim 1, characterized in that Also includes: Correcting the first image and the second image so that characteristic pixel points of the first image and the second image are aligned; Matching characteristic pixel points of the first image and the second image; The disparity of characteristic pixel points of the first image and the second image is calculated to generate a disparity map to detect the disparity between the first image and the second image.

4. The method according to claim 3, characterized in that Also includes: When the maximum value of the disparity of the feature pixel point in the disparity map is less than a maximum preset threshold, converting the disparity map into a depth map to obtain information and depth of the feature pixel point; The three-dimensional image is synthesized based on the depth image.

5. The method according to claim 3, characterized in that: Matching characteristic pixel points of the first image and the second image includes: Detecting a first characteristic pixel point within a first imaging area by the first camera; Identify the plane formed by the optical center of the first camera, the optical center of the second camera, and the first characteristic pixel point; Identify a straight line where the plane intersects the first imaging area and the second imaging area of ​​the second camera; A search is performed on the straight line for a second characteristic pixel point corresponding to a first characteristic pixel point in a second imaging area, so that the characteristic pixel points of the first imaging and the second imaging match.

6. The method according to claim 3, characterized in that Parallax greater than a preset threshold includes: The proportion of feature pixel points in the disparity map having disparities greater than or equal to a maximum preset threshold is higher than the proportion of feature pixel points having disparities less than the maximum preset threshold.

7. The method according to claim 6, characterized in that When the parallax is greater than a preset threshold, controlling the first imaging and / or the second imaging to move so that an overlapping area of ​​the first imaging and the second imaging meets a preset requirement, including: Get the coordinate value (X k ,Y k ) and the disparity value H k ; Based on the coordinate value (X k ,Y k ) and the disparity value H k , we get the disparity set C = {H1, H2, H3..., H k }; Determine that the total distance of movement is P units; wherein the minimum value of the total distance P is the difference between the minimum value in the disparity set C and the maximum preset threshold, the maximum value of the total distance P is the difference between the maximum value in the disparity set C and the minimum preset threshold, and k is the number of characteristic pixels in the imaging.

8. The method according to claim 7, characterized in that Also includes: Moving the first image and the second image by P / 2 units in a direction approaching each other, or moving the first image by P units in a direction approaching the second image, or moving the second image by P units in a direction approaching the first image, so as to form a composite image; Calculating the length of the composite image; Calculating the length of the area that needs to be supplemented between the first new image and the second new image, wherein the length of the area that needs to be supplemented is equal to the length of the composite image minus the length of the overlapping portion of the first new image and the second new image; Based on the length of the area to be supplemented, determining a first newly imaged non-overlapping area in the synthesized image as a first area, and determining a second newly imaged non-overlapping area in the synthesized image as a second area; Combining the first area with the second newly imaged area of ​​the overlapping area in the composite image into a third image, and combining the second area with the first newly imaged area of ​​the overlapping area in the composite image into a fourth image; A disparity map is generated based on the third imaging and the fourth imaging.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program implements the steps of the method described in any one of claims 1 to 8 when executed by a processor.

10. An electronic device comprising one or more memories, one or more processors, and a computer program stored in the memories and executable on the processors, characterized in that: When the processor executes the computer program, the steps of the method described in any one of claims 1 to 8 are implemented.

Images