Three-dimensional imaging method, storage medium, and electronic apparatus

Abstract

The embodiments of the present disclosure provide a three-dimensional imaging method, a storage medium, and an electronic apparatus. The method comprises: acquiring a first image from a first camera and a second image from a second camera; when a disparity between the first image and the second image is greater than a preset threshold value, controlling the first image and / or the second image to move, such that an overlapping region of the first image and an overlapping region of the second image meet a preset requirement; performing an inpainting operation on a non-overlapping region of the first image and a non-overlapping region of the second image to form a third image and a fourth image; and synthesizing the third image and the fourth image to obtain a three-dimensional image. By means of the present disclosure, the problem of a poor imaging effect occurring during short-distance photographing is solved, thereby achieving the effect of improving the imaging quality.

Description

Three-dimensional imaging method, storage medium and electronic device

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This disclosure is based on Chinese patent application CN202311096774.4, filed on August 28, 2023, entitled “Three-dimensional imaging method, storage medium and electronic device”, and claims the priority of the patent application, and all the contents disclosed therein are incorporated into this disclosure by reference. Technical Field

[0003] The embodiments of the present disclosure relate to the technical field of image processing, and in particular to a three-dimensional imaging method, a storage medium, and an electronic device. Background Art

[0004] Traditional photography techniques can only capture two-dimensional images of objects, failing to convey depth and a sense of three-dimensionality. To address this issue, researchers have begun researching the use of dual cameras to simulate the human eye's stereoscopic vision. The principle behind dual-camera 3D images is to simultaneously capture two images with two cameras. These two images must have a parallax difference to form the left and right eye images. These images are then combined using a 3D algorithm into a single image, which can then be viewed on a naked-eye 3D screen. However, as the distance between the camera and the object changes, the parallax between the two images also changes in real time, which can easily lead to poor 3D imaging.

[0005] Summary of the Invention

[0006] The embodiments of the present disclosure provide a three-dimensional imaging method, a storage medium, and an electronic device to at least solve the problem of poor imaging effect when taking photos at close range in the related art.

[0007] According to one embodiment of the present disclosure, a three-dimensional imaging method is provided, including:

[0008] Acquire a first image from the first camera and a second image from the second camera;

[0009] When the parallax between the first image and the second image is greater than a preset threshold, controlling the first image and / or the second image to move so that an overlapping area of ​​the first image and an overlapping area of ​​the second image meet a preset requirement;

[0010] Performing a padding operation on a non-overlapping area of ​​the first image and a non-overlapping area of ​​the second image to form a third image and a fourth image;

[0011] A three-dimensional image is synthesized based on the third imaging and the fourth imaging.

[0012] According to another embodiment of the present disclosure, a computer-readable storage medium is provided, in which a computer program is stored. The computer program is configured to execute the steps of any one of the above method embodiments when running.

[0013] According to another embodiment of the present disclosure, an electronic device is provided, which may include a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to perform the steps in any one of the above method embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] FIG1 is a schematic diagram of a hardware structure block diagram of a mobile terminal for a three-dimensional imaging method according to an embodiment of the present disclosure;

[0015] FIG2 is a schematic diagram of two images formed by two cameras simultaneously photographing the same object;

[0016] FIG3 is a schematic diagram of two images with appropriate parallax;

[0017] FIG4 is a schematic diagram of two images with inappropriate parallax;

[0018] FIG5 is a schematic flow chart of a three-dimensional imaging method according to an embodiment of the present disclosure;

[0019] FIG6 is a schematic diagram of converting two images with large parallax into two images with appropriate parallax according to an embodiment of the present disclosure;

[0020] FIG7 is a schematic diagram of a process for detecting a parallax between two images according to an embodiment of the present disclosure;

[0021] FIG8 is a schematic diagram of the position of a characteristic pixel point, the position of the characteristic pixel point in two images, and the position of two cameras;

[0022] FIG9 is a schematic diagram of a process for matching characteristic pixel points of two images according to an embodiment of the present disclosure;

[0023] FIG10 is a schematic diagram showing the positional relationship between the first camera, the second camera, the first imaging area, the second imaging area, the first characteristic pixel point, and the second characteristic pixel point;

[0024] FIG11 is a schematic structural block diagram of a three-dimensional imaging device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0025] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings and in conjunction with embodiments.

[0026] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0027] The method embodiments provided in the embodiments of the present disclosure can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, FIG1 is a hardware structure block diagram of a mobile terminal of a three-dimensional imaging method in an embodiment of the present disclosure. As shown in FIG1 , the mobile terminal may include one or more (only one is shown in FIG1 ) processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 and an input and output device 108 for communication functions. It will be understood by those skilled in the art that the structure shown in FIG1 is only for illustration and does not limit the structure of the mobile terminal. For example, the mobile terminal may also include more or fewer components than those shown in FIG1 , or have a configuration different from that shown in FIG1 .

[0028] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as a computer program corresponding to a three-dimensional imaging method in an embodiment of the present disclosure. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, implementing the above-mentioned method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely located relative to the processor 102, and these remote memories may be connected to the mobile terminal via a network. Examples of the above-mentioned network may include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0029] The transmission device 106 is used to receive or send data via a network. A specific example of the aforementioned network may include a wireless network provided by the mobile terminal's communications provider. In one embodiment, the transmission device 106 may include a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0030] Figure 2 is a schematic diagram of two images formed by two cameras simultaneously shooting the same object. As shown in Figure 2, the dotted box on the left is the image of one camera, and the dotted box on the right is the image of the other camera. There will be a displacement difference in the horizontal direction between the left image and the right image, and the displacement difference is parallax.

[0031] Figure 3 is a schematic diagram of two images with inappropriate parallax. As shown in Figure 3, the parallax is relatively large, and the three-dimensional effect of the synthesized image is poor. For example, the center area of ​​the synthesized image has obvious ghosting problems, and the edge areas have perspective distortion problems.

[0032] FIG. 4 is a schematic diagram of two images with appropriate parallax. The parallax shown in FIG. 4 can satisfy the parallax of left and right eye imaging, and the synthesized image has a better three-dimensional effect.

[0033] The embodiments of the present disclosure can be run on devices with image acquisition functions, such as mobile phones, tablet computers, cameras, surveillance, vehicle-mounted imaging equipment, medical imaging equipment, etc.

[0034] In this embodiment, a three-dimensional imaging method is provided.

[0035] FIG5 is a flow chart of a three-dimensional imaging method according to an embodiment of the present disclosure, and FIG6 is a schematic diagram of converting two images with large parallax into two images with appropriate parallax according to an embodiment of the present disclosure.

[0036] As shown in FIG5 , the method includes the following steps:

[0037] Step S501, obtaining a first image from a first camera and a second image from a second camera;

[0038] In some exemplary embodiments, camera A and camera B of the terminal are activated so that the terminal obtains two images of the photographed object through camera A and camera B.

[0039] Step S502: When the parallax between the first image and the second image is greater than a preset threshold, controlling the first image and / or the second image to move so that an overlapping area of ​​the first image and an overlapping area of ​​the second image meet a preset requirement;

[0040] When the parallax is greater than a preset threshold, controlling the first imaging movement or the second imaging movement, or the first imaging and the second imaging movement, so that the overlapping area of ​​the two images meets a preset requirement, and performing a padding operation on the non-overlapping areas of the two images to form two new images respectively;

[0041] In some exemplary embodiments, as shown in FIG6 , when the parallax exceeds a preset threshold, the first image or the second image, or both, is controlled to move. The purpose of the movement is to ensure that the overlapping region of the two images meets preset requirements. Specifically, objects in the two images are relatively stable within the overlapping region, and a sufficient number of feature pixels meeting the parallax requirements are located within the overlapping region. Padding facilitates including more feature pixels within the new image, thereby improving the imaging effect. Furthermore, after the feature pixels in the padded non-overlapping region are moved, they meet the parallax requirements, thereby resolving the issue of perspective distortion at the edges of the composite image.

[0042] Step S503: synthesize a three-dimensional image based on the third image and the fourth image.

[0043] The above steps address the issue of poor 3D imaging caused by parallax variations in dual-camera 3D images. By detecting and adjusting parallax, the overlapping area between the two images meets preset requirements, while non-overlapping areas are padded, ultimately creating a composite 3D image with a strong sense of three-dimensionality. This enhances the image's realism and three-dimensionality, allowing users to observe and experience the captured subject more realistically through the generated 3D image.

[0044] In one embodiment, the filling operation may include: stitching the overlapping area of ​​the first image and the non-overlapping area of ​​the second image into a third image, and stitching the non-overlapping area of ​​the first image and the overlapping area of ​​the second image into a fourth image.

[0045] By adopting the above method, all characteristic pixels in the synthesized three-dimensional image are formed by movement, so that as many characteristic pixels as possible meet the parallax requirement, thereby making the imaging effect of the synthesized three-dimensional image better.

[0046] FIG. 7 is a flowchart of detecting a disparity between two images according to an embodiment of the present disclosure.

[0047] In one embodiment, after the step of acquiring the first image of the first camera and the second image of the second camera, the method further includes: detecting the parallax between the first image and the second image.

[0048] In one embodiment, as shown in FIG7 , detecting the parallax between the first image and the second image includes the following steps:

[0049] Step S701, correcting the first image and the second image so that characteristic pixels of the first image and the second image are aligned;

[0050] In some exemplary embodiments, using a camera as an example, the two images are corrected using camera calibration results. Camera calibration results refer to the results describing the camera's internal and external parameters, obtained through a series of calibration processes. Camera calibration is the process of determining the camera's internal parameters (e.g., focal length, principal point location) and external parameters (e.g., camera position and orientation), aiming to correspond pixel coordinates in an image to physical coordinates in the real world.

[0051] Camera calibration typically involves capturing multiple images using a known 3D object or a special calibration target. Feature points are extracted from these images and mapped to known physical coordinates. By analyzing and calculating these images and feature points, the camera's intrinsic and extrinsic parameters can be estimated.

[0052] The intrinsic parameters of a camera may include focal length, principal point position, distortion parameters, etc. These parameters describe the optical characteristics and imaging properties of the camera and are very important for geometric correction and 3D reconstruction of images.

[0053] The camera's extrinsic parameters can include the camera's position and orientation. Extrinsic parameters describe the camera's position and orientation in three-dimensional space and can be used to map pixel coordinates in an image to a world coordinate system.

[0054] Camera calibration results can be presented in various forms, including camera matrices, distortion coefficients, rotation matrices, and translation vectors. These results provide the camera's geometric information and imaging model, which can be used for applications such as image distortion correction, 3D reconstruction, and camera pose estimation.

[0055] Camera calibration results are very important for fields such as computer vision, augmented reality, and robot navigation. Accurate camera calibration results can improve the accuracy and stability of computer vision algorithms and provide accurate spatial references for camera-based applications.

[0056] In an exemplary embodiment, the present disclosure preferably uses a homography matrix transformation method or a binocular correction method to correct the two images.

[0057] The implementation of the homography matrix transformation method is as follows:

[0058] First, obtain the corresponding points between the two images. Select corresponding pixels in the two images that have the same physical location in both images. For example, you can select corner points or feature points in the two images as corresponding points. Next, calculate the homography matrix using the corresponding points. Based on the selected corresponding points, use the least squares method or a direct solution of a linear system of equations to calculate the homography matrix. The homography matrix is ​​a 3*3 matrix that maps pixel coordinates in one image to pixel coordinates in the other image.

[0059] Finally, the images are rectified. Using the calculated homography matrix, one of the images is rectified. For each pixel, the homography matrix is ​​used to map it to the other image, and the pixel value at the corresponding position in the other image is assigned to that pixel. This achieves the goal of aligning the two images.

[0060] The binocular correction method is implemented as follows:

[0061] Camera calibration: Calibrate the two cameras to obtain the camera's intrinsic and extrinsic parameter matrices.

[0062] Construct the basic matrix: Use the camera's intrinsic and extrinsic matrix to calculate the basic matrix between the two cameras.

[0063] Constructing the correction mapping: Based on the basic matrix, the correction mapping matrix of the left and right cameras is calculated to correct the images of the left and right cameras to the same plane.

[0064] Image correction: Use the correction mapping matrix to correct the images of the left and right cameras so that the two images are in the same plane and parallel to each other.

[0065] Pixel coordinate alignment: Based on the rectified image, the pixel coordinates of the same point in the two images are aligned to the same row.

[0066] Through the binocular correction method, the original images captured by the two cameras can be corrected so that the corrected images are located in the same plane and parallel to each other, and the pixel coordinates of the same point in the two images are aligned to the same row.

[0067] Step S702, matching the characteristic pixels of the first image and the second image;

[0068] Step S703 : calculating the disparity between the characteristic pixels of the first image and the second image, and generating a disparity map to detect the disparity between the first image and the second image.

[0069] In one embodiment, after generating the disparity map, the disparity of all feature pixels in the disparity map may be compared. When the maximum disparity of the feature pixels in the disparity map is less than a maximum preset threshold, the disparity map is converted into a depth map to obtain information and depth of the feature pixels.

[0070] Synthesize 3D images based on depth maps.

[0071] In some exemplary embodiments, when the maximum disparity of a feature pixel within a disparity map is less than a maximum preset threshold, indicating that the disparity between the two images is less than or equal to the preset threshold, there is no need to move the two images. The disparity map generated by the two images can be converted into a depth map, and a 3D image can be synthesized based on the depth map. Thus, the purpose of synthesizing the two new images into a 3D image can be achieved.

[0072] FIG8 is a schematic diagram of the positions of characteristic pixels, the positions of characteristic pixels in two images, and the positions of two cameras.

[0073] As shown in Figure 8, taking the characteristic pixel point P in the photographed object as an example, under the imaging effect of cameras C1 and C2, the imaging points are P1 and P2 respectively. Detect the main axis direction of the camera photography and the imaging plane. For example, the two upward arrow lines in the figure represent the main axis direction of the camera photography, and the two horizontal lines along the horizontal direction represent the imaging plane, and the two imaging planes have the same length in the horizontal direction. Taking the left edge of the imaging plane as the starting point, the horizontal coordinates of P1 and P2 are X1 and X2 respectively, and the parallax S is X1-X2. Through the parallax S, using the characteristics of similar triangles, calculate the depth information of point P from the camera. Let the distance between cameras C1 and C2 be B, the distance between P1 and P2 be b, the depth be D, and the focal length be f, then the following relationship can be obtained:

[0074] Based on the above formula, the relationship between B, b, X1, and X2 can be summarized as follows: b is the distance between P1 and P2, which is equal to B minus the bases of the two triangles on the left and right sides; the base on the left is X1-L / 2; the base on the right is L / 2-X2; L is the horizontal length of the imaging plane; therefore, b = B-(X1-L / 2)-(L / 2-X2) = B-X1+X2 = B-(X1-X2) = BS. Substituting b = BS into the above formula, we obtain:

[0075] Finally, we can conclude that:

[0076] In summary, the greater the object distance, the smaller the parallax; the closer the object distance, the larger the parallax. When the parallax is large, the 3D image will have obvious ghosting in the overlapping areas and perspective distortion in the non-overlapping areas. The 3D effect is far less effective than when the object distance is large and the parallax is small.

[0077] FIG9 is a flowchart of matching feature pixels of two images according to an embodiment of the present disclosure.

[0078] FIG10 is a positional relationship diagram of the first camera, the second camera, the first imaging area, the second imaging area, the first characteristic pixel point, and the second characteristic pixel point.

[0079] In one embodiment, as shown in FIG9 and FIG10 , matching the characteristic pixels of the first image and the second image includes:

[0080] Step S901, detecting a first characteristic pixel point in a first imaging area by a first camera;

[0081] Step S902, identifying a plane formed by the optical center of the first camera, the optical center of the second camera, and the first characteristic pixel point;

[0082] Step S903, identifying a straight line where the plane intersects the first imaging area and the second imaging area of ​​the second camera;

[0083] Step S904 : searching on a straight line for a first characteristic pixel point corresponding to a second characteristic pixel point in a second imaging area, so as to match the characteristic pixel points of the two images.

[0084] For example, as shown in FIG10 , when the same spatial point P is imaged in the first imaging area and the second imaging area respectively, it is known that the image in the first imaging area on the left is point P1, and point P and the points C1 and C2 at the centers of the two cameras form a plane PC1C2 in three-dimensional space. The plane PC1C2 intersects the first imaging area on the left and the second imaging area on the right in a straight line (the bold straight line in the figure). Then, the second imaging point P2 of point P in the second imaging area can be found on this straight line, thereby matching the first pixel point with the second pixel point.

[0085] In one embodiment, the disparity being greater than a preset threshold may include:

[0086] The ratio of feature pixel points in the disparity map having disparities greater than or equal to a maximum preset threshold is higher than the ratio of feature pixel points having disparities less than the maximum preset threshold.

[0087] In some exemplary embodiments, whether the parallax between two images is greater than a preset threshold is reflected by the parallax between the characteristic pixels in the images. min , N max There are multiple feature pixels in the disparity map, and the disparity of each feature pixel is compared with the maximum preset threshold within the preset threshold. The disparity of some feature pixels is greater than or equal to the maximum preset threshold N. max , and the proportion of this part of feature pixels is 80%; the disparity of the other part of feature pixels is less than the maximum preset threshold N max, and the proportion of these feature pixels is 20%, indicating that there are relatively many feature pixels that do not meet the requirements, and the disparity between the two images is determined to be greater than the preset threshold. Conversely, when the proportion of feature pixels in the disparity map with disparities greater than or equal to the maximum preset threshold is lower than the proportion of feature pixels with disparities less than the maximum preset threshold, it indicates that there are relatively many feature pixels that meet the requirements, and the disparity between the two images is determined to be less than or equal to the preset threshold.

[0088] In an exemplary embodiment, when the maximum value of the disparity values ​​of the feature pixels in the disparity map is less than the maximum preset threshold N max , we can confirm that in the disparity map, the disparity of all feature pixels is less than the maximum preset threshold N max , the disparity between the two images is less than or equal to a preset threshold. At this time, the disparity map can be converted into a depth map to obtain the information and depth of the feature pixel points, thereby facilitating the synthesis of a three-dimensional image based on the depth map.

[0089] In one embodiment, when the parallax is greater than a preset threshold, controlling the first image and / or the second image to move so that an overlapping area of ​​the first image and the second image meets a preset requirement includes:

[0090] Get the coordinate value of the feature pixel point (X k ,Y k ) and disparity value H k ;

[0091] Based on the coordinate value (X k ,Y k ) and disparity value H k , we get the disparity set C = {H1, H2, H3……, H k};

[0092] Determine the total distance moved to be P units;

[0093] The minimum value of the total distance P is the difference between the minimum value in the disparity set C and the maximum preset threshold, the maximum value of the total distance P is the difference between the maximum value in the disparity set C and the minimum preset threshold, and k is the number of feature pixels in the imaging.

[0094] In some exemplary embodiments, it is necessary to make the element H in the disparity set C k Satisfy the constraints:

[0095] N min <=H k -P<=N max , then -N max <=PH k <=-N min , H k -Nmax <=P<=H k -N min , therefore, the range of the total distance P can be obtained as:

[0096] P min =

[0097] C min -N max <=H k -N max <=P<=H k -N min <=C max -N min =P max .

[0098] In one embodiment, the total distance P may be determined by using a dichotomy method:

[0099] Step S11, set P0=P min , P1=P max , then Pleft=P0+(P1-P0) / 4;Pright=P1-(P1-P0) / 4;

[0100] Step S22, calculate the set: S1 = {k|N min <=H k -Pleft<=N max}; S2 = {k|N min <=H k -Pright<=N max};

[0101] Step S33, comparing the number of elements in S1 and S2;

[0102] If the number of elements in S1 is greater than the number of elements in S2, then let P1'=P0+(P1-P0) / 2, and let P1=P1';

[0103] If the number of elements in S1 is less than or equal to the number of elements in S2, then let P0'=P0+(P1-P0) / 2, and let P0=P0';

[0104] Among them, k is the number of characteristic pixels in the imaging, N min is the minimum preset threshold, N max is the maximum preset threshold.

[0105] Iterate the above steps S22 and S33 until the set S1 or S2 is equal to the set {1, 2, ..., k}, or the maximum number of iterations is reached, then terminate the iterative process and determine the final output total distance P.

[0106] In one embodiment, the method may further include:

[0107] The first image and the second image are both moved in a direction approaching each other by P / 2 units, or the first image is moved in a direction approaching the second image by P units, or the second image is moved in a direction approaching the first image by P units, so as to form a composite image;

[0108] Calculate the length of the composite image;

[0109] Calculating the length of the area that needs to be supplemented between the first new image and the second new image, wherein the length of the area that needs to be supplemented is equal to the length of the composite image minus the length of the overlapping portion of the first new image and the second new image;

[0110] Based on the length of the area to be supplemented, determining a first newly imaged non-overlapping area in the synthesized image as a first area, and determining a second newly imaged non-overlapping area in the synthesized image as a second area;

[0111] synthesizing the first area and the second newly imaged area of ​​the overlapping area in the synthesized image into a third image, and synthesizing the second area and the first newly imaged area of ​​the overlapping area in the synthesized image into a fourth image;

[0112] A disparity map is generated based on the third imaging and the fourth imaging.

[0113] Referring to Figure 6 , for example, the left image is the image before the two images are moved. In the left image, the dashed box on the left represents the first image, and the solid box on the right represents the second image. The first and second images are both moved closer together by P / 2 units, forming a composite image after the right shift. The composite image can include four parts: the overlapping area of ​​the first new image, the non-overlapping area of ​​the first new image, the overlapping area of ​​the second new image, and the non-overlapping area of ​​the second new image. The distance between the leftmost dashed line and the rightmost solid line in the composite image is calculated; this distance is the length F of the composite image. The distance between the leftmost solid line and the rightmost dashed line in the composite image is calculated; this distance is the length G of the overlapping area. Therefore, the lengths of the non-overlapping area of ​​the first new image (the first area) and the non-overlapping area of ​​the second new image (the second area) are both M, where M = (FG) / 2. After the above movement operations, the first area is combined with the overlapping area of ​​the second new image in the composite image to form a third image, and the second area is combined with the overlapping area of ​​the first new image in the composite image to form a fourth image. A disparity map is generated based on the third and fourth images.

[0114] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present disclosure is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and can include a number of instructions to enable a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present disclosure.

[0115] FIG11 is a schematic structural block diagram of a three-dimensional imaging device according to an embodiment of the present disclosure.

[0116] In this embodiment, a three-dimensional imaging device is also provided, which is used to implement the above-mentioned embodiments and preferred embodiments. The details that have been described will not be repeated here. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware, is also possible and contemplated. As shown in Figure 11, the device includes:

[0117] An acquisition module 111 is configured to acquire a first image from a first camera and a second image from a second camera;

[0118] The processing module 112 is configured to control the movement of the first image and / or the second image so that the overlapping area of ​​the first image and the overlapping area of ​​the second image meet a preset requirement when the parallax between the first image and the second image is greater than a preset threshold, and

[0119] performing a padding operation on the non-overlapping area of ​​the first image and the non-overlapping area of ​​the second image to form a third image and a fourth image; and

[0120] synthesizing a three-dimensional image based on the third image and the fourth image;

[0121] The output module 113 is configured to output the three-dimensional image.

[0122] In one embodiment, the device is further configured to: stitch the overlapping area of ​​the first image and the non-overlapping area of ​​the second image into a third image, and stitch the non-overlapping area of ​​the first image and the overlapping area of ​​the second image into a fourth image.

[0123] In one embodiment, the device is further used to:

[0124] Correcting the first image and the second image so that characteristic pixels of the first image and the second image are aligned;

[0125] Matching feature pixels of the first image and the second image;

[0126] Calculate the disparity of the characteristic pixels of the first image and the second image, and generate a disparity map to detect the disparity between the first image and the second image. In one embodiment, the apparatus is further configured to:

[0127] When the maximum value of the disparity of the feature pixel in the disparity map is less than a maximum preset threshold, the disparity map is converted into a depth map to obtain the information and depth of the feature pixel;

[0128] Synthesize 3D images based on depth maps.

[0129] In one embodiment, the device is further used to:

[0130] Detecting a first characteristic pixel point within a first imaging area of ​​the first camera;

[0131] Identify the plane formed by the optical center of the first camera, the optical center of the second camera, and the first characteristic pixel point;

[0132] identifying a straight line where the plane intersects the first imaging area and the second imaging area of ​​the second camera;

[0133] The first characteristic pixel point is searched on the straight line for a second characteristic pixel point in the second imaging area, so that the characteristic pixel points of the first imaging and the second imaging match.

[0134] In one embodiment, the apparatus is further configured to: the ratio of feature pixel points in the disparity map having disparities greater than or equal to a maximum preset threshold is higher than the ratio of feature pixel points having disparities less than the maximum preset threshold.

[0135] In one embodiment, the device is further used to:

[0136] Get the coordinate value of the feature pixel point (X k ,Y k ) and disparity value H k ;

[0137] Based on the coordinate value (X k ,Y k ) and disparity value H k , we get the disparity set C = {H1, H2, H3……, H k};

[0138] The total distance of movement is determined to be P units; wherein the minimum value of the total distance P is the difference between the minimum value in the disparity set C and the maximum preset threshold, the maximum value of the total distance P is the difference between the maximum value in the disparity set C and the minimum preset threshold, and k is the number of characteristic pixels in the imaging.

[0139] In one embodiment, the device is further configured to: determine the total distance P using a dichotomy method:

[0140] Set P0 = P min , P1=P max , then Pleft=P0+(P1-P0) / 4;Pright=P1-(P1-P0) / 4;

[0141] Calculation set: S1 = {k|N min <=H k -Pleft<=N max}; S2 = {k|N min <=H k -Pright<=N max};

[0142] Comparing the number of elements in S1 and S2;

[0143] If the number of elements in S1 is greater than the number of elements in S2, then let P1'=P0+(P1-P0) / 2, and let P1=P1';

[0144] If the number of elements in S1 is less than or equal to the number of elements in S2, then let P0'=P0+(P1-P0) / 2, and let P0=P0';

[0145] Among them, k is the number of characteristic pixels in the imaging, N min is the minimum preset threshold, N max is the maximum preset threshold.

[0146] In one embodiment, the device is further used to:

[0147] The first image and the second image are both moved in a direction approaching each other by P / 2 units, or the first image is moved in a direction approaching the second image by P units, or the second image is moved in a direction approaching the first image by P units, so as to form a composite image;

[0148] Calculate the length of the composite image;

[0149] Calculating the length of the area that needs to be supplemented between the first new image and the second new image, wherein the length of the area that needs to be supplemented is equal to the length of the composite image minus the length of the overlapping portion of the first new image and the second new image;

[0150] Based on the length of the area to be supplemented, determining a first newly imaged non-overlapping area in the synthesized image as a first area, and determining a second newly imaged non-overlapping area in the synthesized image as a second area;

[0151] synthesizing the first area and the second newly imaged area of ​​the overlapping area in the synthesized image into a third image, and synthesizing the second area and the first newly imaged area of ​​the overlapping area in the synthesized image into a fourth image;

[0152] A disparity map is generated based on the third imaging and the fourth imaging.

[0153] It should be noted that the above modules can be implemented through software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0154] An embodiment of the present disclosure further provides a computer-readable storage medium, in which a computer program is stored. The computer program is configured to execute the steps of any one of the above method embodiments when run.

[0155] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0156] An embodiment of the present disclosure further provides an electronic device, which may include a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0157] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0158] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail here.

[0159] Obviously, those skilled in the art should understand that the modules or steps of the present disclosure described above can be implemented using a general-purpose computing device, they can be concentrated on a single computing device, or distributed across a network composed of multiple computing devices, they can be implemented using program code executable by the computing device, and thus, they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be performed in a different order than herein, or they can be fabricated into separate integrated circuit modules, or multiple modules or steps can be fabricated into a single integrated circuit module for implementation. Thus, the present disclosure is not limited to any particular combination of hardware and software.

[0160] The foregoing description is merely a preferred embodiment of the present disclosure and is not intended to limit the present disclosure. Those skilled in the art will readily appreciate that various modifications and variations of the present disclosure are possible. Any modifications, equivalent substitutions, or improvements made within the principles of the present disclosure shall be included within the scope of protection of the present disclosure.

Claims

1. A three-dimensional imaging method, comprising: Acquire a first image from the first camera and a second image from the second camera; When the parallax between the first image and the second image is greater than a preset threshold, controlling the first image and / or the second image to move so that an overlapping area of ​​the first image and an overlapping area of ​​the second image meet a preset requirement; Performing a padding operation on a non-overlapping area of ​​the first image and a non-overlapping area of ​​the second image to form a third image and a fourth image; A three-dimensional image is synthesized based on the third imaging and the fourth imaging.

2. The method according to claim 1, wherein: The padding operation includes: The overlapping area of ​​the first imaging and the non-overlapping area of ​​the second imaging are spliced ​​into a third imaging, and the non-overlapping area of ​​the first imaging and the overlapping area of ​​the second imaging are spliced ​​into a fourth imaging.

3. The method according to claim 1, further comprising: Correcting the first image and the second image so that characteristic pixel points of the first image and the second image are aligned; Matching characteristic pixel points of the first image and the second image; The disparity of characteristic pixel points of the first image and the second image is calculated to generate a disparity map to detect the disparity between the first image and the second image.

4. The method according to claim 3, further comprising: When the maximum value of the disparity of the feature pixel point in the disparity map is less than a maximum preset threshold, converting the disparity map into a depth map to obtain information and depth of the feature pixel point; The three-dimensional image is synthesized based on the depth image.

5. The method according to claim 3, wherein: Matching characteristic pixel points of the first image and the second image includes: Detecting a first characteristic pixel point within a first imaging area by the first camera; Identify the plane formed by the optical center of the first camera, the optical center of the second camera, and the first characteristic pixel point; Identify a straight line where the plane intersects the first imaging area and the second imaging area of ​​the second camera; A search is performed on the straight line for a second characteristic pixel point corresponding to a first characteristic pixel point in a second imaging area, so that the characteristic pixel points of the first imaging and the second imaging match.

6. The method according to claim 3, wherein: Parallax greater than a preset threshold includes: The proportion of feature pixel points in the disparity map having disparities greater than or equal to a maximum preset threshold is higher than the proportion of feature pixel points having disparities less than the maximum preset threshold.

7. The method according to claim 6, wherein: When the parallax is greater than a preset threshold, controlling the first imaging and / or the second imaging to move so that an overlapping area of ​​the first imaging and the second imaging meets a preset requirement, including: Get the coordinate value of the feature pixel point (X k ,Y k ) and the disparity value H k ; Based on the coordinate value (X k ,Y k ) and the disparity value H k , we get the disparity set C = {H1, H2, H3..., H k }; Determine the total distance of movement to be P units; wherein the minimum value of the total distance P is the minimum value in the disparity set C. The maximum value of the total distance P is the difference between the maximum value in the disparity set C and the minimum preset threshold, and k is the number of characteristic pixels in the imaging.

8. The method according to claim 7, further comprising: Moving the first image and the second image by P / 2 units in a direction approaching each other, or moving the first image by P units in a direction approaching the second image, or moving the second image by P units in a direction approaching the first image, so as to form a composite image; Calculating the length of the composite image; Calculating the length of the area that needs to be supplemented between the first new image and the second new image, wherein the length of the area that needs to be supplemented is equal to the length of the composite image minus the length of the overlapping portion of the first new image and the second new image; Based on the length of the area to be supplemented, determining a first newly imaged non-overlapping area in the synthesized image as a first area, and determining a second newly imaged non-overlapping area in the synthesized image as a second area; Combining the first area with the second newly imaged area of ​​the overlapping area in the composite image into a third image, and combining the second area with the first newly imaged area of ​​the overlapping area in the composite image into a fourth image; A disparity map is generated based on the third imaging and the fourth imaging.

9. A computer-readable storage medium having a computer program stored therein, wherein: When the computer program is executed by a processor, the steps of the method described in any one of claims 1 to 8 are implemented.

10. An electronic device comprising one or more memories, one or more processors, and a computer program stored in the memories and executable on the processors, wherein the processor implements the steps of the method described in any one of claims 1 to 8 when executing the computer program.

Images