Image capturing method, apparatus, terminal and storage medium

Abstract

This application relates to an image capturing method, apparatus, terminal, and storage medium. The method includes: obtaining image information, orientation information, and distance detection information of a shooting target; and determining a three-dimensional image of the shooting target based on the image information, orientation information, and distance detection information. In this method, when capturing an image, first obtain the image information, orientation information, and distance detection information of the shooting target, and then perform data fusion on the above-mentioned image information, orientation information, and distance detection information to obtain a three-dimensional image of the shooting target. Through this three-dimensional image, not only can we better understand the specific shape of the shooting target, but also we can obtain the geographical orientation of the shooting target when shooting the target, enriching the content of the image, enhancing the user experience, and better meeting the shooting needs of users.

Description

Technical Field

[0001] This application relates to the technical field of terminals, and in particular, to an image capturing method, apparatus, terminal, and storage medium. Background Art

[0002] Terminals such as mobile phones are becoming more and more popular, and more and more people are used to using mobile phones to capture images. People's requirements for mobile phone shooting are also getting higher and higher.

[0003] Currently, in addition to storing the image information captured by the camera during shooting, a mobile phone can also record the attribute information of the image collected during shooting and image processing in the image information. For example, the shooting date and global positioning system (GPS) positioning data, etc. In this way, when the user views these image information, they can further view this attribute information.

[0004] However, the content provided by the above attribute information is not comprehensive enough, and the user may still not be able to accurately restore the specific details when shooting the image by viewing this attribute information, resulting in a poor experience. Summary of the Invention

[0005] To overcome the problems in the related art, this application provides an image capturing method, apparatus, terminal, and storage medium.

[0006] According to the first aspect of the embodiments of this application, an image capturing method is provided, which is applied to a terminal. The method includes:

[0007] Obtain the image information, orientation information, and distance detection information of the shooting target;

[0008] Determine a three-dimensional image of the shooting target according to the image information, the orientation information, and the distance detection information.

[0009] Optionally, the determining the three-dimensional image of the shooting target according to the image information, the orientation information, and the distance information includes:

[0010] Determine the pixel distance information of each pixel in the image information according to the distance detection information and the image information;

[0011] Determine the pixel orientation information of each pixel in the image information according to the orientation information and the image information;

[0012] Determine the three-dimensional image according to the pixel distance information, the pixel orientation information, and the image information.

[0013] Optionally, the distance detection information includes a distance value matrix denoted as e*f, the pixel resolution of the image information is denoted as a*b, and determining the pixel distance information of each pixel in the image information according to the distance detection information and the image information includes:

[0014] Determine the distance value of each pixel in the image information according to the following rules, and use the determined distance value as the pixel distance information of the corresponding pixel:

[0015] If a = e, in each row of pixels of the image information, each pixel corresponds to a distance value in the distance value matrix;

[0016] If b = f, in each column of pixels of the image information, each pixel corresponds to a distance value in the distance value matrix;

[0017] If a > e, in each row of pixels of the image information, every a / e pixels correspond to a distance value in the distance value matrix;

[0018] If b > f, in each column of pixels of the image information, every b / f pixels correspond to a distance value in the distance value matrix;

[0019] If a < e, in each row of pixels of the image information, the average distance value of every e / a distance values in the distance value matrix corresponds to one pixel;

[0020] If b < f, in each column of pixels of the image information, the average distance value of every f / b distance values in the distance value matrix corresponds to one pixel;

[0021] Wherein, a, b, e, and f are all integers greater than or equal to 1.

[0022] Optionally, the orientation information includes an angle range deviating from a set direction, the angle range is denoted as (β, γ), the image information includes j columns of pixel groups, and determining the pixel orientation information of each pixel in the image information according to the orientation information and the image information includes:

[0023] Determine the pixel orientation information of each pixel in the image information according to the following rules:

[0024] In each column of pixel groups of the image information, the pixel orientation information of each pixel is the same;

[0025] In the image information, the pixel orientation information of the i-th column of pixel groups is: ;

[0026] Wherein, i is an integer greater than or equal to 1 and less than or equal to j.

[0027] Optionally, the method further includes:

[0028] Controlling the image capturing device to have the same rate of collecting the image information, the orientation detecting device to have the same rate of detecting the orientation information, and the distance detecting device to have the same rate of detecting the distance detecting information.

[0029] Optionally, the method further includes:

[0030] Controlling the starting moment of the frame rate of the image capturing device to collect the image information, the starting moment of the orientation detecting device to detect the orientation information, and the starting moment of the distance detecting device to detect the distance detecting information to be the same.

[0031] According to a second aspect of the embodiments of the present application, there is provided an image capturing device, which is applied to a terminal, and the device includes:

[0032] An obtaining module, configured to obtain image information, orientation information, and distance detecting information of a shooting target;

[0033] A determining module, configured to determine a three-dimensional image of the shooting target according to the image information, the orientation information, and the distance detecting information.

[0034] Optionally, the determining module is further configured to:

[0035] Determine pixel distance information of each pixel in the image information according to the distance detecting information and the image information;

[0036] Determine pixel orientation information of each pixel in the image information according to the orientation information and the image information;

[0037] Determine the three-dimensional image according to the pixel distance information, the pixel orientation information, and the image information.

[0038] Optionally, the distance detecting information includes a distance value matrix, which is denoted as e*f, and the pixel resolution of the image information is denoted as a*b. The determining module is further configured to:

[0039] Determine the distance value of each pixel in the image information according to the following rules, and use the determined distance value as the pixel distance information of the corresponding pixel:

[0040] If a = e, in each row of pixels of the image information, each pixel corresponds to a distance value in the distance value matrix;

[0041] If b = f, in each column of pixels of the image information, each pixel corresponds to a distance value in the distance value matrix;

[0042] If a > e, in each row of pixels of the image information, every a / e pixels correspond to one distance value in the distance value matrix;

[0043] If b > f, in each column of pixels of the image information, every b / f pixels correspond to one distance value in the distance value matrix;

[0044] If a < e, in each row of pixels of the image information, the average distance value of every e / a distance values in the distance value matrix corresponds to one pixel;

[0045] If b < f, in each column of pixels of the image information, the average distance value of every f / b distance values in the distance value matrix corresponds to one pixel;

[0046] Wherein, a, b, e, and f are all integers greater than or equal to 1.

[0047] Optionally, the orientation information includes an angle range deviating from a set direction, the angle range is denoted as (β, γ), the image information includes j columns of pixel groups, and the determining module is further configured to:

[0048] Determine the pixel orientation information of each pixel in the image information according to the following rules:

[0049] In each column of pixel groups of the image information, the pixel orientation information of each pixel is the same;

[0050] In the image information, the pixel orientation information of the i-th column of pixel groups is: ;

[0051] Wherein, i is an integer greater than or equal to 1 and less than or equal to j.

[0052] Optionally, the device further includes:

[0053] A control module, configured to control the image capturing device to collect the image information at the same rate as the orientation detection device detects the orientation information and the distance detection device detects the distance detection information.

[0054] Optionally, the device further includes:

[0055] A control module, configured to control the image capturing device to collect the image information at the same starting moment as the orientation detection device detects the orientation information and the distance detection device detects the distance detection information.

[0056] According to the third aspect of the embodiments of the present application, a terminal is provided, and the terminal includes an image capturing device, an orientation determining device, and a distance detection device.

[0057] Wherein, the image capturing device is configured to collect image information of a shooting target; the orientation determining device is configured to detect the orientation information of the shooting target; the distance detecting device is configured to detect distance detection information of the shooting target relative to the terminal;

[0058] The terminal includes:

[0059] A processor;

[0060] A memory for storing instructions executable by the processor;

[0061] Wherein, the processor is configured to execute the method described in the first aspect.

[0062] According to a fourth aspect of the embodiments of the present application, a non-transitory computer-readable storage medium is provided. When the instructions in the storage medium are executed by a processor of a terminal, the terminal is enabled to execute the method described in the first aspect.

[0063] The technical solutions provided by the embodiments of the present application may include the following beneficial effects: In this method, when taking an image, the image information, orientation information, and distance detection information of the shooting target are first obtained, and then the above image information, orientation information, and distance detection information are subjected to data fusion to obtain a three-dimensional image of the shooting target. Through this three-dimensional image, not only can the specific form of the shooting target be better known, but also the geographical orientation of the shooting target when the image is taken can be obtained, enriching the content of the image and improving the user experience, and better meeting the shooting needs of users.

[0064] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] The drawings herein are incorporated into the specification and form a part of the specification, showing embodiments consistent with the present invention and used together with the specification to explain the principles of the present invention.

[0066] Figure 1 is a flowchart of an image shooting method shown according to an exemplary embodiment.

[0067] Figure 2 is a block diagram of an image shooting device shown according to an exemplary embodiment.

[0068] Figure 3 is a block diagram of a terminal shown according to an exemplary embodiment. DETAILED DESCRIPTION

[0069] Exemplary embodiments will be described in detail herein, and examples thereof are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present invention as detailed in the appended claims.

[0070] The present disclosure provides an image capturing method applied to a terminal. In this method, when capturing an image, first obtain the image information, orientation information, and distance detection information of the capture target, and then perform data fusion on the above-mentioned image information, orientation information, and distance detection information to obtain a three-dimensional image (3D image) of the capture target. Through this three-dimensional image, not only can we better know the specific form of the capture target, but also we can obtain the geographical orientation of the capture target when capturing the target, enriching the content of the image and enhancing the user experience, and can better meet the user's capture needs.

[0071] In an exemplary embodiment, an image capturing method applied to a terminal is provided. Refer to Figure 1 As shown, the method includes:

[0072] S110. Obtain the image information, orientation information, and distance detection information of the capture target;

[0073] S120. Determine the three-dimensional image of the capture target according to the distance detection information, image information, and orientation information.

[0074] In step S110, the image information can be collected by an image acquisition device of the terminal, and the image acquisition device may include a camera.

[0075] For example, the processor of the terminal can control the camera to capture an image. After the camera captures the image information of the capture target, it transmits it to the processor, and the processor can then obtain the image information of the capture target.

[0076] It should be noted that since the amount of data of the image information collected by the image acquisition device is large, the image acquisition device and the processor perform data transmission through the mipi (Mobile Industry Processor Interface) protocol.

[0077] The orientation information refers to the information of the geographical orientation where the capture target is located. For example, it can be the deviation angle of the capture target from the due north orientation. The orientation information can be detected by an orientation detection device of the terminal, and the orientation detection device may include an electronic compass (also called an electronic compass, compass).

[0078] For example, when taking an image, the processor of the terminal can control the compass to detect the azimuth information of the shooting direction. It can be understood that the shooting direction of the camera is the direction where the shooting target is located. For the specific azimuth information detection method, reference can be made to the related technology, which will not be elaborated here. After the compass detects the azimuth information, it transmits it to the processor, and the processor can then obtain the geographical azimuth information of the shooting target.

[0079] The distance detection information refers to the information obtained when detecting the distance between the shooting target and the terminal. The distance detection information can be detected by the distance detection device of the terminal, and the distance detection device can include a Time of Flight (Tof) sensor.

[0080] For example, when taking an image, the processor of the terminal can control the Time of Flight sensor to detect the distance between the shooting target and the terminal. When the Time of Flight sensor detects the distance detection information between the shooting target and the terminal, it transmits it to the processor, and the processor can then obtain the distance information between the shooting target and the terminal.

[0081] The principle of the Time of Flight sensor detecting the distance detection information is as follows: The Time of Flight sensor includes a signal transmitting end and multiple signal receiving ends, and the multiple signal receiving ends form a signal receiving matrix. The signal transmitting end emits a ranging signal (such as an electromagnetic wave signal). After the ranging signal is transmitted to the shooting target, it can be reflected back to the Time of Flight sensor. Thus, each signal receiving end can directly receive the ranging signal emitted by the signal transmitting end and also receive the ranging signal feedback from the shooting target. First, determine the time interval between receiving the ranging signal twice and denote it as tn, and then determine that the distance value measured by this signal receiving end is dn = c * tn / 2. Here, c represents the propagation speed of the electromagnetic wave signal, and n represents the number of the signal receiving end. That is, the time interval of the first signal receiving end is t1, and the corresponding distance value is d1; the time interval of the second signal receiving end is t2, and the corresponding distance value is d2, and so on. The above multiple distance values constitute the distance detection information of the shooting target.

[0082] It should be noted that when using the above matrix-type Time of Flight sensor to detect the distance between the shooting target and the terminal, relatively powerful data processing capabilities are required. Therefore, in this method, after the distance detection information is detected by the Time of Flight sensor, the above distance detection information needs to be transmitted to the processor, and then the processor performs calculations and other processing on the distance detection information to finally obtain the target distance detection information of the shooting target.

[0083] In step S120, after the processor obtains the distance detection information, image information, and orientation information of the shooting target, the three-dimensional image can be drawn through image fusion technology. In the obtained three-dimensional image, it includes both the stereoscopic image of the shooting target and the orientation information of the shooting target in the geographical space (such as the angle deviating from the due north direction), improving the user experience.

[0084] Among them, when using a time-of-flight sensor to detect the distance detection information, the distance detection information refers to the detection information of the distance detection device, which may include the energy information of the ranging signals received by each signal receiving end in the time-of-flight sensor. The processor determines the time information of the ranging signals received by each signal receiving end according to the above energy information (for example, when the energy characteristics of the ranging signal received by the signal receiving end match the preset energy characteristics, then determine this moment as the moment when the ranging signal is received. For details, reference can be made to the existing technology and will not be elaborated here). Then, the time interval information is determined, and finally the target distance detection information is calculated. The target distance detection information refers to the information about the distance between the shooting target and the terminal.

[0085] In this method, an image acquisition device, an orientation detection device, and a distance detection device are simultaneously set in the terminal. When taking an image, the image information, orientation information, and distance detection information of the shooting target are simultaneously obtained, and then the above image information, orientation information, and distance detection information are subjected to data fusion processing to obtain the three-dimensional image of the shooting target. Through this three-dimensional image, the stereoscopic form of the shooting target can be observed from multiple angles and in multiple directions. Not only can we better know the specific form of the shooting target, but also we can obtain the geographical orientation of the shooting target when the image is taken, enriching the content of the image and improving the user experience.

[0086] In an exemplary embodiment, an image shooting method applied to a terminal is provided. In this method, according to the image information, orientation information, and distance detection information, determining the three-dimensional image of the shooting target includes:

[0087] S210. Determine the pixel distance information of each pixel in the image information according to the distance detection information and the image information;

[0088] S220. Determine the pixel orientation information of each pixel in the image information according to the orientation information and the image information;

[0089] S230. Determine the three-dimensional image according to the pixel distance information, pixel orientation information, and image information.

[0090] In step S210, when the distance detection information is the detection information of the time-of-flight sensor, the processor also needs to determine the target distance information according to the distance detection information, and then determine the pixel distance information of each pixel in the image information, where the pixel distance information represents the information of the distance between each pixel and the terminal.

[0091] When the distance detection information directly represents the information of the distance between the shooting target and the terminal, the distance detection information is directly determined as the target distance information.

[0092] Among them, the pixel resolution of the image information is denoted as a*b, that is, the image information includes a*b pixels; the distance value matrix of the target distance information is e*f, that is, the distance detection information includes a distance value matrix of e*f, and this distance value matrix includes e*f distance values.

[0093] In this method, the distance value of each pixel in the image information can be determined according to the following rule (equal division method), and the determined distance value is used as the pixel distance information of the corresponding pixel:

[0094] If a = e, in each row of pixels of the image information, each pixel corresponds to a distance value in the distance value matrix;

[0095] If b = f, in each column of pixels of the image information, each pixel corresponds to a distance value in the distance value matrix;

[0096] If a > e, in each row of pixels of the image information, every a / e pixels correspond to a distance value in the distance value matrix;

[0097] If b > f, in each column of pixels of the image information, every b / f pixels correspond to a distance value in the distance value matrix;

[0098] If a < e, in each row of pixels of the image information, the average distance value of every e / a distance values in the distance value matrix corresponds to one pixel;

[0099] If b < f, in each column of pixels of the image information, the average distance value of every f / b distance values in the distance value matrix corresponds to one pixel;

[0100] Among them, a, b, e, and f are all integers greater than or equal to 1.

[0101] It should be noted that when determining the pixel distance information of each pixel, it is necessary to correspond in sequence.

[0102] Example 1

[0103] The terminal is equipped with a camera, and the pixel resolution of the camera is denoted as 800*1600. That is, the resolution of the image information obtained by the camera is 800*1600. The terminal is also equipped with a time-of-flight sensor, and the signal receiving end in the time-of-flight sensor forms a matrix of 800*1600. That is, the distance value matrix of the target distance information is denoted as 800*1600.

[0104] In each row of pixels of the image information, each pixel corresponds to a distance value in the distance value matrix. That is, the first pixel corresponds to the first distance value, the second pixel corresponds to the second distance value, and so on.

[0105] In each column of pixels of the image information, each pixel corresponds to a distance value in the distance value matrix. That is, the first pixel corresponds to the first distance value, the second pixel corresponds to the second distance value, and so on.

[0106] In the above way, the pixel distance value of each pixel can be determined.

[0107] Example 2,

[0108] The terminal is equipped with a camera, and the pixel resolution of the camera is denoted as 800*1600. That is, the resolution of the image information obtained by the camera is 800*1600. The terminal is also equipped with a time-of-flight sensor, and the signal receiving end in the time-of-flight sensor forms an 8*8 matrix. That is, the distance value matrix of the target distance information is denoted as 8*8.

[0109] In each row of pixels of the image information, every 800 / 8 = 100 pixels correspond to a distance value in the distance value matrix. That is, the first 100 pixels correspond to the first distance value, the second 100 pixels correspond to the second distance value, and so on.

[0110] In each column of pixels of the image information, every 1600 / 8 = 200 pixels correspond to a distance value in the distance value matrix. That is, the first 200 pixels correspond to the first distance value, the second 200 pixels correspond to the second distance value, and so on.

[0111] In the above way, the pixel distance value of each pixel can be determined.

[0112] Example 3,

[0113] The terminal is equipped with a camera, and the pixel resolution of the camera is denoted as 300*600. That is, the resolution of the image information obtained by the camera is 300*400. The terminal is also equipped with a time-of-flight sensor, and the signal receiving end in the time-of-flight sensor forms a 600*800 matrix. That is, the distance value matrix of the target distance information is denoted as 600*800.

[0114] In each row of pixels of the image information, the average distance value of every 600 / 300 = 2 distance values in the distance value matrix corresponds to one pixel. That is, the average distance value of the first 2 distance values in the distance value matrix is used as the distance value of the first pixel, the average distance value of the second 2 distance values is used as the distance value of the second pixel, and so on.

[0115] In each column of pixels of the image information, the average distance value of every 800 / 400 = 2 distance values in the distance value matrix corresponds to one pixel. That is, the average distance value of the first 2 distance values in the distance value matrix is used as the distance value of the first pixel, the average distance value of the second 2 distance values is used as the distance value of the second pixel, and so on.

[0116] In the above way, the pixel distance value of each pixel can be determined.

[0117] It should be noted that currently, the ranging accuracy of the time-of-flight sensor is much smaller than the pixel resolution of the camera. Therefore, generally, each distance value in the target distance information corresponds to multiple pixels in the image information.

[0118] In step S220, the method for determining the pixel orientation information is similar to the method for determining the pixel distance information described above.

[0119] Currently, when using the compass application of the terminal, generally only the orientation information in the due front of the terminal is displayed, that is, the orientation information corresponding to the exact middle of the terminal. In this method, when taking an image, the orientation information corresponding to the edge of the imaging area in the terminal is detected by the electronic compass. In this way, the orientation range where the shooting target in the captured image is located can be determined (for example, the range deviating from the true north direction by 10° - 30°), and then the equal division method is used to determine the pixel orientation information of each pixel.

[0120] In this step, the orientation information may include the angle range deviating from the set direction (such as the true north direction), and the angle range is denoted as (β, γ). The image information includes j columns of pixel groups. According to the orientation information and the image information, determining the pixel orientation information of each pixel in the image information includes:

[0121] Determine the pixel orientation information of each pixel in the image information according to the following rule (equal division method):

[0122] In each column of pixel groups of the image information, the pixel orientation information of each pixel is the same;

[0123] In each row of pixels of the image, the pixel orientation information of the i-th pixel is ;

[0124] where i is an integer greater than or equal to 1 and less than or equal to j.

[0125] It should be noted that each column of pixel groups may include only one column of pixels, or may include two columns or even more columns of pixels.

[0126] Example 1

[0127] The image information includes 100 columns of pixel groups, and each column of pixel groups includes only 1 column of pixels, that is, the image information includes 100 columns of pixels. The orientation information is in the range deviating from the true north direction of the earth by (10°, 30°). It should be noted that the orientation information includes the critical angles 10° and 30°.

[0128] Among them, in the first column of pixels, the pixel orientation information of each pixel is:

[0129] ;

[0130] In the second column of pixels, the pixel orientation information of each pixel is:

[0131]

[0132] And so on, the pixel orientation information of each pixel in the image information can be determined. In this example, each column of pixels determines a pixel orientation information, and the imaging effect is better.

[0133] Example 1

[0134] The image information includes 50 columns of pixel groups, and each column of pixel groups includes only 2 columns of pixels, that is, the image information includes 100 columns of pixels. The orientation information is in the range deviating from the true north direction of the earth by (10°, 30°). It should be noted that the orientation information includes the critical angles 10° and 30°.

[0135] Among them, the first column of pixels and the second column of pixels are used as the first pixel group. In this pixel group, the pixel orientation information of each pixel is:

[0136] ;

[0137] The third column of pixels and the fourth column of pixels are used as the second pixel group. In this pixel group, the pixel orientation information of each pixel is:

[0138]

[0139] And so on, the pixel orientation information of each pixel in the image information can be determined. In this example, multiple columns of pixels are used as a pixel group to determine a pixel orientation information, which can reduce the requirement for data processing ability and reduce the energy consumption of the terminal.

[0140] In step S230, after determining the pixel distance information, pixel orientation information, and image information, the pixels in the image information can be processed according to the pixel distance information of each pixel and the pixel orientation information of each pixel, and then a three-dimensional image including orientation information, distance information, and image information is obtained.

[0141] In this method, a pixel distance information and a pixel orientation information are assigned to each pixel in the image information, so that the finally formed three-dimensional image is more vivid and three-dimensional, and the contained information is more abundant. When the user observes the three-dimensional image subsequently, not only can the user better understand the shape of the shooting target, but also can understand the geographical orientation where the shooting target is located, which can better meet the different needs of the user and improve the user experience.

[0142] In an exemplary embodiment, an image shooting method is provided, which is applied to a terminal. In this method, when using an image shooting device to collect image information, using an orientation detection device to detect orientation information, and using a distance detection device to detect distance detection information, the method further includes:

[0143] Controlling the image shooting device, the orientation detection device, and the distance detection device to have the same rate of collecting image information, detecting orientation information, and detecting distance detection information, respectively.

[0144] It can be understood that when taking pictures, multiple frames of pictures are often taken. In this method, the image information includes multiple frames of sub-picture information, the orientation information also includes multiple frames of sub-orientation information, and the distance detection information includes multiple frames of sub-detection information. Controlling the image shooting device, the orientation detection device, and the distance detection device to have the same rate of collecting image information, detecting orientation information, and detecting distance detection information, respectively, is to make the above-mentioned multiple frames of sub-picture information, multiple frames of sub-orientation information, and multiple frames of sub-detection information correspond one by one.

[0145] For example, when the picture information includes 20 frames of sub-picture information, the orientation information includes 20 frames of sub-orientation information, and the distance detection information includes 20 frames of sub-detection information, since the image shooting device, the orientation detection device, and the distance detection device have the same rate of collecting image information, detecting orientation information, and detecting distance detection information, respectively, the first frame of sub-picture information, the first frame of sub-orientation information, and the first frame of sub-distance information correspond one by one, the second frame of sub-picture information, the second frame of sub-orientation information, and the second frame of sub-distance information correspond one by one, and so on. Thus, it is determined that each frame of sub-picture information corresponds to one frame of sub-orientation information and one frame of sub-distance information, so as to improve the synthesis effect of the image information, the orientation information, and the distance information, enhance the shooting effect, and better ensure that the finally drawn three-dimensional image is more realistic.

[0146] Wherein, the method further includes:

[0147] The starting time of the frame rate at which the image capture device captures image information, the starting time of the orientation detection device detecting orientation information, and the starting time of the distance detection device detecting distance detection information are the same.

[0148] In this method, when taking an image, the starting time of the frame rate at which the image capture device captures image information, the starting time of the orientation detection device detecting orientation information, and the starting time of the distance detection device detecting distance detection information are the same. That is, the above three devices are controlled to simultaneously capture the corresponding information of the shooting target, so that the three pieces of information are more matched, and the effect of the finally obtained three-dimensional image is better.

[0149] In an exemplary embodiment, an image shooting method is provided, which is applied to a terminal, and the terminal can be a mobile phone. In this method, image information is captured by a camera in the mobile phone, orientation information is detected by an electronic compass in the mobile phone, and distance detection information is detected by a time-of-flight sensor in the mobile phone.

[0150] When taking an image, control the camera, electronic compass, and time-of-flight sensor to start simultaneously and collect the above information at the same rate to ensure that the timestamps of the finally obtained image information, orientation information, and distance detection information are the same, better perform image synthesis, and improve the final imaging effect.

[0151] For example, control the three to start at 10:00, and then collect a corresponding piece of information every 1 / 1000 second. That is, the camera captures sub-image information of a shooting target every 1 / 1000 second, the electronic compass detects sub-orientation information every 1 / 1000 second, the time-of-flight sensor collects a set of sub-detection information for determining distance information every 1 / 1000 second, and then the processor determines a distance matrix according to the sub-detection information.

[0152] Suppose 100 sub-image information, 100 sub-orientation information, and 100 sub-distance detection information are finally obtained.

[0153] When performing image synthesis, first the processor processes the sub-distance detection information to obtain 100 sub-target distance information.

[0154] Among them, the nth sub-image information, the nth sub-orientation information, and the nth sub-distance information correspond one by one (n is an integer greater than 0). First, respectively synthesize multiple sub-image information into an overall image information, synthesize multiple sub-orientation information into an overall orientation information, synthesize multiple sub-distance information into an overall distance information, and then perform data fusion processing on the above overall image information, overall orientation information, and overall distance information to obtain the final overall three-dimensional image.

[0155] It should be noted that the corresponding sub-image information, sub-orientation information, and sub-distance information can also be first subjected to data fusion to obtain 100 sub-three-dimensional images, and then the 100 sub-three-dimensional images are synthesized to obtain the final complete three-dimensional image.

[0156] The three-dimensional image obtained by this method contains richer information. When the user observes this three-dimensional image subsequently, not only can the user better understand the shape of the shooting target, but also can understand the geographical orientation where the shooting target is located, which can better meet the different needs of the user and improve the user experience.

[0157] In an exemplary embodiment, an image capturing device is provided, which is applied to a terminal. This device is used to implement the above-mentioned image capturing method. Refer to Figure 2 As shown, this device includes an acquisition module 101 and a determination module 102. In the process of implementing the above method,

[0158] The acquisition module 101 is used to acquire the image information, orientation information, and distance detection information of the shooting target;

[0159] The determination module 102 is used to determine the three-dimensional image of the shooting target according to the image information, orientation information, and distance detection information.

[0160] In an exemplary embodiment, an image capturing device is provided, which is applied to a terminal. In this device, the determination module 102 is further used for:

[0161] Determine the pixel distance information of each pixel in the image information according to the distance detection information and the image information;

[0162] Determine the pixel orientation information of each pixel in the image information according to the orientation information and the image information;

[0163] Determine the three-dimensional image according to the pixel distance information, pixel orientation information, and image information.

[0164] In an exemplary embodiment, an image capturing device is provided, which is applied to a terminal. The distance detection information includes a distance value matrix, which is denoted as e*f, and the pixel resolution of the image information is denoted as a*b. Refer to Figure 2 As shown, in this device, the determination module 102 is further used for:

[0165] Determine the distance value of each pixel in the image information according to the following rules, and use the determined distance value as the pixel distance information of the corresponding pixel:

[0166] If a = e, in each row of pixels of the image information, each pixel corresponds to a distance value in the distance value matrix;

[0167] If b = f, in each column of pixels of the image information, each pixel corresponds to a distance value in the distance value matrix;

[0168] If a > e, in each row of pixels of the image information, every a / e pixels correspond to a distance value in the distance value matrix;

[0169] If b > f, in each column of pixels of the image information, every b / f pixels correspond to a distance value in the distance value matrix;

[0170] If a < e, in each row of pixels of the image information, the average distance value of every e / a distance values in the distance value matrix corresponds to one pixel;

[0171] If b < f, in each column of pixels of the image information, the average distance value of every f / b distance values in the distance value matrix corresponds to one pixel;

[0172] Wherein, a, b, e, and f are all integers greater than or equal to 1.

[0173] In an exemplary embodiment, an image capturing device is provided and applied to a terminal. The orientation information includes an angular range deviating from a set direction, and the angular range is denoted as (β, γ). The image information includes j columns of pixel groups. Refer to Figure 2 As shown, in this device, the determining module 102 is further configured to:

[0174] Determine the pixel orientation information of each pixel in the image information according to the following rules:

[0175] In each column of pixel groups of the image information, the pixel orientation information of each pixel is the same;

[0176] In the image information, the pixel orientation information of the i-th column of pixel groups is: ;

[0177] Wherein, i is an integer greater than or equal to 1 and less than or equal to j.

[0178] In an exemplary embodiment, an image capturing device is provided and applied to a terminal. The device further includes:

[0179] A control module 103, configured to control the image capturing device, the orientation detection device, and the distance detection device to have the same rate of collecting image information, detecting orientation information, and detecting distance detection information, respectively.

[0180] In an exemplary embodiment, an image capturing device is provided and applied to a terminal. The device further includes:

[0181] The control module 103 is configured to make the starting moments of the frame rate at which the image capturing device captures image information, the starting moment at which the orientation detection device detects orientation information, and the starting moment at which the distance detection device detects distance detection information the same.

[0182] This application also provides a terminal, which can be a device with an image capturing function such as a mobile phone, a tablet computer, a laptop computer, a camera, a digital camera, etc. The terminal includes an image capturing device, an orientation determination device, and a distance detection device. Among them, the image capturing device is configured to capture image information of a shooting target; the orientation determination device is configured to detect the orientation information of the shooting target; the distance detection device is configured to detect the distance detection information of the shooting target relative to the terminal. The terminal further includes a processor and a memory for storing processor-executable instructions. Among them, the processor is configured to execute the above-mentioned image capturing method.

[0183] In this terminal, when taking an image, first obtain the image information, orientation information, and distance detection information of the shooting target, and then perform data fusion on the above-mentioned image information, orientation information, and distance detection information to obtain a three-dimensional image of the shooting target. Through this three-dimensional image, not only can we better know the specific form of the shooting target, but also we can obtain the geographical orientation where the shooting target is located when shooting the target, enriching the content of the image and enhancing the user experience, and can better meet the shooting needs of users.

[0184] In an exemplary embodiment, as shown in Figure 3 Terminal 400 may include one or more of the following components: a processing component 402, a memory 404, a power component 406, a multimedia component 408, an audio component 410, an input / output (I / O) interface 412, a sensor component 414, and a communication component 416.

[0185] The processing component 402 generally controls the overall operation of the terminal 400, such as operations associated with display, telephone call, data communication, camera operation, and recording operation. The processing component 402 may include one or more processors 420 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 402 may include one or more modules to facilitate the interaction between the processing component 402 and other components. For example, the processing component 402 may include a multimedia module to facilitate the interaction between the multimedia component 408 and the processing component 402.

[0186] The memory 404 is configured to store various types of data to support the operation of the terminal 400. Examples of such data include instructions for any application or method operating on the terminal 400, contact data, phone book data, messages, pictures, videos, and the like. The memory 404 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0187] The power component 406 provides power for various components of the terminal 400. The power component 406 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the terminal 400.

[0188] The multimedia component 408 includes a screen that provides an output interface between the terminal 400 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors can not only sense the boundaries of the touch or swipe actions, but also detect the duration and pressure associated with the touch or swipe operations. In some embodiments, the multimedia component 408 includes a front camera module and / or a rear camera module. When the terminal 400 is in an operating mode, such as a shooting mode or a video mode, the front camera module and / or the rear camera module can receive external multimedia data. Each of the front camera module and the rear camera module can be a fixed optical lens system or have a focal length and optical zoom capabilities.

[0189] The audio component 410 is configured to output and / or input audio signals. For example, the audio component 410 includes a microphone (MIC) that is configured to receive external audio signals when the terminal 400 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals can be further stored in the memory 404 or transmitted via the communication component 416. In some embodiments, the audio component 410 further includes a speaker for outputting audio signals.

[0190] The I / O interface 412 provides an interface between the processing component 402 and a peripheral interface module, which can be a keyboard, click wheel, buttons, etc. These buttons can include, but are not limited to: a home button, volume buttons, a power button, and a lock button.

[0191] The sensor assembly 414 includes one or more sensors for providing a status assessment of various aspects of the terminal 400. For example, the sensor assembly 414 can detect the on / off state of the terminal 400, the relative positioning of components, such as the display and keypad of the terminal 400. The sensor assembly 414 can also detect a change in the position of the terminal 400 or a component of the terminal 400, the presence or absence of user contact with the terminal 400, the orientation or acceleration / deceleration of the terminal 400, and the temperature change of the terminal 400. The sensor assembly 414 can include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 414 can also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 414 can also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0192] The communication component 416 is configured to facilitate communication between the terminal 400 and other devices in a wired or wireless manner. The device 700 can access a wireless network based on communication standards, such as WiFi, 2G, 3G, 4G, 5G, or a combination thereof. In an exemplary embodiment, the communication component 416 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 416 further includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0193] In an exemplary embodiment, the terminal 400 can be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components for performing the above method.

[0194] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions, such as the memory 404 including instructions, is also provided. The above instructions can be executed by the processor 420 of the terminal 400 to complete the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device, etc. When the instructions in the storage medium are executed by the processor of the terminal, the terminal is enabled to execute the method shown in the above embodiments.

[0195] Other embodiments of the present invention will be readily apparent to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include known common general knowledge or conventional technical means in the technical field not disclosed in this application. The specification and examples are only to be considered as exemplary, and the true scope and spirit of the invention are pointed out by the claims.

[0196] It should be understood that the present invention is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present invention is only limited by the appended claims.

Claims

1. An image capturing method, applied to a terminal, characterized in that, The method includes: Obtaining image information, orientation information, and distance detection information of a shooting target; Determining a three-dimensional image of the shooting target according to the image information, the orientation information, and the distance detection information; Determining pixel distance information of each pixel in the image information according to the distance detection information and the image information, where the distance detection information includes a distance value matrix denoted as e*f, the pixel resolution of the image information is denoted as a*b, determining the distance value of each pixel in the image information according to the equal division method, and using the determined distance value as the pixel distance information of the corresponding pixel.

2. The method according to claim 1, wherein The determining the three-dimensional image of the shooting target according to the image information, the orientation information, and the distance information includes: Determining pixel orientation information of each pixel in the image information according to the orientation information and the image information; Determining the three-dimensional image according to the pixel distance information, the pixel orientation information, and the image information.

3. The method according to claim 2, wherein The determining the pixel distance information of each pixel in the image information according to the distance detection information and the image information includes: Determining the distance value of each pixel in the image information according to the following rules, and using the determined distance value as the pixel distance information of the corresponding pixel: If a = e, in each row of pixels of the image information, each pixel corresponds to a distance value in the distance value matrix; If b = f, in each column of pixels of the image information, each pixel corresponds to a distance value in the distance value matrix; If a > e, in each row of pixels of the image information, every a / e pixels correspond to a distance value in the distance value matrix; If b > f, in each column of pixels of the image information, every b / f pixels correspond to a distance value in the distance value matrix; If a < e, in each row of pixels of the image information, the average distance value of every e / a distance values in the distance value matrix corresponds to one pixel; If b < f, in each column of pixels of the image information, the average distance value of every f / b distance values in the distance value matrix corresponds to one pixel; Where a, b, e, and f are all integers greater than or equal to 1.

4. The method according to claim 3, characterized in that, The orientation information includes an angle range deviating from a set direction, the angle range is denoted as (β, γ), the image information includes j columns of pixel groups, and the determining the pixel orientation information of each pixel in the image information according to the orientation information and the image information includes: Determining the pixel orientation information of each pixel in the image information according to the following rules: In each column of pixel groups of the image information, the pixel orientation information of each pixel is the same; In the image information, the pixel orientation information of the pixel group in the i-th column is as follows: ; Where i is an integer greater than or equal to 1 and less than or equal to j.

5. The method according to any one of claims 1-4, characterized in that, The method further includes: Controlling the image capture device, the orientation detection device, and the distance detection device to have the same rate of collecting the image information, detecting the orientation information, and detecting the distance detection information, respectively.

6. The method according to any one of claims 1-4, characterized in that, The method further includes: The starting moment of the frame rate at which the image capturing device acquires the image information, the starting moment at which the orientation detection device detects the orientation information, and the starting moment at which the distance detection device detects the distance detection information are the same.

7. An image capturing device, applied to a terminal, characterized in that The device includes: An acquisition module, configured to acquire image information, orientation information, and distance detection information of a shooting target; A determination module, configured to determine a three-dimensional image of the shooting target according to the image information, the orientation information, and the distance detection information; The determination module is further configured to determine the pixel distance information of each pixel in the image information according to the distance detection information and the image information, where the distance detection information includes a distance value matrix, the distance value matrix is denoted as e*f, the pixel resolution of the image information is denoted as a*b, determine the distance value of each pixel in the image information according to the equal division method, and use the determined distance value as the pixel distance information of the corresponding pixel.

8. The device according to claim 7, characterized in that, The determination module is further configured to: Determine the pixel orientation information of each pixel in the image information according to the orientation information and the image information; Determine the three-dimensional image according to the pixel distance information, the pixel orientation information, and the image information.

9. The device according to claim 8, wherein The determination module is further configured to: Determine the distance value of each pixel in the image information according to the following rules, and use the determined distance value as the pixel distance information of the corresponding pixel: If a = e, in each row of pixels of the image information, each pixel corresponds to a distance value in the distance value matrix; If b = f, in each column of pixels of the image information, each pixel corresponds to a distance value in the distance value matrix; If a > e, in each row of pixels of the image information, every a / e pixels correspond to a distance value in the distance value matrix; If b > f, in each column of pixels of the image information, every b / f pixels correspond to a distance value in the distance value matrix; If a < e, in each row of pixels of the image information, the average distance value of every e / a distance values in the distance value matrix corresponds to one pixel; If b < f, in each column of pixels of the image information, the average distance value of every f / b distance values in the distance value matrix corresponds to one pixel; Wherein, a, b, e, and f are all integers greater than or equal to 1.

10. The device according to claim 9, wherein, The orientation information includes an angle range deviating from a set direction, the angle range is denoted as (β, γ), the image information includes j columns of pixel groups, and the determination module is further configured to: Determine the pixel orientation information of each pixel in the image information according to the following rules: In each column of pixel groups of the image information, the pixel orientation information of each pixel is the same; In the image information, the pixel orientation information of the pixel group in the i-th column is as follows: ; Wherein, i is an integer greater than or equal to 1 and less than or equal to j.

11. The device according to any one of claims 7-10, characterized in that, The device further includes: A control module, configured to control the image capturing device, the orientation detection device, and the distance detection device to have the same rate of acquiring the image information, detecting the orientation information, and detecting the distance detection information, respectively.

12. The device according to any one of claims 7 to 10, characterized in that, The device further includes: A control module is used to control the starting time of the frame rate at which the image capturing device captures the image information, the starting time of the azimuth detection device detecting the azimuth information, and the starting time of the distance detection device detecting the distance detection information to be the same.

13. A terminal, characterized in that, The terminal includes an image capturing device, an azimuth determination device, and a distance detection device. Wherein, the image capturing device is configured to capture image information of a shooting target; the azimuth determination device is configured to detect the azimuth information of the shooting target; the distance detection device is configured to detect distance detection information of the shooting target relative to the terminal. The terminal further includes: A processor; A memory for storing instructions executable by the processor; Wherein, the processor is configured to execute the method according to any one of claims 1 to 6.

14. A non-transitory computer-readable storage medium, characterized in that, When the instructions in the storage medium are executed by the processor of the terminal, the terminal is enabled to execute the method according to any one of claims 1 to 6.

Images