Image shooting method and device, electronic equipment and storage medium

Abstract

The invention provides an image shooting method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining the distance between a shooting device and a to-be-shot object in response to the detection of the starting of the shooting device in the electronic equipment; based on the distance and a calibration gradient matrix, the position of a photosensitive element in the shooting device is adjusted, and the calibration gradient matrix is used for representing the corresponding relation between the distance and the adjustment position of the photosensitive element; and shooting an image based on the photosensitive element with the adjusted position. The distance between the shooting device and the object to be shot is obtained, the position of the photosensitive element in the shooting device is determined based on the calibration gradient matrix, and the photosensitive element is adjusted according to the determined position. The image is shot based on the adjusted photosensitive element, so that a globally clear image is obtained, and shooting experience with higher definition is brought to a user.

Description

Technical Field

[0001] The present disclosure relates to the technical field of electronic devices, and in particular to an image capturing method, device, electronic device, and storage medium. Background Art

[0002] With the development of science and technology and advancements in electronic device technology, the shooting performance of everyday electronic devices has gradually improved. This improvement can be achieved by improving the image acquisition process or by improving the image processing process. Improving the image acquisition process ensures clearer image quality; improving the image processing process can provide users with a better image quality and clarity experience.

[0003] However, in related technologies, neither improvements to the image acquisition process nor improvements to the image processing process can enable users to obtain globally clear images. Therefore, solutions for obtaining globally clear images have become the focus and direction of related human eye research. Summary of the Invention

[0004] To overcome the problems existing in the related art, the present disclosure provides an image capturing method, an apparatus, an electronic device, and a storage medium.

[0005] According to a first aspect of an embodiment of the present disclosure, there is provided an image capturing method, comprising: in response to detecting that a capturing device in an electronic device is turned on, obtaining a distance between the capturing device and an object to be captured; adjusting a position of a photosensitive element in the capturing device based on the distance and a calibration gradient matrix, wherein the calibration gradient matrix is ​​used to characterize a correspondence between the distance and the adjusted position of the photosensitive element; and capturing an image based on the photosensitive element after the adjusted position.

[0006] In one embodiment, the calibration gradient matrix is ​​determined as follows: based on the focusing range of the camera device, the focusing range is divided into multiple equal points, and each equal point is determined to be the target distance; the photographed object is moved to the i-th target distance, and the photosensitive element in the photographing device is moved until the brightness value obtained by the photosensitive element at the corresponding pixel position of the calibration template is equal to the preset brightness threshold, and the first correspondence between the position of the photosensitive element and the i-th target distance is recorded; wherein the calibration template is determined based on the number of pixels of the image captured by the camera device and includes multiple pixel positions; the object to be photographed is moved to the i+1-th target distance, and the above process is repeated until the first correspondence between the target distance corresponding to each of the multiple equal points and the position of the photosensitive element is obtained; based on the first correspondence between the target distance corresponding to each of the multiple equal points and the position of the photosensitive element, the calibration gradient matrix is ​​determined.

[0007] In one embodiment, the calibration gradient matrix is ​​determined based on the first correspondence between the target distance corresponding to each of the multiple equally divided points and the position of the photosensitive element, including: fitting the first correspondence between the target distance corresponding to each of the multiple equally divided points and the position of the photosensitive element, obtaining a second correspondence between the corresponding focus distance within the focus range of the shooting device and the position of the photosensitive element, and using the second correspondence as the calibration gradient matrix.

[0008] In one embodiment, the calibration template is determined based on the number of pixels of an image captured by a camera device in the following manner: based on the width of the image captured by the camera device and the number of pixels included in a row direction of a preset target template, X target templates included in the width of the image captured by the camera device are determined, where X is a value obtained by rounding down the ratio of the width to the number of pixels included in the row direction; based on the height of the image captured by the camera device and the number of pixels included in a column direction of a preset target template, Y target templates included in the height of the image captured by the camera device are determined, where Y is a value obtained by rounding down the ratio of the height to the number of pixels included in the column direction; and X*Y target templates are used as the calibration template.

[0009] In one embodiment, the position of the photosensitive element in the shooting device is adjusted based on a voice coil motor, and the photosensitive element includes a plurality of photosensitive elements.

[0010] In one embodiment, obtaining the distance between the shooting device and the object to be photographed includes: obtaining the distance between each pixel point of the shooting device and the object to be photographed based on a pixel-level distance sensor.

[0011] According to a second aspect of an embodiment of the present disclosure, an image capturing device is provided, comprising: an acquisition unit for acquiring the distance between the capturing device and an object to be captured in response to detecting that the capturing device in an electronic device is turned on; an adjustment unit for adjusting the position of a photosensitive element in the capturing device based on the distance and a calibration gradient matrix, wherein the calibration gradient matrix is ​​used to characterize the correspondence between the distance and the adjusted position of the photosensitive element; and a capturing unit for capturing an image based on the photosensitive element after the position is adjusted.

[0012] In one embodiment, the adjustment unit determines the calibration gradient matrix in the following manner: based on the focusing range of the camera device, the focusing range is divided into multiple equal points, and each equal point is determined as the target distance; the shooting object is moved to the i-th target distance, and the photosensitive element in the shooting device is moved until the brightness value obtained by the photosensitive element at the corresponding pixel position of the calibration template is equal to the preset brightness threshold, and the first correspondence between the position of the photosensitive element and the i-th target distance is recorded; wherein the calibration template is determined based on the number of pixels of the image captured by the camera device and includes multiple pixel positions; the object to be shot is moved to the i+1-th target distance, and the above process is repeated until the first correspondence between the target distance corresponding to each of the multiple equal points and the position of the photosensitive element is obtained; based on the first correspondence between the target distance corresponding to each of the multiple equal points and the position of the photosensitive element, the calibration gradient matrix is ​​determined.

[0013] In one embodiment, the adjustment unit determines the calibration gradient matrix based on the first correspondence between the target distance corresponding to each of the multiple equally divided points and the position of the photosensitive element in the following manner: the first correspondence between the target distance corresponding to each of the multiple equally divided points and the position of the photosensitive element is fitted to obtain a second correspondence between the corresponding focus distance within the focus range of the shooting device and the position of the photosensitive element, and the second correspondence is used as the calibration gradient matrix.

[0014] In one embodiment, the calibration template is determined based on the number of pixels of an image captured by a camera device in the following manner: based on the width of the image captured by the camera device and the number of pixels included in a row direction of a preset target template, X target templates included in the width of the image captured by the camera device are determined, where X is a value obtained by rounding down the ratio of the width to the number of pixels included in the row direction; based on the height of the image captured by the camera device and the number of pixels included in a column direction of a preset target template, Y target templates included in the height of the image captured by the camera device are determined, where Y is a value obtained by rounding down the ratio of the height to the number of pixels included in the column direction; and X*Y target templates are used as the calibration template.

[0015] In one embodiment, the position of the photosensitive element in the shooting device is adjusted based on a voice coil motor, and the photosensitive element includes a plurality of photosensitive elements.

[0016] In one embodiment, the acquisition unit acquires the distance between the shooting device and the object to be photographed in the following manner: based on a pixel-level distance sensor, the distance between the shooting device and each pixel point of the object to be photographed is acquired.

[0017] According to a third aspect of an embodiment of the present disclosure, there is provided an electronic device, including:

[0018] processor;

[0019] a memory for storing processor-executable instructions;

[0020] The processor is configured to: execute the method described in the first aspect or any one of the embodiments of the first aspect.

[0021] According to a fourth aspect of an embodiment of the present disclosure, a storage medium is provided, in which instructions are stored. When the instructions in the storage medium are executed by a processor of a terminal, the terminal is enabled to execute the image capture method described in the first aspect or any one of the embodiments of the first aspect.

[0022] The technical solutions provided by the embodiments of the present disclosure may have the following beneficial effects: by obtaining the distance between a camera and a subject to be photographed, determining the position of a photosensitive element in the camera based on a calibration gradient matrix, and adjusting the photosensitive element according to the determined position, an image is captured based on the adjusted photosensitive element, thereby obtaining a globally clear image and providing users with a higher-definition photography experience.

[0023] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0025] Figure 1 The figure is a flowchart of an image capturing method according to an exemplary embodiment.

[0026] Figure 2 FIG. 4 is a schematic structural diagram of a Dual PD according to an exemplary embodiment.

[0027] Figure 3 The figure is a flow chart showing a method for determining a calibration gradient matrix according to an exemplary embodiment.

[0028] Figure 4 is a schematic diagram showing a phase focusing principle according to an exemplary embodiment.

[0029] Figure 5 The figure is a flow chart showing a method for determining a calibration gradient matrix according to an exemplary embodiment.

[0030] Figure 6The figure is a flowchart of a method for determining a calibration template according to an exemplary embodiment.

[0031] Figure 7 is a schematic diagram showing a calibration template according to an exemplary embodiment.

[0032] Figure 8 The figure is a flowchart of an image shooting method according to an exemplary embodiment.

[0033] Figure 9 The figure is a block diagram of an image capturing device according to an exemplary embodiment.

[0034] Figure 10 The figure is a block diagram showing a device for capturing an image according to an exemplary embodiment. DETAILED DESCRIPTION

[0035] Exemplary embodiments are described in detail herein, with examples illustrated in the accompanying drawings. When the following description refers to the drawings, identical numerals in different drawings represent identical or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present disclosure.

[0036] With the development of science and technology and the advancement of electronic equipment technology, users' requirements for the shooting performance of electronic devices used in daily life are gradually increasing. In related technologies, the clarity of images taken by electronic devices can be improved by multi-frame super-resolution algorithms and autofocus (AutoFocus, AF) technology. Among them, the multi-frame super-resolution algorithm can use the complementary information of the pixels of adjacent frames to perform super-resolution reconstruction through the collected multi-frame images, thereby improving the clarity of the image quality. In autofocus technology, for non-fixed-focus sensors (Sensor), AF can use phase detection (PD) technology, time of flight measurement (TOF) technology and the clarity of the previous frame to determine whether the focus of the lens needs to be adjusted, so as to obtain a clear image.

[0037] However, in the related art, the multi-frame super-resolution algorithm performs multi-frame super-resolution reconstruction on the acquired image. Therefore, the multi-frame super-resolution algorithm is limited by the clarity of the input image. It can be understood that the clarity of the image acquired based on the multi-frame super-resolution algorithm is determined by the clarity of the input multi-frame algorithm image. If the clarity of the input image is poor, a clearer image cannot be obtained through the multi-frame super-resolution algorithm. On the other hand, the autofocus technology obtains a local focus image by moving the lens, thereby obtaining a clear image at the focus point. However, the autofocus technology cannot obtain a globally clear image. For example, when an electronic device shoots an image, the image to be shot includes a relatively close person and a relatively distant building. The autofocus technology can only adjust the lens of the shooting device so that the lens focus is on the person or on the building. Therefore, the autofocus technology can only obtain a locally clear image, but cannot obtain a globally clear image.

[0038] Therefore, the present disclosure provides an image capture method that obtains the distance between a camera and the subject to be captured, matches the obtained distance with a calibration gradient matrix, and thereby determines the position information of a photosensitive element in the camera. The photosensitive element is then adjusted based on the determined position information. The image is then captured based on the adjusted photosensitive element, thereby obtaining a globally clear image and providing users with a higher-definition shooting experience.

[0039] Figure 1 FIG. 1 is a flow chart showing an image capturing method according to an exemplary embodiment. Figure 1 As shown, the image capturing method is used in an electronic device and includes the following steps.

[0040] In step S11 , in response to detecting that a photographing device in the electronic device is turned on, the distance between the photographing device and the object to be photographed is acquired.

[0041] In the embodiment of the present disclosure, the distance to the object to be photographed can be acquired by a TOF sensor in the photographing device.

[0042] The TOF sensor may be a pixel-level TOF sensor, and the distance information of each pixel in the object to be photographed may be acquired based on the pixel-level TOF sensor.

[0043] In step S12 , the position of the photosensitive element in the camera is adjusted based on the distance and the calibration gradient matrix.

[0044] The calibration gradient matrix is ​​used to characterize the correspondence between the distance and the adjustment position of the photosensitive element.

[0045] In the embodiment of the present disclosure, the position of the photosensitive element corresponding to the distance can be obtained by matching the obtained distance with the calibration gradient matrix.

[0046] In the embodiment of the present disclosure, the adjusted position of the photosensitive element may be a position of the photosensitive element that can capture a clear image.

[0047] In the embodiment of the present disclosure, the photosensitive element may be a pixel-level sensor, for example, a pixel point sensor. In one example, the photosensitive element in the electronic device may be a dual photodiode (Dual PD). Figure 2 FIG. 1 is a schematic diagram showing the structure of a Dual PD according to an exemplary embodiment. Figure 2 As shown, the two photodiodes of the Dual PD are located at different positions at the bottom of the same pixel. By receiving light signals from different directions and determining the focal length based on the intensity and time difference of the light signals, the Dual PD determines the position at which a clear image can be obtained. It should be understood that the embodiments of this disclosure assume that the refractive power of the Dual PD light-transmitting lens is perfect and does not produce scattering or other phenomena.

[0048] In the embodiment of the present disclosure, the distances between the pixels of the object to be photographed are obtained and matched with the calibration gradient matrix, so that the positions of the photosensitive elements corresponding to the pixels can be obtained.

[0049] In step S13 , an image is captured based on the photosensitive element after the position adjustment.

[0050] In the disclosed embodiment, the photosensitive element is adjusted to a position determined based on the calibration gradient matrix to capture an image. The image may be captured by adjusting the position of each photosensitive element corresponding to each pixel based on the position of the photosensitive element corresponding to each pixel.

[0051] In the embodiment of the present disclosure, by obtaining the distance between the shooting device and the object to be photographed, the obtained distance is matched with the calibration gradient matrix to determine the position information of the photosensitive element in the shooting device, and the photosensitive element is adjusted based on the determined position information. The image is captured based on the adjusted photosensitive element, thereby obtaining a globally clear image, bringing a higher-definition shooting experience to the user.

[0052] Figure 3 FIG. 1 is a flow chart showing a method for determining a calibration gradient matrix according to an exemplary embodiment. Figure 3 As shown, the calibration gradient matrix determination method is used in an electronic device and includes the following steps.

[0053] In step S21 , based on the focus range of the camera device, the focus range is divided into a plurality of equally divided points, and each equally divided point is determined as a target distance.

[0054] In the embodiment of the present disclosure, the focus range of the camera device may be a range from the closest point to the farthest point at which the camera device can focus.

[0055] In the embodiment of the present disclosure, the focus range of the camera device is divided, for example, the focus range of the camera device can be equally divided, so as to determine multiple target distances.

[0056] In step S22, the photographed object is moved to the i-th target distance, and the photosensitive element in the photographing device is moved until the brightness value obtained by the photosensitive element at the corresponding pixel position of the calibration template is equal to the preset brightness threshold, and the first corresponding relationship between the position of the photosensitive element and the i-th target distance is recorded.

[0057] The calibration template is determined based on the number of pixels of an image captured by the camera device and includes multiple pixel positions.

[0058] In the disclosed embodiment, the subject can be moved to a target distance. When the subject is determined to be at the target distance, the photosensitive element in the camera device is moved so that the brightness value obtained in the calibration template is equal to the preset brightness value. A first correspondence between the current position of the photosensitive element and the target distance is recorded. It is understood that for the same subject at the same target distance, the positions of the photosensitive element corresponding to different pixels of the subject may be different.

[0059] In the implementation of the present disclosure, the focus position, that is, the position of the photosensitive element, can be determined by the phase focusing technology of the sensor. The principle of the phase focusing technology can be understood as determining the focus position by finding the point with the smallest phase difference. Figure 4 FIG. 1 is a schematic diagram showing a phase focusing principle according to an exemplary embodiment. Figure 4 As shown, assuming that the same pixel is mapped to the sensor through different light rays, there will be a corresponding relationship between P1, P2 and d1, and P1', P2' and d2 similar triangles. Among them, by receiving the position relationship of P1, P2, P1', P2' with the original pixel light, it can be determined whether P1, P2 and P1', P2' are located before or after the focal length. Among them, P1, P2 and d1 can be the data designed by the sensor at the factory. Therefore, based on the position relationship of the PD point, the distance between P2' and P1' is calculated, and the distance of d2 can be calculated based on the similar triangles, thereby determining the focus position.

[0060] In step S23, the photographed object is moved to the (i+1)th target distance, and the above process is repeated until a first corresponding relationship between the target distance corresponding to each of the multiple equally divided points and the position of the photosensitive element is obtained.

[0061] In the disclosed embodiment, the subject is moved to another target distance, for example, the next target distance. When the subject is at the next target distance, the photosensitive element in the camera device is re-moved so that the brightness value obtained in the calibration template is equal to the preset brightness value. The first correspondence between the current position of the photosensitive element and the target distance is recorded. It is understood that the brightness values ​​corresponding to different pixels at different target distances may be different, and the brightness values ​​can be determined by relevant technicians.

[0062] In step S24 , a calibration gradient matrix is ​​determined based on a first correspondence between a target distance corresponding to each of the plurality of equally divided points and a position of the photosensitive element.

[0063] In the embodiment of the present disclosure, a first correspondence between the target distance divided based on the focus range and the position of the photosensitive element is obtained, so that a calibration gradient matrix can be determined.

[0064] In the disclosed embodiments, the focus range of the camera device is divided to determine multiple target distances, and a calibration gradient matrix is ​​determined by obtaining a first correspondence between the multiple target distances and the position of the photosensitive element. By determining the calibration gradient matrix, the position of the photosensitive element can be quickly determined when the camera device captures an image, resulting in a globally clear image and an improved user experience.

[0065] In the implementation of the present disclosure, since the target distance cannot represent the entire focus range, it is necessary to fit the first correspondence between the target distance and the position of the photosensitive element to determine the correspondence between all distances in the focus range and the photosensitive element.

[0066] Figure 5 FIG. 1 is a flow chart showing a method for determining a calibration gradient matrix according to an exemplary embodiment. Figure 5 As shown, the calibration gradient matrix determination method is used in an electronic device and includes the following steps.

[0067] In step S31 , based on the focus range of the camera device, the focus range is divided into a plurality of equally divided points, and each equally divided point is determined as a target distance.

[0068] In step S32, the photographed object is moved to the i-th target distance, and the photosensitive element in the photographing device is moved until the brightness value obtained by the photosensitive element at the corresponding pixel position of the calibration template is equal to the preset brightness threshold, and the first corresponding relationship between the position of the photosensitive element and the i-th target distance is recorded.

[0069] In step S33, the photographed object is moved to the (i+1)th target distance, and the above process is repeated until a first corresponding relationship between the target distance corresponding to each of the multiple equally divided points and the position of the photosensitive element is obtained.

[0070] In step S34, the first correspondence between the target distance corresponding to each of the multiple equally divided points and the position of the photosensitive element is fitted to obtain the second correspondence between the corresponding focus distance within the focus range of the shooting device and the position of the photosensitive element, and the second correspondence is used as the calibration gradient matrix.

[0071] In the embodiment of the present disclosure, step S31, step S32, and step S33 are consistent with step S21, step S22, and step S23, and will not be described in detail here.

[0072] In an embodiment of the present disclosure, based on the first correspondence between the target distance and the position of the photosensitive element that has been obtained, the first relationship is fitted to obtain a second correspondence between all focus distances within the focus range and the position of the photosensitive element, and the second relationship is used as a calibration gradient matrix.

[0073] In one example, a one-dimensional vector can be constructed for the movement position of each photosensitive element. However, it is understandable that this one-dimensional vector is not continuous. Therefore, to avoid the possibility that the distance of the object to be photographed is not within the target distance, the constructed one-dimensional vector data is converted into a two-dimensional function template. This facilitates the determination of the movement position of the photosensitive element at different distances. A two-dimensional function template is constructed for all positions, ultimately resulting in a constructed gradient matrix.

[0074] In the embodiment of the present disclosure, by fitting the first correspondence between the acquired target distance and the position of the photosensitive element, a second correspondence between all focus distances in the focus range and the position of the photosensitive element is obtained, and the second correspondence is used as a calibration gradient matrix, so that the calibration gradient matrix can include the correspondence between all focus distances in the shooting device and the photosensitive element, and thus when the camera device shoots an image, the position of the photosensitive element can be accurately and more comprehensively determined, so that the captured image can be clearer.

[0075] In the embodiments of the present disclosure, different camera devices can capture images of different sizes, or the same camera device can capture images of different sizes. Furthermore, images of different sizes may contain different numbers of pixels. It is understood that a photosensitive element can be used to receive light signals corresponding to pixels, and therefore, different photosensitive elements may correspond to images of different sizes.

[0076] In the disclosed embodiment, the calibration template can be used to obtain the brightness values ​​of different photosensitive elements. Different camera devices may have different calibration templates. Furthermore, the calibration template can be determined based on the size of the image that the camera device can capture.

[0077] Figure 6 FIG. 1 is a flow chart showing a method for determining a calibration template according to an exemplary embodiment. Figure 6 As shown, the method for determining a calibration template is used in an electronic device and includes the following steps.

[0078] In step S41, based on the width of the image captured by the camera and the number of pixels included in the row direction of the preset target template, X target templates included in the width of the image captured by the camera are determined, where X is the value obtained by rounding down the ratio of the width to the number of pixels included in the row direction.

[0079] In the embodiment of the present disclosure, the target template may be a matrix of a preset size.

[0080] In the disclosed embodiment, the number of pixels contained in the width of the image that the camera can capture can be determined, and the number of pixels contained in the row direction of the target template can be determined, thereby determining that the image that the camera can capture contains X target templates in the width. Where X is the ratio of the width to the number of pixels contained in the row direction, rounded down.

[0081] In step S42, based on the height of the image captured by the shooting device and the number of pixels included in the column direction of the preset target template, the Y target templates included in the height of the image captured by the shooting device are determined, where Y is the value obtained by rounding down the ratio of the height to the number of pixels included in the column direction.

[0082] In the disclosed embodiment, the number of pixels contained in the height of the image that the camera can capture can be determined, and the number of pixels contained in the column direction of the target template can be determined, thereby determining that the image that the camera can capture contains Y target templates in height. Where Y is the value obtained by rounding down the ratio of the height to the number of pixels contained in the column direction.

[0083] In step S43, X*Y target templates are used as calibration templates.

[0084] In the embodiment of the present disclosure, the product of X and Y may be determined as the number of target templates included in the calibration template.

[0085] In the embodiment of the present disclosure, the target template can be set as an m*n matrix. For example, each pixel in the image to be captured needs to have its own identity information, and the brightness value is generally 0-255, so a 256*256 matrix can be established as the target matrix.

[0086] In one example, Figure 7 FIG. 1 is a schematic diagram showing a calibration template according to an exemplary embodiment. Figure 7 As shown in , if the size of the image captured by the camera is determined to be M*N, where M and N represent the number of pixels in the width and height directions of the image respectively. For example, the size of the image can be 512*512. And the target template is determined to be a 256*256 matrix, then the calibration template contains 4 target templates, and the arrangement of the four target templates is as follows Figure 7 shown.

[0087] In the embodiment of the present disclosure, a calibration template is determined based on the images that can be captured by the shooting device, so that a calibration template that meets the functional requirements of the shooting device can be obtained, and then a gradient calibration matrix can be obtained based on the determined calibration template to obtain a globally clear captured image, thereby improving the user experience.

[0088] In the embodiment of the present disclosure, the position of the photosensitive element can be adjusted by a motor, for example, the position of the photosensitive element can be adjusted by a voice coil motor.

[0089] In the disclosed embodiment, the position of each photosensitive element is adjusted by a voice coil motor, thereby achieving multi-angle shooting of images and improving the clarity of the captured images.

[0090] In the embodiment of the present disclosure, a pixel-level distance sensor is used to obtain the distance between the shooting device and each pixel point in the object to be photographed, so that the position of the photosensitive element corresponding to the pixel point can be determined according to the distance of each pixel point, and then adjusted to achieve the capture of a global clear image and enhance the user's shooting experience.

[0091] In the embodiments of the present disclosure, the image capturing method is described with reference to the following examples.

[0092] In the embodiment of the present disclosure, Figure 8 FIG. 1 is a flowchart of an image shooting method according to an exemplary embodiment. Figure 8As shown, the calibration module, which includes template construction and multi-dimensional calibration, calibration gradient matrix construction, pixel-level TOF distance estimation, and pixel-level sensor adjustment, generates a single-frame globally clear image. When the sensor is powered on (i.e., when the user turns on the camera in the electronic device), the module constructs a calibration template that matches the image size that the camera can capture. The calibration template can include multiple preset target templates. The target template can be a matrix determined based on brightness values, for example, a 256*256 matrix. After determining the calibration template, the focus range can be divided into equal parts based on the maximum and minimum supported focus ranges of the camera. For example, a 1-meter range can be divided into N equal parts. At each distance, the constructed template is used to adjust the distance of each sensor pixel to ensure that each pixel achieves the same brightness value as the template. Each pixel has left and right PD points. Based on the principle of light transmission, the same pixel receives the same brightness value if the pixel sensor is in the focal length imaging position when different light rays pass through it. Therefore, based on the calibration template, the focal length can be easily calculated, and then the position of the pixel sensor can be appropriately adjusted to make the entire image appear clear. In addition, since the imaging distance presented by the object at different positions is different, it is necessary to simulate at multiple divided distances.

[0093] Furthermore, a correspondence between the distance of the entire focusing range of the camera device and the position of the pixel sensor can be established based on the positions of the pixel sensors at each equally divided point. First, a one-dimensional vector is constructed based on the position of each equally divided point and the movement position of the pixel sensor, but this one-dimensional vector is not continuous. To avoid the situation where the distance of the object is not within the simulated division distance, a two-dimensional function template is constructed from the constructed one-dimensional vector data to facilitate the determination of the movement position of the pixel sensor at different distances. A two-dimensional function template is constructed for all positions, and finally a constructed gradient matrix is ​​obtained.

[0094] In the embodiment of the present disclosure, the above process can be understood as a process of constructing a calibration gradient matrix, and the above process can be completed before the electronic device leaves the factory, so that the calibration module parameters can be quickly obtained when the electronic device is used to calculate the moving distance of the pixel point sensor.

[0095] In the embodiment of the present disclosure, when the user turns on a camera, such as a camera, the pixel-level TOF sensor emits infrared light to obtain the distance information of each pixel in the scene and record the distance information of the pixel in real time. The distance information is obtained based on the pixel-level TOF, and the position of the sensor pixel that needs to be moved is calculated using the constructed gradient matrix, and the position information of the sensor is adjusted at the pixel level, so that each pixel can be clearly presented. After adjusting the sensor position, based on the pixel information obtained by DualPD, the left and right PD data are added together to obtain the brightness value of the pixel at this position. The Sensor sets the exposure parameters according to the exposure information and completes the output of the raw data.

[0096] In the disclosed embodiment, the distance information from the camera to each pixel in the shooting scene is obtained, and the position where the sensor pixel needs to be moved is calculated based on the constructed gradient matrix, thereby adjusting the position of each pixel sensor, so that each pixel can be clearly presented, bringing the user a higher-definition shooting experience.

[0097] Based on the same concept, an embodiment of the present disclosure further provides an image capturing device.

[0098] It is understandable that the image capture device provided by the embodiments of the present disclosure includes hardware structures and / or software modules corresponding to the execution of each function in order to realize the above functions. In combination with the units and algorithm steps of the various examples disclosed in the embodiments of the present disclosure, the embodiments of the present disclosure can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the technical solution of the embodiments of the present disclosure.

[0099] Figure 9 FIG. 1 is a block diagram of an image capturing device according to an exemplary embodiment. Figure 9 The device 100 includes an acquiring unit 101 , an adjusting unit 102 and a shooting unit 103 .

[0100] The acquiring unit 101 is configured to acquire the distance between the camera and an object to be photographed in response to detecting that a camera in the electronic device is turned on, wherein the object to be photographed is within a focus range of the camera.

[0101] The adjustment unit 102 is used to adjust the position of the photosensitive element in the shooting device based on the distance and the calibration gradient matrix, wherein the calibration gradient matrix is ​​used to represent the corresponding relationship between the distance and the adjusted position of the photosensitive element.

[0102] The photographing unit 103 is configured to photograph an image based on the photosensitive element after adjusting its position.

[0103] In one embodiment, the adjustment unit 102 determines the calibration gradient matrix in the following manner: based on the focus range of the camera device, the focus range is divided into a plurality of equally divided points, and each equally divided point is determined as the target distance. The photographed object is moved to the i-th target distance, and the photosensitive element in the photographing device is moved until the brightness value obtained by the photosensitive element at the corresponding pixel position of the calibration template is equal to the preset brightness threshold, and the first correspondence between the position of the photosensitive element and the i-th target distance is recorded. The calibration template is determined based on the number of pixels of the image captured by the camera device and includes a plurality of pixel positions. The object to be photographed is moved to the i+1-th target distance, and the above process is repeated until the first correspondence between the target distance corresponding to each of the plurality of equally divided points and the position of the photosensitive element is obtained. Based on the first correspondence between the target distance corresponding to each of the plurality of equally divided points and the position of the photosensitive element, the calibration gradient matrix is ​​determined.

[0104] In one embodiment, the adjustment unit 102 determines the calibration gradient matrix based on the first correspondence between the target distance corresponding to each of the multiple equally divided points and the position of the photosensitive element in the following manner: the first correspondence between the target distance corresponding to each of the multiple equally divided points and the position of the photosensitive element is fitted to obtain a second correspondence between the corresponding focus distance within the focus range of the shooting device and the position of the photosensitive element, and the second correspondence is used as the calibration gradient matrix.

[0105] In one embodiment, the calibration template is determined based on the number of pixels in the image captured by the camera device in the following manner: based on the width of the image captured by the camera device and the number of pixels included in the row direction of the preset target template, X target templates included in the width of the image captured by the camera device are determined, where X is the value obtained by rounding down the ratio of the width to the number of pixels included in the row direction. Based on the height of the image captured by the camera device and the number of pixels included in the column direction of the preset target template, Y target templates included in the height of the image captured by the camera device are determined, where Y is the value obtained by rounding down the ratio of the height to the number of pixels included in the column direction. The X*Y target templates are used as the calibration template.

[0106] In one embodiment, the position of a photosensitive element in a shooting device is adjusted based on a voice coil motor, and the photosensitive element includes a plurality of photosensitive elements.

[0107] In one embodiment, the acquiring unit 101 acquires the distance between the photographing device and the object to be photographed in the following manner: based on a pixel-level distance sensor, the distance between the photographing device and each pixel point of the object to be photographed is acquired.

[0108] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0109] Figure 10 FIG2 is a block diagram of an apparatus 200 for capturing an image according to an exemplary embodiment. For example, the apparatus 200 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0110] Reference Figure 10 , apparatus 200 may include one or more of the following components: a processing component 202 , a memory 204 , a power component 206 , a multimedia component 208 , an audio component 210 , an input / output (I / O) interface 212 , a sensor component 214 , and a communication component 216 .

[0111] The processing component 202 generally controls the overall operation of the device 200, such as operations associated with display, phone calls, data communications, camera operation, and recording operations. The processing component 202 may include one or more processors 220 to execute instructions to perform all or part of the steps of the above-described method. In addition, the processing component 202 may include one or more modules to facilitate interaction between the processing component 202 and other components. For example, the processing component 202 may include a multimedia module to facilitate interaction between the multimedia component 208 and the processing component 202.

[0112] The memory 204 is configured to store various types of data to support operations on the device 200. Examples of such data include instructions for any application or method operating on the device 200, contact data, phone book data, messages, pictures, videos, etc. The memory 204 can be implemented by any type of volatile or non-volatile storage device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0113] The power component 206 provides power to the various components of the device 200. The power component 206 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device 200.

[0114] The multimedia component 208 includes a screen that provides an output interface between the device 200 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, slides, and gestures on the touch panel. The touch sensor can not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 208 includes a front camera and / or a rear camera. When the device 200 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have a focal length and optical zoom capability.

[0115] The audio component 210 is configured to output and / or input audio signals. For example, the audio component 210 includes a microphone (MIC) that is configured to receive external audio signals when the device 200 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals may be further stored in the memory 204 or transmitted via the communication component 216. In some embodiments, the audio component 210 further includes a speaker for outputting audio signals.

[0116] I / O interface 212 provides an interface between processing component 202 and peripheral interface modules, such as a keyboard, click wheel, buttons, etc. These buttons may include but are not limited to: a home button, volume buttons, a start button, and a lock button.

[0117] The sensor assembly 214 includes one or more sensors for providing various aspects of the status assessment of the device 200. For example, the sensor assembly 214 can detect the open / closed state of the device 200, the relative positioning of components, such as the display and keypad of the device 200. The sensor assembly 214 can also detect changes in the position of the device 200 or a component of the device 200, the presence or absence of user contact with the device 200, the orientation or acceleration / deceleration of the device 200, and temperature changes of the device 200. The sensor assembly 214 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 214 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 214 may also include an accelerometer, a gyroscope, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0118] The communication component 216 is configured to facilitate wired or wireless communication between the device 200 and other devices. The device 200 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 216 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 216 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0119] In an exemplary embodiment, the apparatus 200 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above-described method.

[0120] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as the memory 204 including instructions, which can be executed by the processor 220 of the apparatus 200 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0121] It is understood that in this disclosure, "plurality" refers to two or more than two, and other quantifiers are similar. "And / or" describes the association relationship of related objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the related objects before and after are in an "or" relationship. The singular forms "a", "the" and "the" are also intended to include the plural forms, unless the context clearly indicates otherwise.

[0122] It will be further understood that the terms "first," "second," and the like are used to describe various types of information, but such information should not be limited to these terms. These terms are used solely to distinguish information of the same type from one another and do not indicate a particular order or level of importance. In fact, the terms "first," "second," and the like are fully interchangeable. For example, first information could be referred to as second information, and similarly, second information could be referred to as first information without departing from the scope of this disclosure.

[0123] It can be further understood that the terms "center", "longitudinal", "lateral", "front", "back", "up", "down", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the accompanying drawings, and are only for the convenience of describing this embodiment and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation.

[0124] It is further understood that, unless otherwise specified, “connection” includes a direct connection where there are no other components between the two elements, and also includes an indirect connection where there are other elements between the two elements.

[0125] It is further understood that although operations are described in a particular order in the drawings in the embodiments of the present disclosure, this should not be construed as requiring that the operations be performed in the particular order shown or in a serial order, or that all of the operations shown be performed to obtain the desired results. In certain circumstances, multitasking and parallel processing may be advantageous.

[0126] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein.

[0127] It should be understood that the present disclosure is not limited to the exact structures described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the scope of the appended claims.

Claims

1. An image capturing method, characterized in that: include: In response to detecting that a camera in the electronic device is turned on, obtaining a distance between the camera and an object to be photographed; Adjusting the position of the photosensitive element in the camera device based on the distance and a calibration gradient matrix, wherein the calibration gradient matrix is ​​used to represent the corresponding relationship between the distance and the adjusted position of the photosensitive element; The image is captured based on the photosensitive element after adjusting its position.

2. The method according to claim 1, characterized in that The calibration gradient matrix is ​​determined in the following manner: Based on the focus range of the camera device, the focus range is divided into a plurality of equally divided points, and each equally divided point is determined to be a target distance; Move the photographed object to the i-th target distance, and move the photosensitive element in the photographing device until the brightness value obtained by the photosensitive element at the corresponding pixel position of the calibration template is equal to the preset brightness threshold, and record a first correspondence between the position of the photosensitive element and the i-th target distance; Wherein, the calibration template is determined based on the number of pixels of the image captured by the camera device and includes a plurality of pixel positions; Moving the photographed object to the (i+1)th target distance, and repeating the above process until a first corresponding relationship between the target distance corresponding to each of the multiple equally divided points and the position of the photosensitive element is obtained; The calibration gradient matrix is ​​determined based on a first correspondence between a target distance corresponding to each of the plurality of equally divided points and a position of a photosensitive element.

3. The method according to claim 2, characterized in that The step of determining the calibration gradient matrix based on a first correspondence between a target distance corresponding to each of the plurality of equally divided points and a position of a photosensitive element comprises: The first correspondence between the target distance corresponding to each of the multiple equally divided points and the position of the photosensitive element is fitted to obtain the second correspondence between the focus distance corresponding to the focus range of the shooting device and the position of the photosensitive element, and the second correspondence is used as the calibration gradient matrix.

4. The method according to claim 2, characterized in that The calibration template is determined based on the number of pixels of the image captured by the camera device in the following manner: Determining, based on the width of the image captured by the camera and the number of pixels included in the row direction of the preset target template, X target templates included in the width of the image captured by the camera, where X is a value obtained by rounding down the ratio of the width to the number of pixels included in the row direction; Determining, based on the height of the image captured by the camera and the number of pixels included in the column direction of the preset target template, Y target templates included in the height of the image captured by the camera, where Y is a value obtained by rounding down the ratio of the height to the number of pixels included in the column direction; X*Y target templates are used as the calibration templates.

5. The image capturing method according to claim 1, wherein: The position of the photosensitive element in the shooting device is adjusted based on a voice coil motor, and the photosensitive element includes multiple photosensitive elements.

6. The image capturing method according to claim 1, wherein: The obtaining of the distance between the photographing device and the object to be photographed includes: Based on the pixel-level distance sensor, the distance between the shooting device and each pixel point of the object to be shot is obtained.

7. An image capturing device, characterized in that: include: an acquiring unit, configured to acquire a distance between a photographing device in the electronic device and an object to be photographed in response to detecting that the photographing device in the electronic device is turned on; an adjusting unit, configured to adjust a position of a photosensitive element in the photographing device based on the distance and a calibration gradient matrix, wherein the calibration gradient matrix is ​​used to represent a correspondence between the distance and the adjusted position of the photosensitive element; The shooting unit is used to shoot an image based on the photosensitive element after adjusting its position.

8. The device according to claim 7, characterized in that The adjustment unit determines the calibration gradient matrix in the following manner: Based on the focus range of the camera device, the focus range is divided into a plurality of equally divided points, and each equally divided point is determined to be a target distance; Move the photographed object to the i-th target distance, and move the photosensitive element in the photographing device until the brightness value obtained by the photosensitive element at the corresponding pixel position of the calibration template is equal to the preset brightness threshold, and record a first correspondence between the position of the photosensitive element and the i-th target distance; Wherein, the calibration template is determined based on the number of pixels of the image captured by the camera device and includes a plurality of pixel positions; Moving the photographed object to the (i+1)th target distance, and repeating the above process until a first corresponding relationship between the target distance corresponding to each of the multiple equally divided points and the position of the photosensitive element is obtained; The calibration gradient matrix is ​​determined based on a first correspondence between a target distance corresponding to each of the plurality of equally divided points and a position of a photosensitive element.

9. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to execute the image capturing method according to any one of claims 1 to 6.

10. A storage medium, characterized in that: The storage medium stores instructions, and when the instructions in the storage medium are executed by a processor of the terminal, the terminal is enabled to execute the method according to any one of claims 1 to 6.

Images