Camera module distortion determination method, distortion correction method, and depth measurement device

Abstract

The application discloses a camera module distortion determination method, a distortion correction method and a depth measurement device. The scheme establishes a mapping relationship between a distortion value of a module and a field angle based on a grating equation. Based on the established mapping relationship, the distortion value of the module can be quickly obtained only by knowing the field angle of the module, so that further design is made according to the distortion value of the module, the distortion value of a receiving module and the distortion value of a projecting module are offset to each other, distortion correction is realized, and the distortion of the whole depth measurement device can be reduced, and the influence of the distortion on the depth measurement precision can be effectively avoided.

Description

Technical Field

[0001] This application relates to the field of optical imaging technology, and in particular to a method for determining camera module distortion, a distortion correction method, and a depth measurement device. Background Technology

[0002] In recent years, 3D imaging has been increasingly used in the consumer electronics field, such as in 3D scanning, facial recognition payment, and scene modeling. 3D imaging technology can not only image a target object but also acquire its depth information. Structured light depth cameras or Time-of-Flight (TOF) depth cameras are currently the most widely used 3D imaging devices.

[0003] In the field of 3D depth measurement, image distortion is one of the most important factors affecting the accuracy of 3D depth measurement, especially in areas with large field of view or at the edges, where the depth accuracy problem caused by image distortion is particularly serious. Image distortion mainly originates from the projection module and the receiving module of the depth measurement device. The projection module typically projects a distorted speckle pattern, which, when reflected from the object onto the receiving module, still forms a distorted speckle pattern, thus affecting the measurement accuracy of the depth measurement device.

[0004] Application content

[0005] To reduce image distortion, this application provides a distortion correction method and a depth measurement device.

[0006] According to one aspect of the embodiments of this application, a distortion correction method is disclosed, comprising:

[0007] Obtain the field of view of the first module;

[0008] The field of view of the first module is mapped to obtain the first distortion value of the first module;

[0009] A corrected distortion value is determined based on the first distortion value, and the corrected distortion value cancels out the first distortion value.

[0010] The second module is selected with the distortion value being the corrected distortion value; one of the second module and the first module is a projection module for projecting the speckle pattern, and the other is a receiving module for receiving the speckle pattern.

[0011] In one exemplary embodiment, one of the second module and the first module has a pincushion distortion type, and the other has a barrel distortion type.

[0012] In one exemplary embodiment, the step of mapping the field of view of the first module to obtain a first distortion value of the first module includes:

[0013] The first field of view angle of the first module in the first direction and the second field of view angle in the second direction are obtained, wherein the first direction and the second direction are two coordinate axes in Cartesian coordinates;

[0014] Using half of the first field of view as the incident angle and zero as the diffraction angle, a first grating equation is constructed; and using half of the second field of view as the incident angle and zero as the diffraction angle, a second grating equation is constructed.

[0015] Based on any point in space, construct the third grating equation corresponding to the first direction and the fourth grating equation corresponding to the second direction in spherical coordinates respectively;

[0016] Based on the first grating equation, the second grating equation, the third grating equation, and the fourth grating equation, a first mapping relationship between the coordinates of the speckle spots in the first direction and the first field of view and the second field of view, and a second mapping relationship between the coordinates in the second direction and the first field of view and the second field of view, are obtained under the distortion influence of the first module.

[0017] Based on the first mapping relationship and the second mapping relationship, a third mapping relationship between the field of view of the first module and the first distortion value is determined, and the first distortion value of the first module is obtained.

[0018] In an exemplary embodiment, determining a third mapping relationship between the field of view of the first module and the first distortion value based on the first mapping relationship and the second mapping relationship, and obtaining the first distortion value of the first module, includes:

[0019] Obtain a speckle pattern under the distortion effect of the first module, and take two speckle spots located at the edge of the speckle pattern, the two speckle spots being located on the same edge of the speckle pattern;

[0020] Obtain the quotient of the coordinate values ​​of the two speckle points in the first direction or the second direction;

[0021] The quotient is used as the first distortion value of the first module.

[0022] In one exemplary embodiment, the third mapping relationship is:

[0023]

[0024] Wherein, α represents the first distortion value, FOVX_A represents the field of view of speckle A in the x direction, FOVY_A represents the field of view of speckle A in the y direction, FOVY_D represents the field of view of speckle D in the y direction, and the field of view of the first module includes the field of view in the x direction and the field of view in the y direction.

[0025] According to one aspect of the embodiments of this application, a depth measuring device is disclosed, the depth measuring device comprising:

[0026] The distortion value of the first module is obtained by mapping the field of view of the first module.

[0027] The second module has a distortion value that is a corrected distortion value, which cancels out the distortion value of the first module.

[0028] One of the second module and the first module is used to project the speckle pattern, and the other is used to receive the speckle pattern.

[0029] In one exemplary embodiment, the depth measuring device includes a substrate, and the first module and the second module are disposed on the same surface of the substrate.

[0030] In one exemplary embodiment, the first module includes a light source, a collimating lens, and a diffractive optical element. The light source is used to emit a light beam, the collimating lens is used to collimate the light beam emitted by the light source, and the diffractive optical element is used to project a speckle pattern based on the collimated light beam. The second module includes an imaging lens, a filter element, and a photosensitive element. The imaging lens is used to converge light, the filter element is used to perform wavelength filtering on the light converged by the imaging lens, and the photosensitive element receives the wavelength-filtered light to image a speckle pattern.

[0031] The technical solutions provided by the embodiments of this application have at least the following beneficial effects:

[0032] The technical solution provided in this application maps the field of view of the projection module to obtain the distortion value of the projection module, or maps the field of view of the receiving module to obtain the distortion value of the receiving module; and by making the distortion value of the receiving module cancel out the distortion value of the projection module, distortion correction is achieved, thereby reducing the distortion of the entire depth measurement device and effectively avoiding the impact of distortion on the depth measurement accuracy.

[0033] To facilitate obtaining the distortion value of the camera module, this application also provides a method for determining camera module distortion, including:

[0034] The first field of view angle of the camera module in the first direction and the second field of view angle in the second direction are obtained, wherein the first direction and the second direction are two coordinate axes in Cartesian coordinates;

[0035] Using half of the first field of view as the incident angle and zero as the diffraction angle, a first grating equation is constructed; and using half of the second field of view as the incident angle and zero as the diffraction angle, a second grating equation is constructed.

[0036] Based on any point in space, construct the third grating equation corresponding to the first direction and the fourth grating equation corresponding to the second direction in spherical coordinates respectively;

[0037] Based on the first grating equation, the second grating equation, the third grating equation, and the fourth grating equation, a first mapping relationship between the coordinates of the speckle spots in the first direction and the first field of view and the second field of view in the speckle pattern under the distortion influence of the camera module is obtained, as well as a second mapping relationship between the coordinates in the second direction and the first field of view and the second field of view.

[0038] Based on the first mapping relationship and the second mapping relationship, a third mapping relationship between the field of view and the distortion value of the camera module is determined, and the distortion value of the camera module is obtained.

[0039] In one exemplary embodiment, the third mapping relationship is:

[0040]

[0041] Wherein, α represents the distortion value of the camera module, FOVX_A represents the field of view angle of speckle A in the x direction, FOVY_A represents the field of view angle of speckle A in the y direction, and FOVY_D represents the field of view angle of speckle D in the y direction.

[0042] The camera module distortion determination scheme provided in this application establishes a mapping relationship between the distortion value of the camera module and the field of view based on the grating equation. Based on the established mapping relationship, the distortion value of the camera module can be quickly obtained by knowing only the field of view of the camera module. Further design can then be carried out based on the distortion value of the camera module, such as selecting another module whose distortion value cancels out the distortion value of the camera module to obtain a depth measurement device with smaller pattern distortion.

[0043] Corresponding to the camera module distortion determination method, this application also provides an electronic device, the electronic device comprising:

[0044] The acquisition module is used to acquire the first field of view angle of the camera module in the first direction and the second field of view angle in the second direction, wherein the first direction and the second direction are two coordinate axes in Cartesian coordinates;

[0045] The first construction module is used to construct a first grating equation by taking half of the first field of view as the incident angle and zero as the diffraction angle, and to construct a second grating equation by taking half of the second field of view as the incident angle and zero as the diffraction angle.

[0046] The second construction module is used to construct, in spherical coordinates, a third grating equation corresponding to the first direction and a fourth grating equation corresponding to the second direction, based on any point in space.

[0047] The first processing module is used to obtain, based on the first grating equation, the second grating equation, the third grating equation, and the fourth grating equation, a first mapping relationship between the coordinates of the speckle spots in the speckle pattern in the first direction and the first field of view and the second field of view, and a second mapping relationship between the coordinates in the second direction and the first field of view and the second field of view, under the influence of the distortion of the camera module.

[0048] The second processing module is used to determine a third mapping relationship between the field of view and distortion value of the camera module based on the first mapping relationship and the second mapping relationship, and to obtain the distortion value of the camera module.

[0049] Corresponding to the camera module distortion determination method, this application also provides an electronic device, the electronic device comprising:

[0050] One or more processors;

[0051] A memory is used to store one or more programs, which, when executed by one or more processors, enable the intelligent query device to implement the aforementioned camera module distortion determination method.

[0052] According to one aspect of the embodiments of this application, a computer-readable storage medium is disclosed, the computer-readable storage medium storing computer-readable instructions, which, when executed by a computer's processor, cause the computer to perform the aforementioned camera module distortion determination method.

[0053] It should be understood that the above general description and the following detailed description are merely exemplary and do not limit this application. Attached Figure Description

[0054] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0055] Figure 1 This is a schematic diagram illustrating the definition of the field of view.

[0056] Figure 2This is a flowchart illustrating a distortion correction method according to an exemplary embodiment.

[0057] Figure 3 yes Figure 2 A detailed flowchart of step S102 in the corresponding embodiment.

[0058] Figure 4 yes Figure 3 A detailed flowchart of step S1025 in the corresponding embodiment.

[0059] Figure 5 It is a distorted image of a projection module according to an exemplary embodiment.

[0060] Figure 6 It is a distorted image received by a receiving module according to an exemplary embodiment.

[0061] Figure 7 This is a structural diagram of a depth measuring device shown in an exemplary embodiment.

[0062] Figure 8 This is a flowchart illustrating a method for determining camera module distortion according to an exemplary embodiment.

[0063] Figure 9 This is a block diagram illustrating an electronic device according to an exemplary embodiment.

[0064] Figure 10 This is a computer system architecture block diagram illustrating an electronic device for implementing embodiments of the present application, according to an exemplary embodiment.

[0065] The annotations in the attached figures are explained as follows:

[0066] 101. Substrate; 102. Light source; 103. Collimating lens; 104. Diffractive optical element; 105. Photosensitive element; 106. Filter element; 107. Imaging lens; 200. Electronic device; 201. Acquisition module; 202. First building module; 203. Second building module; 204. First processing module; 205. Second processing module; 301. CPU; 302. ROM; 303. Storage section; 304. RAM; 305. Bus; 306. I / O interface; 307. Input section; 308. Output section; 309. Communication section; 310. Driver; 311. Removable medium. Detailed Implementation

[0067] Although this application can be readily embodied in various forms of implementation, only some specific embodiments are shown in the accompanying drawings and will be described in detail in this specification. It is understood that this specification should be regarded as an exemplary illustration of the principles of this application and is not intended to limit the application to what is described herein.

[0068] Furthermore, the terms "comprising," "having," and any variations thereof used in the description of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or modules is not limited to the steps or modules listed, but may optionally include other steps or modules not listed, or may optionally include other steps or modules inherent to such processes, methods, products, or devices.

[0069] Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first," "second," or "third" may explicitly or implicitly include one or more features.

[0070] In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0071] It should be noted that in the embodiments of this application, the terms "exemplary" or "example" are used to indicate that they are examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplary" or "example" in the embodiments of this application should not be construed as being better or more advantageous than other embodiments or design schemes. Specifically, the use of terms such as "exemplary" or "example" is intended to present the relevant concepts in a specific manner.

[0072] The exemplary embodiments will now be described in detail. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the foregoing description.

[0073] First, let me explain some of the terms used in this application:

[0074] Field of View (FOV): The angle between the two edges of an optical instrument's lens, representing the maximum range through which the image of the target object can pass through the lens. The size of the FOV determines the field of view of the optical instrument; a larger FOV results in a wider field of view but a lower optical magnification. FOV includes the vertical field of view (e.g., ...). Figure 1 The included angle ω1 and the horizontal field of view (as shown) are shown in the figure. Figure 1 The included angle ω2 and the diagonal field of view are shown in the figure.

[0075] Pincushion Distortion: Also known as pincushion distortion, it is a phenomenon caused by the lens causing the image to "shrink" towards the center.

[0076] Barrel distortion, also known as barrel distortion, is a distortion phenomenon caused by the physical properties of the lens and the structure of the lens group, resulting in an image that appears as a barrel-shaped expansion.

[0077] A grating is an optical device consisting of a large number of parallel slits of equal width and spacing.

[0078] The grating equation is: D(sinα±sinβ)=mλ, which means that the spectral order m is a mixture of light of different wavelengths λ1, λ2, λ3... projected onto the grating at the same incident angle α. The interference maxima produced by each wavelength are located at different angular positions, that is, the diffracted light of different wavelengths exits at different diffraction angles β.

[0079] This application provides a method for determining camera module distortion, a distortion correction method, and a depth measurement device. The method for determining camera module distortion, the distortion correction method, and the depth measurement device provided in this application are specifically described through the following embodiments. First, the distortion correction method in this application embodiment is described.

[0080] This application first provides a distortion correction method. The distortion correction method includes:

[0081] Obtain the field of view of the first module;

[0082] The field of view of the first module is mapped to obtain the first distortion value of the first module;

[0083] The corrected distortion value is determined based on the first distortion value, and the corrected distortion value cancels out the first distortion value.

[0084] The distortion value is selected as the second module to correct the distortion value; one of the second module and the first module is a projection module for projecting the speckle pattern, and the other is a receiving module for receiving the speckle pattern.

[0085] The technical solution provided in this application maps the field of view of the projection module to obtain the distortion value of the projection module, or maps the field of view of the receiving module to obtain the distortion value of the receiving module; and by making the distortion value of the receiving module cancel out the distortion value of the projection module, distortion correction is achieved, thereby reducing the distortion of the entire depth measurement device and effectively avoiding the impact of distortion on the depth measurement accuracy.

[0086] The embodiments of this application will be further described in detail below with reference to the accompanying drawings.

[0087] See Figure 2 As shown, an exemplary embodiment of this application provides a distortion correction method including the following steps S101 to S104.

[0088] S101, Obtain the field of view of the projection module.

[0089] like Figure 1 As shown, the field of view includes a vertical field of view ω1 and a horizontal field of view ω2. For each projection module, the projection point of viewpoint O along the optical axis Z onto the bottom surface ABCD of the cone is G. OG is along the camera's optical axis Z, and its length is the field of view depth. The horizontal field of view ω2 is determined by the field of view depth and the width of the imaging plane (i.e., the width of the bottom surface ABCD of the cone), while the vertical field of view ω1 is determined by the field of view depth and the height of the imaging plane (i.e., the height of the bottom surface ABCD of the cone). Each projection module has a defined field of view.

[0090] S102, perform mapping processing on the field of view of the first module to obtain the first distortion value of the first module.

[0091] In one exemplary embodiment, such as Figure 3 As shown, step S102 includes steps S1021 to S1025.

[0092] S1021, obtain the first field of view angle of the first module in the first direction and the second field of view angle in the second direction, wherein the first direction and the second direction are two coordinate axes in Cartesian coordinates.

[0093] Understandably, the first field of view in the first direction and the second field of view in the second direction correspond to the horizontal field of view and the vertical field of view, respectively. Figure 1 Taking the angle shown as an example, the first direction is the x-axis direction in rectangular coordinates, and the second direction is the y-axis direction in rectangular coordinates.

[0094] S1022, using half of the first field of view as the incident angle and zero as the diffraction angle, construct the first grating equation; and using half of the second field of view as the incident angle and zero as the diffraction angle, construct the second grating equation.

[0095] In detail, the grating equation is D(sinα±sinβ)=mλ, where D represents the grating period, α represents the incident angle of light onto the grating, β represents the diffraction angle of light through the grating, the incident angle α and the diffraction angle β are the angles between the incident ray and the diffracted ray and the grating normal, respectively. If the incident angle α and the diffraction angle β are on the same side of the grating normal, a plus sign is taken inside the parentheses of the above grating equation; if the incident angle α and the diffraction angle β are on different sides of the grating normal, a minus sign is taken inside the parentheses of the above grating equation. m represents the diffraction order, and λ represents the wavelength projected onto the grating.

[0096] The first field of view (horizontal field of view) is denoted as FOVX, and the second field of view (vertical field of view) is denoted as FOVY. Then the equation for the first grating is given by equation (1), and the equation for the second grating is given by equation (2):

[0097] d x *sin(FOVX / 2)=mλ (1)

[0098] d y *sin(FOVY / 2)=nλ (2)

[0099] In the first grating equation, dx represents the grating period in the x-direction (first direction), m represents the diffraction order in the x-direction, and λ represents the wavelength projected onto the grating; in the second grating equation, dy represents the grating period in the y-direction (second direction), n represents the diffraction order in the y-direction, and λ represents the wavelength projected onto the grating.

[0100] S1023, based on any point in space, construct the third grating equation corresponding to the first direction and the fourth grating equation corresponding to the second direction in spherical coordinates.

[0101] In detail, suppose there is an arbitrary point H in space, the angle between the line connecting the origin to point H and the +Z axis is θ, and the angle between the projection of the line connecting the origin to point H onto the XY plane and the +X axis is φ. Then, for point H, according to the grating equation, the following equation also holds true.

[0102] d x *sinθcosφ=mλ (3)

[0103] d y *sinθsinφ=nλ (4)

[0104] Among them, equation (3) above is the third grating equation, and equation (4) above is the fourth grating equation.

[0105] In the third grating equation, dx represents the grating period in the x-direction (first direction), θ represents the spatial angle between the line connecting the origin to point H and the +Z axis, φ represents the angle between the projection of the line connecting the origin to point H onto the XY plane and the +X axis, m represents the diffraction order in the x-direction, and λ represents the wavelength projected onto the grating. In the fourth grating equation, dy represents the grating period in the y-direction (second direction), θ represents the spatial angle between the line connecting the origin to point H and the +Z axis, φ represents the angle between the projection of the line connecting the origin to point H onto the XY plane and the +X axis, n represents the diffraction order in the y-direction, and λ represents the wavelength projected onto the grating.

[0106] S1024, based on the first grating equation, the second grating equation, the third grating equation and the fourth grating equation, obtain the first mapping relationship between the coordinates of the speckle spots in the speckle pattern in the first direction and the first field of view and the second field of view, and the second mapping relationship between the coordinates in the second direction and the first field of view and the second field of view, under the influence of the distortion of the first module.

[0107] In detail, by combining the first grating equation, the second grating equation, the third grating equation, and the fourth grating equation (1)-(4) above, we can obtain:

[0108]

[0109] Based on the above formula (5) regarding the included angles θ and φ, assuming the projected distance is P, then the coordinates of a speckle point A in the speckle diagram can be expressed as (xa, ya).

[0110] xa=P tanθ a cosφ a

[0111] ya=P tanθ a sinφ a (6)

[0112] xa represents the coordinates of speckle point A in the first direction, and ya represents the coordinates of speckle point A in the second direction.

[0113] Substituting equation (5) into equation (6), we can obtain the first mapping relationship between the coordinates of point A in the first direction and the first and second field of view, as well as the second mapping relationship between the coordinates in the second direction and the first and second field of view.

[0114] S1025, determine the third mapping relationship between the field of view of the first module and the first distortion value based on the first mapping relationship and the second mapping relationship, and obtain the first distortion value of the first module.

[0115] In one exemplary embodiment, such as Figure 4 As shown, step S1025 includes steps S10251 to S10253.

[0116] S10251, Obtain the speckle map under the distortion effect of the first module, and take two speckle spots located at the edge of the speckle map. The two speckle spots are located on the same edge of the speckle map.

[0117] S10252, obtain the quotient of the coordinate values ​​of two speckles in the first or second direction.

[0118] S10253, the quotient is used as the first distortion value of the first module.

[0119] S103, determine the corrected distortion value based on the first distortion value, and the corrected distortion value cancels out the first distortion value.

[0120] In an exemplary embodiment, after the corrected distortion value cancels out the first distortion value, the speckle pattern received by the receiving module does not have any distortion.

[0121] Assuming a distortion value α = 1 indicates that the speckle pattern has no distortion, to ensure that the distortion value α = 1 in the speckle pattern received by the receiving module, the following must be satisfied:

[0122]

[0123] Where, α T α represents the first distortion value. R This represents the corrected distortion value, where 0 < α R A value less than 1 is called barrel distortion, such as... Figure 5 As shown. From the formula It can be seen that when the distortion value of the receiving module is the reciprocal of the distortion value of the projection module, the distortion produced by the projection module is just corrected by the distortion of the receiving module, and the entire optical system has the minimum pattern distortion.

[0124] S104, Select the receiving module with the distortion value as the correction distortion value. The projection module and the receiving module are each components of the depth measurement device.

[0125] A specific example:

[0126] The distortion type of the projection module is pincushion distortion, and the projected speckle pattern is as follows: Figure 5 As shown. The distortion type of the receiving module is barrel distortion. For a speckle pattern projected by the projection module that has no distortion, the speckle pattern received by the receiving module is as follows. Figure 6 As shown.

[0127] Please refer to the following: Figure 5 As shown, ABCEGFHD is the distorted profile of the speckle pattern projected by the projection module. Assuming the coordinates of point O are (0,0), point A are (xa,ya), and point D are (0,yd), based on the first grating equation, second grating equation, third grating equation, fourth grating equation, and the coordinate expression of the speckle spots in the speckle pattern (5) described above, the coordinates of point A are expressed as:

[0128] xa=P tanθ a cosφ a

[0129] ya=P tanθ a sinφ a (6)

[0130] For point D, the coordinates are represented as:

[0131] yd=P tanθ d sinφ d (7)

[0132] In step S1025, the quotient α = ya / yd of the y-axis coordinates of points A and D is defined as the first distortion value:

[0133] Substituting equation (5) into equation (6), and equation (5) into equation (7), and combining α = ya / yd, we get:

[0134]

[0135] Where α represents the first distortion value, FOVX_A and FOVY_A represent the field of view angles of point A in the x and y directions, respectively, and FOV_D represents the field of view angle of point D in the y direction. From the above description, it can be seen that the distortion of the speckle pattern projected by the projection module is only related to the field of view angle, and α > 1. Therefore, in the early stages of product design, once the field of view angle of the projection module is obtained, the magnitude of the pattern distortion of the projection module can be quickly calculated. After obtaining the distortion α of the projection module, in order to minimize the distortion of the speckle pattern received by the receiving module, the distortion of the projection module is compensated. A receiving module whose distortion cancels out the first distortion value α is selected, thereby improving the measurement accuracy of the depth measurement device.

[0136] Understandably, in the above embodiments, the pattern distortion of the depth measuring device is minimized when the first distortion value αT > 1 and 0 < the corrected distortion value αR < 1. In some embodiments, it can also be 0 < αT < 1 and αR > 1, which can also achieve the minimum pattern distortion of the depth measuring device.

[0137] Understandably, using the quotient of the coordinate values ​​of two speckle points in the first or second direction as the first distortion value of the first module is merely an exemplary embodiment. In other embodiments, the first distortion value can be defined in other ways, for example, by... Figure 5 The quotient of the length of line ac and the length of line de is used as the first distortion value; for example, the quotient of the length of line ad and the length of line bo is used as the first distortion value. It is understood that, and is not limited to, using the quotient between the two as the first distortion value, for example, using the difference between the two as the first distortion value, as long as the distortion value definition method of the projection module is compatible with the distortion value definition method of the receiving module.

[0138] In the above embodiments, the selection of the receiving module is based on the distortion value of the projection module. It can be understood that in other embodiments, the selection of the projection module may also be based on the distortion value of the receiving module.

[0139] Please refer to the following. Figure 7 , Figure 7 This is an exemplary structural diagram of a depth measuring device, such as... Figure 7As shown, the depth measuring device includes a projection module, a receiving module, and a substrate 101. The projection module and the receiving module are mounted on the same surface of the substrate 101 by means of surface mount.

[0140] In one exemplary embodiment, such as Figure 7 As shown, the projection module includes a light source 102, a collimating lens 103, and a diffractive optical element 104 (DOE). The light source 102 is used to emit a light beam, the collimating lens 103 is used to collimate the light beam emitted by the light source 102, and the diffractive optical element 104 is used to project a speckle pattern based on the collimated light beam. The receiving module includes an imaging lens 107, a filter element 106, and a photosensitive element 105. The imaging lens 107 is used to converge the light, the filter element 106 is used to filter the wavelength of the light converged by the imaging lens 107, and the photosensitive element 105 receives the wavelength-filtered light to image a speckle pattern.

[0141] In detail, the light source 102 can be an edge-emitting laser or a perpendicular-surface laser. The collimating lens 103 collimates the diverging light source and replicates and expands it through the diffractive optical element 104, projecting a speckle pattern with a specific field of view into the target space. After reflection from the object, the speckle pattern first passes through the imaging lens 107 to converge the light, and after wavelength filtering by the filtering element 106, the speckle pattern is received by the photosensitive element 105. As mentioned above, the speckle pattern projected by the projection module is projected through the diffractive optical element 104, which exhibits distortion. The distortion increases towards the edge of the pattern. After reflection from the object, when the pattern passes through the imaging lens 107, the distortion of the imaging lens 107 itself is superimposed on the distortion of the speckle pattern, resulting in severe distortion of the final image pattern on the photosensitive element 105, thus affecting the measurement accuracy of the depth measurement device.

[0142] Therefore, based on the distortion value of one module (projection module / receiving module), another module (receiving module / projection module) whose distortion value cancels out is selected to minimize the pattern distortion of the entire depth measurement device, thereby improving the measurement accuracy of the depth measurement device. The distortion value of the projection module is obtained by mapping the field of view of the projection module. The distortion value of the receiving module is the corrected distortion value, which cancels out the distortion value of the projection module.

[0143] Please refer to the following. Figure 8 An exemplary embodiment of this application provides a camera module distortion determination method, which includes the following steps S201 to S205.

[0144] S201, obtain the first field of view angle of the camera module in the first direction and the second field of view angle in the second direction, where the first direction and the second direction are two coordinate axes in Cartesian coordinates.

[0145] S202, using half of the first field of view as the incident angle and zero as the diffraction angle, construct the first grating equation; and using half of the second field of view as the incident angle and zero as the diffraction angle, construct the second grating equation.

[0146] S203, based on any point in space, construct the third grating equation corresponding to the first direction and the fourth grating equation corresponding to the second direction in spherical coordinates.

[0147] S204, based on the first grating equation, the second grating equation, the third grating equation and the fourth grating equation, obtain the first mapping relationship between the coordinates of the speckle spots in the speckle pattern in the first direction and the first field of view and the second field of view, and the second mapping relationship between the coordinates in the second direction and the first field of view and the second field of view, under the influence of the distortion of the camera module.

[0148] S205, determine the third mapping relationship between the field of view of the camera module and the distortion value based on the first mapping relationship and the second mapping relationship, and obtain the distortion value of the camera module.

[0149] Understandably, the camera module can be either the projection module or the receiving module described above.

[0150] For a more detailed explanation of the implementation process of each step in the above-mentioned camera module distortion determination method, please refer to the implementation process of the corresponding step in the above-mentioned distortion correction method, which will not be repeated here.

[0151] like Figure 9 As shown, corresponding to the camera module distortion determination method, the electronic device 200 provided in this application embodiment includes:

[0152] The acquisition module 201 is used to acquire the first field of view angle of the camera module in the first direction and the second field of view angle in the second direction, wherein the first direction and the second direction are two coordinate axes in Cartesian coordinates.

[0153] The first construction module 202 is used to construct a first grating equation by taking half of the first field of view as the incident angle and zero as the diffraction angle, and to construct a second grating equation by taking half of the second field of view as the incident angle and zero as the diffraction angle.

[0154] The second construction module 203 is used to construct, in spherical coordinates, a third grating equation corresponding to the first direction and a fourth grating equation corresponding to the second direction, based on any point in space.

[0155] The first processing module 204 is used to obtain, based on the first grating equation, the second grating equation, the third grating equation and the fourth grating equation, a first mapping relationship between the coordinates of the speckle spots in the speckle pattern in the first direction and the first field of view and the second field of view, and a second mapping relationship between the coordinates in the second direction and the first field of view and the second field of view, under the influence of the distortion of the camera module.

[0156] The second processing module 205 is used to determine the third mapping relationship between the field of view of the camera module and the distortion value based on the first mapping relationship and the second mapping relationship, and to obtain the distortion value of the camera module.

[0157] For a more detailed explanation of the functions and roles of each module in the aforementioned electronic device 200, please refer to the implementation process of the corresponding steps in the aforementioned distortion correction method, which will not be repeated here.

[0158] The aforementioned electronic device 200 can be any electronic device with information processing capabilities, such as a desktop computer, a laptop computer, etc.

[0159] Figure 10 A schematic block diagram of a computer system architecture for an electronic device used to implement the camera module distortion determination method of the embodiments of this application is shown.

[0160] It should be noted that, Figure 10 The computer system 300 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0161] like Figure 10 As shown, the computer system 300 includes a central processing unit (CPU) 301, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 302 or programs loaded from storage section 303 into random access memory (RAM) 304. The RAM 304 also stores various programs and data required for system operation. The CPU 301, ROM 302, and RAM 304 are interconnected via a bus 305. An input / output interface 306 (I / O interface) is also connected to the bus 305.

[0162] The following components are connected to the input / output interface 306: an input section 307 including a keyboard, mouse, etc.; an output section 308 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 303 including a hard disk, etc.; and a communication section 309 including a network interface card such as a local area network card, modem, etc. The communication section 309 performs communication processing via a network such as the Internet. A drive 310 is also connected to the input / output interface 306 as needed. A removable medium 311, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 310 as needed so that computer programs read from it can be installed into the storage section 303 as needed.

[0163] Specifically, according to embodiments of this application, the processes described in the various method flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 309, and / or installed from removable medium 311. When the computer program is executed by central processing unit 301, it performs various functions defined in the system of this application.

[0164] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such transmitted data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0165] Those skilled in the art will recognize that, in one or more of the examples above, the functions described in this application can be implemented using hardware, software, firmware, or any combination thereof. When implemented in software, these functions can be stored in a computer-readable storage medium or transmitted as one or more instructions or code on a computer-readable storage medium.

[0166] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0167] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, modular division is only a logical functional division, and other division methods may exist in actual implementation. For example, multiple units or components may be combined or integrated into another apparatus, or some features may be ignored or not executed.

[0168] As described above, this application quickly calculates the distortion value of the projection module based on its field of view, and compensates for the distortion in the projection module in the opposite direction through the imaging lens of the receiving module, minimizing the distortion of the depth measurement device. This method can be used in product design to effectively avoid the impact of pattern distortion on the accuracy of depth measurement.

[0169] It should be understood that this application is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A distortion correction method, characterized in that, include: Obtain the field of view of the first module; Mapping the field of view of the first module to obtain the first distortion value of the first module includes: obtaining the first field of view of the first module in a first direction and the second field of view in a second direction, where the first direction and the second direction are two coordinate axes in Cartesian coordinates; constructing a first grating equation by taking half of the first field of view as the incident angle and zero as the diffraction angle; and constructing a second grating equation by taking half of the second field of view as the incident angle and zero as the diffraction angle; and constructing a third grating equation corresponding to the first direction and the second grating equation corresponding to the second direction in spherical coordinates based on any point in space. The fourth grating equation in the second direction; based on the first grating equation, the second grating equation, the third grating equation, and the fourth grating equation, a first mapping relationship is obtained between the coordinates of the speckle spots in the speckle pattern in the first direction and the first and second field angles of view, and a second mapping relationship is obtained between the coordinates in the second direction and the first and second field angles of view, under the influence of the distortion of the first module; a third mapping relationship is determined between the field angle of view of the first module and the first distortion value according to the first and second mapping relationships, and the first distortion value of the first module is obtained; A corrected distortion value is determined based on the first distortion value, and the corrected distortion value cancels out the first distortion value. The second module is selected with the distortion value being the corrected distortion value; one of the second module and the first module is a projection module for projecting the speckle pattern, and the other is a receiving module for receiving the speckle pattern.

2. The method according to claim 1, characterized in that, The distortion type of one of the second module and the first module is pincushion distortion, and the distortion type of the other is barrel distortion.

3. The method according to claim 1, characterized in that, The step of determining a third mapping relationship between the field of view of the first module and the first distortion value based on the first mapping relationship and the second mapping relationship, and obtaining the first distortion value of the first module, includes: Obtain a speckle pattern under the distortion effect of the first module, and take two speckle spots located at the edge of the speckle pattern, the two speckle spots being located on the same edge of the speckle pattern; Obtain the quotient of the coordinate values ​​of the two speckle points in the first direction or the second direction; The quotient is used as the first distortion value of the first module.

4. The method according to claim 1, characterized in that, The third mapping relationship is as follows: Wherein, α represents the first distortion value, FOVX_A represents the field of view of speckle A in the x direction, FOVY_A represents the field of view of speckle A in the y direction, FOVY_D represents the field of view of speckle D in the y direction, speckle A and speckle D are located at the edge of the speckle pattern and on the same edge of the speckle pattern, and the field of view of the first module includes the field of view in the x direction and the field of view in the y direction.

5. A depth measuring device, characterized in that, include: The distortion value of the first module is obtained by mapping the field of view of the first module. The second module has a distortion value that is a corrected distortion value, which cancels out the distortion value of the first module. One of the second module and the first module is used to project the speckle pattern, and the other is used to receive the speckle pattern; The distortion value of the first module is obtained as follows: the first field of view angle in the first direction and the second field of view angle in the second direction are obtained, where the first direction and the second direction are two coordinate axes in Cartesian coordinates; half of the first field of view angle is taken as the incident angle and zero is taken as the diffraction angle to construct the first grating equation; and half of the second field of view angle is taken as the incident angle and zero is taken as the diffraction angle to construct the second grating equation. Based on any point in space, a third grating equation corresponding to the first direction and a fourth grating equation corresponding to the second direction are constructed in spherical coordinates. Based on the first grating equation, the second grating equation, the third grating equation, and the fourth grating equation, a first mapping relationship is obtained between the coordinates of the speckle spots in the speckle pattern in the first direction and the first and second field of view angles, and a second mapping relationship is obtained between the coordinates in the second direction and the first and second field of view angles, under the influence of the distortion of the first module. A third mapping relationship is determined based on the first and second mapping relationships to obtain the distortion value of the first module.

6. The depth measuring device according to claim 5, characterized in that, Includes a substrate, and the first module and the second module are disposed on the same surface of the substrate.

7. The depth measuring device according to claim 5, characterized in that, The first module includes a light source, a collimating lens, and a diffractive optical element. The light source is used to emit a light beam, the collimating lens is used to collimate the light beam emitted by the light source, and the diffractive optical element is used to project a speckle pattern based on the collimated light beam. The second module includes an imaging lens, a filter element, and a photosensitive element. The imaging lens is used to converge light, the filter element is used to perform wavelength filtering on the light converged by the imaging lens, and the photosensitive element receives the wavelength-filtered light to image a speckle pattern.

8. A method for determining camera module distortion, characterized in that, include: The first field of view angle of the camera module in the first direction and the second field of view angle in the second direction are obtained, wherein the first direction and the second direction are two coordinate axes in Cartesian coordinates; Using half of the first field of view as the incident angle and zero as the diffraction angle, a first grating equation is constructed; and using half of the second field of view as the incident angle and zero as the diffraction angle, a second grating equation is constructed. Based on any point in space, construct the third grating equation corresponding to the first direction and the fourth grating equation corresponding to the second direction in spherical coordinates respectively; Based on the first grating equation, the second grating equation, the third grating equation, and the fourth grating equation, a first mapping relationship between the coordinates of the speckle spots in the first direction and the first field of view and the second field of view in the speckle pattern under the distortion influence of the camera module is obtained, as well as a second mapping relationship between the coordinates in the second direction and the first field of view and the second field of view. Based on the first mapping relationship and the second mapping relationship, a third mapping relationship between the field of view and the distortion value of the camera module is determined, and the distortion value of the camera module is obtained.

9. The method according to claim 8, characterized in that, The third mapping relationship is as follows: Wherein, α represents the distortion value of the camera module, FOVX_A represents the field of view of speckle A in the x direction, FOVY_A represents the field of view of speckle A in the y direction, FOVY_D represents the field of view of speckle D in the y direction, and speckle A and speckle D are located at the edge of the speckle pattern and on the same edge of the speckle pattern.

Images