Imaging system with movable asymmetric lens

Abstract

An imaging system includes an image sensor in an xy plane, a lens module directing light onto the image sensor, and a controller. The lens module includes a fixed group of optical lenses and a movable group of optical lenses. The fixed group includes a first group of asymmetric optical lenses, and the movable group includes a symmetric optical lens and a second group of asymmetric optical lenses. The controller is configured to focus the imaging system within a focus range by generating an adjustment signal to move the movable group along the optical axis. The lens module focuses light on the image sensor along both x and y axes of the image sensor and within a focus range, and the first and second sets of asymmetric optical lenses are positioned such that astigmatism is below a threshold within the focus range.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of U.S. Provisional Application No. 63 / 329,045, filed on April 8, 2022, “Imaging System with Non-Symmetric Lens Element,” the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present disclosure relates generally to the field of imaging systems, and in particular to imaging systems having asymmetric optical lenses. Background Art

[0004] A camera consists of a housing, a light-sensitive surface, and a lens that images the scene onto the light-sensitive surface. In the past, the light-sensitive surface was film. In modern devices such as mobile devices and cameras (e.g., single-lens reflex (SLR) cameras, drone cameras, video cameras), film has been replaced by image sensors that convert the amount of light in a small area called a pixel into an electrical signal. Common image sensor devices are charge-coupled device (CCD) sensors and complementary metal oxide semiconductor (CMOS) sensors.

[0005] The size of a camera is generally determined by the image sensor size, lens size, and housing size. The larger the image sensor, the larger the lens. The size of the lens also depends on the focal length of the lens, which determines the camera's field of view. The lens design may also affect the size of the lens.

[0006] After years of evolution and technological advances in image sensors, lens design, lens manufacturing, and mechanical housing technology, cameras have converged on a few dimensions that vary between applications. For example, a camera system in a mobile device may be only a few millimeters in all 3 dimensions. This size may come from the desire for the mobile device to be small enough to fit into a small space such as a pants pocket or a small purse. As a result, in some cases, the largest image sensor that can be used in such a device is limited to a few millimeters by a few millimeters. In some cases, the image sensor may be further limited by the size of the lens that forms the image on the image sensor. To compensate for these limitations, modern mobile devices may have more than one camera, and the various cameras can be used to capture different fields of view and allow the user to perform operations such as zooming while maintaining good image quality.

[0007] As mentioned earlier, the size of image sensors in mobile devices is limited. Therefore, this limits the ability of the image sensor to collect a large amount of light when taking a picture. In addition to the sensor size, the amount of light that the sensor can capture can depend on the lens aperture and exposure time. Over the years, lens aperture (also known as F / #) has decreased due to better manufacturing capabilities and assembly techniques, but for mobile devices, lens aperture has reached a limit of approximately F / 1.8. This limits considerations such as image quality when the field of view moves or shakes. Specifically, exposure time may be limited by the user's ability to keep the camera still and the motion of objects in the scene. Optical image stabilization (OIS) methods may help increase this exposure time. However, these OIS methods cannot affect the "blue effect" that originates from the motion of objects in the scene.

[0008] As a result of the above constraints, the exposure time limit has reached a limit and is about 50 milliseconds for wide-angle lenses. This can be extended to 200 to 300 milliseconds using OIS methods, but as mentioned earlier, OIS methods cannot avoid blur caused by moving objects in the scene. In some cases, mobile devices use bracketing schemes to capture short and long exposures, and use information from the short exposure to reduce motion blur in the long exposure.

[0009] Therefore, these limitations on exposure time, lens aperture, and image sensor size restrict the amount of light that the camera can capture, and therefore restrict the image quality that the camera can deliver in terms of noise, resolution, dynamic range, color, and more. Summary of the invention

[0010] Some embodiments relate to an imaging system, which includes an image sensor on an xy plane (e.g., in a Cartesian coordinate system), a lens module that directs light onto the image sensor, and a controller. The lens module includes: a fixed group of one or more optical lenses, and a movable group of two or more optical lenses. The fixed group includes a first group of one or more asymmetric optical lenses, and the movable group includes at least a symmetric optical lens and a second group of one or more asymmetric optical lenses. The controller is configured to focus the imaging system within a focus range by generating an adjustment signal to move the movable group along an optical axis. The lens module focuses light on the image sensor along both the x-axis and the y-axis of the image sensor and within the focus range, and the first group of asymmetric optical lenses and the second group of asymmetric optical lenses are positioned so that the astigmatism is below a threshold within the focus range. If there is some astigmatism at a focus position within the focus range, it may be an astigmatism that reduces the MTF (modulus transfer function) by no more than 2 times (e.g., 1.5 times) compared to a focus position without astigmatism. In some embodiments, within the focus range, the astigmatism in any dimension does not increase by a factor greater than 2.

[0011] The combined optical power P of the first group of asymmetric optical lenses and the second group of asymmetric optical lenses along the x-axis or the y-axis may have a positive value (e.g., the combined optical power P of the first group of asymmetric optical lenses and the second group of asymmetric optical lenses along the x-axis or the y-axis is 10 diopters to 50 diopters). One of the groups of asymmetric optical lenses may have a negative combined optical power value along the x-axis or the y-axis, and the other group of asymmetric optical lenses may have a positive combined optical power value along the x-axis or the y-axis. In some embodiments, one of the groups of asymmetric optical lenses has a combined optical power value of -nP, and the other group of asymmetric optical lenses has a combined optical power value of +(n+1)P, where n is a real number greater than zero.

[0012] Due to the first group of asymmetric optical lenses and the second group of asymmetric optical lenses, the lens module can have different magnifications along the x-axis and the y-axis. In some embodiments, the fixed group is configured to be fixed relative to the image sensor. The lens module can be configured to move the movable group along the z-axis. In some embodiments, the fixed group includes only the first asymmetric optical lens. In some embodiments, the fixed group does not include a symmetric optical lens. The movable group may include one or more additional asymmetric optical lenses configured to correct optical aberrations (e.g., chromatic aberration, higher-order aberrations, spherical aberration, and coma).

[0013] The symmetric optical lens can be closer to the image sensor along the optical axis than the first group of asymmetric optical lenses and the second group of asymmetric optical lenses. In some embodiments, the movable group is closer to the image sensor along the optical axis than the fixed group. In some embodiments, the distance along the optical axis from the first group of asymmetric optical lenses to the aperture stop of the imaging system is less than half of the total length of the lens module along the optical axis.

[0014] In some embodiments, at least one of: the first group of asymmetric optical lenses includes two or more asymmetric optical lenses; or the second group of asymmetric optical lenses includes two or more asymmetric optical lenses.

[0015] The controller can be configured to generate an adjustment signal to move the movable group of optical lenses together. The controller can be configured to generate an adjustment signal to move each optical lens in the movable group by substantially the same amount. In some embodiments, the controller is configured to generate an adjustment signal to move the symmetric optical lens and the second group of asymmetric optical lenses separately.

[0016] In some embodiments, the lens module further comprises a field lens coupled to the image sensor, the field lens introducing asymmetric distortion to light passing through the field lens. In some embodiments, the imaging system comprises an asymmetric aperture having a larger axis and a smaller axis.

[0017] Some embodiments relate to an imaging system, which includes an image sensor and a lens module. The lens module is configured to direct light onto the image sensor, and the lens module has different magnifications along axes substantially orthogonal to a sensing plane of the image sensor. The lens module includes: a first group of one or more optical lenses, which includes symmetric optical lenses; a second group of one or more optical lenses, which includes a first asymmetric optical lens, which has a first optical focal length along a first axis parallel to the plane; and a third group of one or more optical lenses, which includes a second asymmetric optical lens, which has a second optical focal length along a second axis parallel to the plane; the imaging system also includes a controller, which is configured to generate an adjustment signal to focus the image onto the plane of the image sensor by adjusting the distance of the first group and the second group along the optical axis of the imaging system.

[0018] In some embodiments, the controller is configured to generate an adjustment signal to move the first group and the second group by substantially the same amount to focus the image on the plane. In some embodiments, the controller is configured to generate an adjustment signal to move the first group and the second group by substantially different amounts to focus the image on the plane.

[0019] Other aspects include components, devices, systems, improvements, methods, processes, applications, computer-readable media, and other techniques related to any of the above. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The disclosed embodiments have advantages and features, which will become more apparent from the detailed description, the appended claims and the accompanying drawings (or drawings). The following is a brief description of the drawings.

[0021] Figure 1A and Figure 1B An imaging system included in a mobile device according to an embodiment is shown.

[0022] FIG. 2A to FIG. 2C An imaging system capturing images of different portions of a view of an external environment is shown according to an embodiment.

[0023] Figure 2D Images of portions of views are shown according to embodiments.

[0024] Figure 2E According to an embodiment, Figure 2D The image in the image forms an image of the view of the external environment.

[0025] Figure 3A view of a lens module having different magnifications along the x-axis and the y-axis according to an embodiment is shown.

[0026] Figure 4 is a cross-sectional view of an example lens module having one set of symmetric lens elements and another set of asymmetric lens elements according to an embodiment.

[0027] FIG. 5A to FIG. 5B A cross section of another lens module having a symmetric group and an asymmetric group according to an embodiment is shown.

[0028] FIG. 6A to FIG. 6B A cross section of another lens module according to an embodiment is shown, wherein the lens module comprises a first group of symmetric optical lenses, a second group of optical lenses with a focal power, and a third group of optical lenses with a second focal power and another asymmetric optical lens.

[0029] FIG. 7A to FIG. 7B A cross section of another lens module is shown according to an embodiment, wherein the lens module includes a fixed group of lenses and a single movable group of lenses.

[0030] Fig. 8A A cross section of a lens module without a power optical lens according to an embodiment is shown.

[0031] Figure 8B According to the implementation method Fig. 8A A plot of the point spread function of light at the sensor in the lens module.

[0032] Figure 8C According to the embodiment Fig. 8A A lens module wherein one of the lenses is moved.

[0033] Fig.8D It is due to Figure 8C A plot of the point spread function of light at the sensor in the lens module.

[0034] Fig. 9A A cross section of a lens module with a focal power optical lens is shown.

[0035] Fig. 9B It is due to Fig. 9A A plot of the point spread function of light at the sensor in the lens module.

[0036] Fig. 9C According to the embodiment Fig. 9A A lens module in which two lenses in the lens are moved.

[0037] Fig.9D It is due to Fig. 9C A plot of the point spread function of light at the sensor in the lens module.

[0038] Fig.10 A cross section of a lens module with a field lens according to an embodiment is shown.

[0039] Fig.11 Example "string" distortion of a field lens according to an embodiment is shown.

[0040] Fig.12 is a block diagram illustrating components of an example machine capable of reading instructions from a machine-readable medium and executing the instructions in a processor (or controller) according to an embodiment. DETAILED DESCRIPTION

[0041] The drawings and the following description relate to preferred embodiments by way of illustration only. It should be noted that from the following discussion, alternative embodiments of the structures and methods disclosed herein will be readily considered as viable alternatives that can be employed without departing from the principles claimed.

[0042] Reference will now be made in detail to several embodiments, examples of which are shown in the accompanying drawings. Note that, where feasible, similar or like reference numerals may be used in the accompanying drawings and may indicate similar or like functions. The accompanying drawings depict embodiments of the disclosed systems (or methods) for illustrative purposes only. Those skilled in the art will readily appreciate from the following description that alternative embodiments of the structures and methods shown herein may be employed without departing from the principles described herein.

[0043] Configuration Overview

[0044] Some embodiments relate to an imaging system having a lens module that focuses light on an image sensor (e.g., having a large aspect ratio, such as 17:9). The lens module may have different magnifications in orthogonal axes (referred to as the x-axis and the y-axis) parallel to the sensing plane of the image sensor. The lens module may include at least a first group of optical lenses and a second group of optical lenses. The first group includes an asymmetric optical lens and may be fixed relative to the image sensor. The second group may include an asymmetric optical lens and one or more symmetric optical lenses. The second group of optical lenses may be movable relative to the image sensor (e.g., along an axis perpendicular to the sensing plane of the image sensor) to adjust the focus of the lens module. The optical power of the asymmetric optical lenses in each group is configured so that for any focus position of the second group, light is focused on the image sensor along both the x-axis and the y-axis.

[0045] Some embodiments relate to an imaging system having a lens module that focuses light on an image sensor (e.g., having a large aspect ratio, such as 17:9). The lens module may include an asymmetric optical lens coupled to the image sensor (or near the image sensor) that introduces asymmetric distortion to the light passing through. For example, the optical lens distorts the focused image along at least one axis parallel to the sensing plane of the image sensor. Such an optical lens may be referred to as a field lens. The distortion may increase with the distance from the center of the field lens. For example, the field lens includes a central portion having the same or similar focal length along the x-axis and the y-axis, and an outer portion surrounding the central portion having different focal lengths along the x-axis and the y-axis. The distortion enables the imaging system to capture a larger field of view in the direction along the axis. Thus, for example, the imaging system enables an image sensor having an aspect ratio of 4:2 to capture a scene having an aspect ratio of 4:3.

[0046] Imaging System

[0047] Figure 1A to Figure 1B An example imaging system 101 is shown as being included in an example mobile device 103 according to an embodiment. Specifically, Figure 1A shows a front view, a rear view, and a side view of a mobile device 103, and Figure 1B A cross-sectional rear view and a cross-sectional side view of a mobile device 103 are shown. The mobile device 103 includes an imaging system 101, a housing 117 having a window 102, and a display 119. The imaging system 101 includes a rotatable reflector 105, a motor 111, a motor 112, a lens module 107 (also referred to as a lens design), an image sensor 109, and a controller module 113.

[0048] The reflector 105 directs light passing through the window 102 downward toward the lens module 107. The lens module 107 focuses the light on the image sensor 109. The motor 111 rotates the reflector 105 about an axis 115, which is substantially parallel (e.g., within one or two degrees) to the image sensor plane. Rotating the reflector 105 allows the reflector 105 to direct light from different parts of the external environment toward the image sensor 109. The controller 113 is electrically coupled to the image sensor 109 and the motor 111. In order to form an image of the external environment, the imaging system 101 captures images of multiple parts of the view of the external environment while rotating the reflector 105. The rotation of the reflector 105 from the initial angular position to the final angular position can be referred to as scanning. The sequence of captured images contains information of several adjacent parts of the environment, and after combining (e.g., stitching or fusing) the images together, the imaging system 101 forms a larger image of the external environment with a predetermined aspect ratio.

[0049] The housing 117 contains one or more components of the imaging system 101. The position and orientation of the imaging system components can be described relative to the housing 117 and the housing window 102. For example, the housing 117 is defined by a plurality of walls containing the imaging system 101, and one of the walls includes the housing window 102, the housing window 102 having a plane, for example, defined by the boundary of the window 102. The plane can be parallel to the yz (or yz) plane in a three-dimensional reference system. The housing 117 can have a lower profile along an axis perpendicular to the plane of the window 102 (e.g., along the x-axis). The length of the housing along the x-axis can be referred to as the thickness of the housing 117 and can be in a range of, for example, from 5 mm to 15 mm. In an embodiment where the housing 117 is part of the mobile device 103, the window plane can be parallel to the display 119 of the mobile device 103. Unlike conventional imaging systems, the image sensor surface does not face the window plane. For example, the image sensor surface is perpendicular to the window plane (e.g., parallel to the xy plane) and outside the boundary of the window 102. Thus, the reflector 105 can be aligned with the window 102 to direct light to propagate through the window 102 to the image sensor plane. The lens module 107 can be between the reflector 105 and the image sensor 109. The aperture plane can be between the reflector 105 and the lens module 107 and can be perpendicular to the window plane and parallel to the image sensor plane. The reflector allows the optical path of the imaging system 101 to be folded into the yz plane. The folding allows the optical path to increase beyond the limit of the housing thickness and into the width (e.g., length along the y-axis) and height (e.g., length along the z-axis) of the housing, which are typically greater than its thickness. Therefore, the aperture of the lens module 107, the image sensor and / or the reflector can have an aspect ratio that is not 1:1, and their long axes can be parallel to each other.

[0050] As used herein, the terms "parallel" and "perpendicular" may refer to components being substantially parallel or substantially perpendicular (eg, within one or two degrees), as making components that are perfectly parallel or perpendicular may be practically difficult to achieve.

[0051] Image sensor 109 is an imaging device that captures images of multiple parts of the external environment. Examples of image sensor 109 include CCD sensors and CMOS sensors. As shown in FIG1 , the image sensor surface can be located in an xy plane relative to the xy plane of the mobile device 103, and the image sensor surface faces the vertical direction (along the z-axis) to the xy plane. Due to this positioning, the sensor plane of image sensor 109 does not face the field of view of the external environment. By placing image sensor 109 in the xy plane, the size of image sensor 109 can be larger than that of an image sensor in a conventional camera. The smaller dimension (along the x-axis) of the image sensor plane may be limited by the thickness of the mobile device, while the longer dimension (along the y-axis) may be limited by the width of the mobile device, and they may be many centimeters long. This allows image sensor 109 to have a high aspect ratio, such as a ratio greater than 17:9 (e.g., 1:10). The scene image produced by a conventional camera has a less high aspect ratio (e.g., a sensor limited to 2:3, 3:4, or 16:9). Due to the high aspect ratio of image sensor 109, image sensor 109 can create a narrow image ("image swath") corresponding to a narrow view of a scene. For conventional imaging systems in mobile devices, the size of the image sensor may be limited by the focal length of the camera lens. However, by changing the position and orientation of image sensor 109 as described herein, the image sensor size can be larger than an image sensor in a conventional imaging system having the same or similar housing.

[0052] As described above, reflector 105 (also referred to as a scanning mirror) is an optical component that rotates about axis 115 to direct light to image sensor 109. Typically, axis 115 is substantially parallel to the long dimension of the image sensor plane, and reflector 105 is centered on window 102. If the plane of window 102 (e.g., yz plane) is perpendicular to the plane of image sensor 109 (e.g., xy plane), reflector 105 can direct light at a position of approximately 45 degrees relative to the image sensor plane to direct light to image sensor 109. Due to the high aspect ratio of image sensor 109, reflector 105 can also have a high aspect ratio to ensure that light is reflected to the entire surface of image sensor 109. Reflector 105 is preferably disposed at a substantially horizontal position relative to the image sensor plane. Figure 1B is shown as having a rectangular plan, but other shapes such as concave or convex (eg, which may be used to expand or reduce the field of view) are possible.

[0053] The reflector 105 is described herein in terms of "directing" light, however this is for ease of description. The reflector 105 can optically direct, widen, thin, reflect, diffract, refract, scatter, amplify, reduce, combine, separate, polarize, or otherwise change the characteristics of the light as it propagates in the imaging system 101. To this end, the reflector 105 can include a reflective coating, metallized features, gratings, mirrors, prismatic structures, Fresnel structures, corner reflectors, retroreflectors, etc. on one or more of its surfaces.

[0054] The lens module 107 includes one or more optical components and is designed to form an image on the image sensor 109. The lens module 107 can disperse, focus, redirect, and otherwise modify the light passing through the lens module 107. The lens module 107 can be as simple as a single lens, or the lens module 107 can include additional optical components, such as diffusers, phase screens, beam expanders, mirrors, and lenses (e.g., anamorphic lenses). In some embodiments, the entrance pupil of the lens module 107 is adjacent to the reflector 105. This can allow the reflector 105 to have a smaller size. In some embodiments, the lens module 107 includes an asymmetric aperture with a large axis and a small axis (the stretched axis can be used for devices with size constraints, such as smartphones, and in these cases, if the aperture is not symmetric, it can be larger).

[0055] Due to the high aspect ratio of the image sensor 109, the lens module 107 can be designed and manufactured to be non-circular or asymmetric and follow the dimensions of the image sensor 109 in terms of its aperture. Using a lens module 107 with an asymmetric aperture can allow the lens module 107 to be adapted in a mobile device housing 117. In addition, the focal length of the lens module 107 can be different in the x and y directions. In some embodiments, this results in the imaging system 101 not preserving the aspect ratio, so that, for example, a 4:3 scene can be imaged by an 8:3 image sensor. One or more optical components of the lens module 107 can have a cylindrically symmetric surface, but the aperture of other components can be rectangular or another elongated shape. The lens module 107 can be manufactured using wafer-level techniques, which may be advantageous for producing optical components with a rectangular shape by cutting the lens surface at a desired aspect ratio. In some embodiments, the lens module 107 is manufactured using injection molding techniques by creating a mold with an asymmetric aperture. The components of the lens module 107 can be glass or plastic injection molded or machined (e.g., via wafer-level techniques). Reference will be made to Figures 3 to 11 Example lens modules are further described.

[0056] The motor 112 is controlled by the controller 113 and is configured to move the lens module or one or more optical components of the lens module 107. For example, the motor 112 moves one or more optical lenses along the optical axis to focus light on the sensing plane of the image sensor 109. For example, if multiple optical components should be moved separately or by different amounts, the imaging system may include multiple motors 112. The motor 112 may include one or more actuator-type mechanisms, galvanometer-type mechanisms, mems-type mechanisms, motorized-type mechanisms, or stepper-motor-type mechanisms. The motor 112 may also be referred to as a lens shift mechanism.

[0057] As described above, the motor 111 rotates the reflector 105 about the axis 115. To this end, the motor 111 may include one or more actuator-type mechanisms, galvanometer-type mechanisms, mems-type mechanisms, motor-type mechanisms, or stepper-motor-type mechanisms. In some embodiments, as further described below, the motor 111 can move the reflector 105 in other directions. For example, the motor 111 can translate and / or rotationally move the reflector 105 along the x-axis, y-axis, and / or z-axis.

[0058] In some embodiments, while the image sensor 109 is capturing an image of a portion of the scene, the motor 111 tilts the reflector 105 (e.g., a few degrees in either direction) to compensate for motion (e.g., hand motion). For example, if the user tilts the mobile device 103 slightly downward, the motor can tilt the reflector 105 upward to compensate for the motion so that the image sensor 109 receives the same portion of the scene despite the tilt. In some embodiments, the imaging system 101 includes a sensor shifting mechanism (e.g., another motor) to shift the image sensor 109 in one or more directions (e.g., in the xy plane) to compensate for the motion. In some embodiments, the imaging system 101 includes a motor 112 for shifting the lens module 107 (or a component of the lens module 107) in one or more directions (e.g., in the xy plane) to compensate for the motion. If the imaging system 101 includes multiple motion compensation mechanisms, the controller 113 can coordinate the multiple mechanisms to work together to offset the motion. For example, motor 111 tilts reflector 105 to compensate for motion in one direction, and a sensor shift mechanism or lens shift mechanism (e.g., 112) compensates for motion in another direction. In some embodiments, reflector 105 rotates about multiple substantially perpendicular axes (e.g., x-axis and z-axis) to compensate for motion (e.g., in place of a sensor or lens shift mechanism).

[0059] The motor 111 and the shift mechanism (e.g., 112) can also be used as an autofocus mechanism. For example, the lens shift mechanism shifts the lens module 107 (or its components) closer to or farther away from the image sensor 109 (e.g., along the z-axis) to achieve a desired focus. In another example, the sensor shift mechanism shifts the image sensor 109 closer to or farther away from the lens module 107 (e.g., along the z-axis) to achieve a desired focus.

[0060] The controller module 113 may constitute software (e.g., program code implemented on a machine-readable medium and executable by a processing system to cause the processing system to operate in a particular manner) and / or hardware to provide control signals (also referred to as adjustment signals) to the motor 111, the motor 112, the image sensor 109, or some combination thereof. Thus, the controller 113 may: (1) rotate the reflector 105 via the motor 111 to direct light from different portions of the external environment toward the image sensor 109; (2) focus light on the image sensor 109 by adjusting the optical components of the lens module 107 via the motor 112; (3) synchronize the image sensor 109 with the reflector 105 to capture images of different portions of the environment; or (4) some combination thereof. In addition, the controller 113 may receive captured images and combine the captured images to form a larger continuous image of the external environment.

[0061] In some embodiments, the imaging system 101 includes one or more motion sensors (e.g., accelerometers, gyroscopes, etc.) to track the motion of the imaging system relative to the external environment. The controller module 113 can receive motion data from the motion sensor. If the determined motion is above a threshold amount, the module 113 can provide instructions to the motor 111 and / or the sensor displacement mechanism to compensate for the motion.

[0062] In some embodiments, the imaging system 101 is not contained in the mobile device 103. For example, the imaging system 101 is contained in a housing of a stand-alone device such as a mobile phone 103.

[0063] FIG. 2A to FIG. 2C An imaging system 101 is shown capturing images of different portions of a view of an external environment according to an embodiment. FIG. 2A to FIG. 2C In the example of , the external environment includes one or more objects within the field of view. In this example, for ease of discussion, the objects are a vertically aligned cube 211A, a sphere 211B, and a pyramid 211C. Figure 2A In FIG. 1 , reflector 105 is tilted at a first rotational position (e.g., reflector 105 forms an angle θ1 with respect to the yz plane) to direct light from the top of the external environment toward image sensor 109. Thus, image sensor 109 captures an image of cube 211A. Figure 2B, the reflector is tilted at a second rotational position (e.g., the reflector forms an angle θ2>θ1 relative to the yz plane) to direct light from the middle portion of the external environment toward the image sensor 109. Thus, the image sensor 109 captures an image of the sphere 211B. Figure 2C , the reflector is tilted at a third rotational position (e.g., the reflector forms an angle θ3>θ2 relative to the yz plane) to direct light from the bottom of the external environment toward the image sensor 109. Therefore, the image sensor 109 captures an image of the pyramid 211C. In some example embodiments, to capture a set of images, the reflector angle θ may vary symmetrically around a 45 degree position relative to the xy plane (e.g., from 25 degrees to 65 degrees).

[0064] Figure 2D 1 shows three image strips captured by image sensor 109 according to an embodiment. Since each strip is captured when reflector 105 is in a different rotational position, each image strip is an image of a different portion of the external environment. Due to the high aspect ratios of reflector 105, lens module 107, and image sensor 109, the image strips have a high aspect ratio. Image strip A is an image of cube 211A and is represented by Figure 2A The image strip B is an image of the sphere 211B and is captured by the image system 101 in FIG. Figure 2B The image strip C is an image of the pyramid 211C and is captured by the imaging system 101 in FIG. Figure 2C The image system 101 is captured.

[0065] The exposure time for capturing each image strip may be limited by user motion (the user may inadvertently move the device 103 while holding the device 103) and objects moving in the scene. In addition, the total exposure time for the image strips may be limited by possible changes in the external environment between image strip captures. The image strip exposure time and the total exposure time may be limited to a predetermined threshold time or determined dynamically (e.g., based on the amount of movement of the mobile device 103).

[0066] Figure 2E An image 201 of a view of an external environment according to an embodiment is shown. The image 201 is obtained by Figure 2D The image strips A to C shown are combined (e.g., fused or stitched) to form a combined image 201. The combined image 201 may be referred to as a composite image. The horizontal field of view of the combined image 201 may be based on the width (along the y-axis) of the window 102, the reflector 105, the lens module 107 (e.g., its aperture), and / or the image sensor 109, and the vertical field of view of the combined image 201 may be based on the scanning range of the reflector 105. Typically, the vertical field of view is greater than the horizontal field of view.

[0067] Depending on the position of the reflector 105 when capturing the image strips, the image strips may have some overlap with each other (e.g., 10 to 300 rows of pixels). Capturing image strips with overlap can help ensure that the image strips do not miss portions of the environment view (e.g., so as to capture the entire view) and can reduce the noise value of the combined image 201. Capturing image strips with overlap can also help the combination process to ensure that the image strips are correctly combined. For example, the controller 113 uses the overlapping portions to align the image strips during the combination process. In another example, if an object in the environment moves between the capture of the image strips, or if the mobile device 103 moves between the capture of the image strips, the control system 101 can use the overlapping portions to correct artifacts caused by the movement.

[0068] Rotate the reflector

[0069] The rotation of the reflector 105 can be discrete so that it rotates from an initial (e.g., maximum) angular position of the reflector 105 to a final (e.g., minimum) angular position having N stops, where N is the number of image strips that will form the combined image. N can be as small as 2. N can depend on the desired exposure time of the combined image and / or the size of the smaller dimension of the image sensor 109 and the desired size or aspect ratio of the combined image. For example, if the image sensor has 24,000×6,000 pixels and if the final combined image has an aspect ratio of 4:3, the reflector 105 will have three discrete positions and the combined image will be 24,000 pixels×18,000 pixels. The previous scanning example does not include any overlap in the image strips. If N is increased, certain areas in the scene will appear more than once in the image strips. For example, if scanning is performed using six discrete angular positions, each point in the scene will appear in two image strips.

[0070] The imaging system 101 may be capable of capturing video. In these cases, the combined images may form frames of the video. For example, if the video frame rate or preview frame rate is 25 FPS (frames per second), the total exposure time for each combined image is 40 milliseconds or less. In the case of three discrete position scans, each position may be exposed for 13.33 milliseconds. However, the reflector 105 takes time to change its position and stop, which means that the exposure time for each image strip may be around 10 milliseconds.

[0071] For still image capture, the image preview displayed to the user can be interrupted when the user presses the capture button and allow for longer exposures than would be limited by the image preview speed.

[0072] The above considerations take into account the full field of view. If the imaging system 101 captures a narrower field of view, the scanning range of the reflector 105 may be reduced. For example, if the user zooms in three times (i.e., 3x zoom), the imaging system 101 may not perform any scanning. Therefore, the reflector 105 can be stationary. For example, if the image sensor 109 has 24,000 pixels by 6,000 pixels and the final image has a height of 6,000 pixels and an aspect ratio of 4:3, the reflector 105 may not be rotated and the other scale of the image may be 8,000 pixels (e.g., read out and cropped from the 24,000 pixel scale of the image sensor 109).

[0073] In some embodiments, the rotation of the reflector 105 is continuous rather than discrete. In the continuous scanning mode, the reflector 105 rotates continuously at a sufficiently slow speed so that the captured image is not blurred, but rotates continuously at a sufficiently fast speed to complete the scanning of the desired field of view at a desired frame rate (e.g., 40 milliseconds). In the continuous mode, the rotation rate of the reflector 105 can be determined by the desired frame rate. For example, if the frame rate is 30FPS (33 milliseconds between frames), the scene scan requires approximately 25 milliseconds, and then the reflector 105 is rotated back to its initial position. Other example values ​​(e.g., 30 milliseconds) are also possible, depending on how quickly the reflector is rotated back to its initial position. In the embodiment where the reflector 105 is bilateral, the reflector 105 may not need to be rotated back to its initial position.

[0074] In the continuous scanning mode, points in the external environment can appear on each row of pixels during scanning. The image sensor 109 can capture enough images so that each row of pixels of the continuous image strip captures points. For example, if the image sensor 109 includes 6000 rows of pixels, the image sensor 109 can capture 6000 images during a single scan. To do this, for example, the image sensor can integrate the charge from the changing pixels instead of integrating the charge on one pixel within a certain number of milliseconds. If this change (scanning) is synchronized with the reflector rotation speed, the output can correspond to a point in space. An example implementation using the image sensor 109 is to read only one row of pixels, which can occur very quickly. Therefore, for example, a sensor that performs 30FPS (frames per second) and has 6000 rows can perform 15000FPS and read only one row. As an alternative to capturing enough images to capture points for each row of pixels, the image sensor 109 can capture a predetermined number of images less than the number of pixel rows during scanning.

[0075] Lens modules with different magnifications along the axis

[0076] In some example embodiments, the desired output aspect ratio of the image strips is different from the aspect ratio of the image sensor 109. For example, the imaging system 101 images a scene having an aspect ratio of 4:3 or 16:9 (or other desired ratio) onto the image sensor 109 having a much higher aspect ratio, such as 5:1 or 10:1. More generally, the lens module 107 may have different magnifications or distortions along different axes (e.g., the x-axis and the y-axis) when imaging a (e.g., rectangular) field of view having an aspect ratio of 1:n onto an image sensor having an aspect ratio of 1:m, where m is greater than n. For example, the lens design focuses a scene having an aspect ratio of 4:3 onto an image sensor having an aspect ratio of 4:2. In order to have different magnifications or distortions along different axes (e.g., the x-axis and the y-axis), the lens module may include one or more asymmetric optical lenses (if the lens module includes only a single asymmetric optical lens, the optical lens may have two asymmetric surfaces). Note that lens modules with the same magnification or distortion along different axes (eg, x-axis and y-axis) can also have asymmetric optical lenses.

[0077] To this end, the lens module 107 can have different focal lengths along the x-axis and the y-axis (z is the optical axis). The different focal lengths can be achieved by using asymmetric optical lenses such as free-form optical elements (also known as anamorphic lenses) or cylindrical optical elements that have optical power in only one direction. As a result of using such a lens module 107, each pixel on the sensor (assuming that the pixel is a symmetrical pixel (e.g., square)) can collect photons from a rectangular area in the scene.

[0078] To achieve similar resolution in the x-direction and the y-direction, it may be advantageous for the image sensor 109 to have asymmetric pixels (e.g., rectangular pixels). Another option is to apply (e.g., by the controller module 113) analog or digital binning in the longer dimension (e.g., along the y-direction) of the image sensor 109 to reduce the resolution in that dimension to a resolution corresponding to the desired output aspect ratio.

[0079] Figure 3 Side and back views of an example lens module 107 with different magnifications along the x-axis and y-axis are shown (individual optical lenses are not shown) according to an embodiment. Image sensor 109 has a high aspect ratio, which can be seen by comparing its length along the x-axis and y-axis. Lens module 107 has a shorter focal length in the xz plane and a longer focal length in the yz plane. To do this, Figure 3 The lens module 107 in FIG. 1 has one or more asymmetric optical lenses to ensure that the lens module 107 can focus the image on the plane of the image sensor 109 .

[0080] In order to design a lens module 107 with different focal lengths in two dimensions perpendicular to the optical axis, at least two of the optical lens surfaces can be asymmetric. Using only one asymmetric surface may result in different focal lengths, but may also result in different focal planes, which may make it difficult or impossible to capture images using a single image sensor 109. Therefore, in these embodiments, the lens module 107 can have an optical design with at least two asymmetric surfaces (e.g., a single lens or multiple asymmetric lenses with asymmetric front and back surfaces). Having two asymmetric surfaces can provide different magnifications along two dimensions, and both dimensions are focused on the same plane.

[0081] As used herein, an asymmetric optical lens is an optical lens whose surface focuses or diverges light differently along one axis (e.g., the x-axis) than along another axis (e.g., the y-axis). For example, an asymmetric optical lens may have a P along the x-axis. x >0 optical power and P along the y axis y =0 optical power. In order to have different optical powers along different axes, an asymmetric optical lens can have a surface that is asymmetric in shape (e.g., within manufacturing standards or capabilities). For example, an asymmetric optical lens has a surface whose radius of curvature is not constant or is different in the x-direction and the y-direction. In some embodiments, asymmetry refers to asymmetry about the optical axis of the optical lens. Example asymmetric optical lenses include cylindrical lenses and anamorphic lenses.

[0082] As used herein, a symmetrical optical lens refers to an optical lens whose two or more surfaces focus or diverge light along a certain axis (e.g., the x-axis) in the same way as along another axis (e.g., the y-axis) (e.g., within manufacturing standards or capabilities). For example, a symmetrical optical lens has substantially the same optical power along both the x-axis and the y-axis (e.g., a difference of less than 1% or 0.1%). In order to have the same optical power along different axes, a symmetrical optical lens can have two or more optical surfaces that are symmetrical (e.g., within manufacturing standards or capabilities). For example, a symmetrical surface has a constant radius of curvature. In some embodiments, symmetry refers to symmetry about the optical axis of the optical lens. Example symmetrical optical lenses include convex lenses and concave lenses.

[0083] exist Figure 4 An example lens module 107 having a symmetric optical lens and an asymmetric optical lens is shown in FIG. Figure 4 The example lens module 107 in FIG. 4 includes a set of symmetrical optical lenses (“symmetrical group”) and a set of asymmetrical optical lenses (“asymmetrical group”) along the optical axis 405. By comparing the side view and the top view, the asymmetrical surfaces of the optical lenses in the asymmetrical group can be identified. Figure 4In the example of , the symmetric group includes five optical lenses and the asymmetric group includes two cylindrical lenses, however each group may include more or fewer optical lenses (eg, only one).

[0084] Typically, the lens module can be designed so that for an object at a certain distance from the imaging system, both axes (e.g., the x-axis and the y-axis) are focused on the same plane. To enable the lens module to focus to other distances, a set of one or more symmetrical optical lenses (e.g., Figure 4 A symmetric group in the lens module (e.g., 114) can be moved backward or forward along an optical path or relative to an image sensor (e.g., via a controller module (e.g., 113) and a motor (e.g., 112)). Such movement of the symmetric lens can change the focus distance as desired, but there may be some astigmatism at the new focus position due to one or more asymmetric optical lenses in the lens module. For example, at least one of the axes may be (e.g., slightly or partially) out of focus. In order to adjust (e.g., correct or update) the focus of at least one axis, the asymmetric optical lens (or optical lens group) of the lens module may be moved (e.g., together with the symmetric group or separately from the symmetric group). Such movement of the asymmetric optical lens can eliminate astigmatism along the axis and can enable both axes to be focused at the same desired distance.

[0085] The asymmetric optical lens (or lens group) in the lens module can be close to the aperture stop rather than close to the image sensor (for example, the optical lens closest to the aperture stop is less than 5 mm from the aperture stop, or less than half of the total optical length of the lens module along the optical axis). If the asymmetric lens (or lens group) is close to the image sensor plane, it can affect the distortion, but it will not introduce different magnifications on different axes (for example, the x-axis and the y-axis). If the asymmetric lens (or lens group) is located at the aperture, it may introduce different magnifications but not affect the distortion. Therefore, if the asymmetric lens (or lens group) is located between the two, it can affect the distortion and provide different magnifications. In some embodiments, the asymmetric optical lens is a pupil lens, meaning that it has the same or similar focusing effect in the center and corners / edges of the image (in contrast to a field lens, which focuses differently at the center of the image than in the corners / edges of the image). This typically means that the asymmetric optical lens is close to the aperture stop (which is typically located at or close to the entrance pupil) (e.g., a distance less than one-third of the total length of the lens module along the optical axis). Figure 4 In the example of FIG. 5 , the asymmetric group is close to the aperture stop (eg, the distance between them is less than one-third of the total length of the lens module along the optical axis), so it is effective for focusing and eliminating astigmatism.

[0086] The amount of movement of an asymmetric optical lens (or optical lens group) when focusing light onto an image sensor is a function of the power of the optical lens. When the optical power of an asymmetric lens is low (e.g., less than 10 diopters), large movements may be required to adjust (e.g., correct) the focus. In some cases, the movement may be greater than the imaging system can provide (e.g., the desired movement of the asymmetric element would mechanically interfere with another optical lens). FIG. 5A to FIG. 5B This situation is shown in . Figure 5A A cross section of a lens module 507 and an image sensor 509 is shown, wherein the lens module 507 includes a symmetrical group and an asymmetrical group (optical lenses of the symmetrical group are not shown). In addition to the group being moved to double the focusing distance of the lens module 507, Figure 5B and Figure 5A In this example, the single optical lens 501 of the symmetric group and the asymmetric group is moved closer to the image sensor 509 to focus the lens module 507. However, in this particular example, in order to double the focus distance, the optical lens 501 is moved further than the symmetric group to focus the image on the image sensor (e.g., so that both the x-axis and the y-axis are focused at the sensing plane of the image sensor 509), resulting in the optical lens 501 contacting the optical lens in the symmetric group.

[0087] Therefore, in order to increase the focusing distance of the lens module without bringing the optical lenses into contact with each other, it may be desirable to reduce the amount of movement of the optical lenses in the lens module. Reducing the movement of the optical lenses can also reduce the focusing time and power consumption of the imaging system. In addition, large movements may negatively affect optical performance or introduce undesirable optical effects (e.g., vignetting).

[0088] (i) Lens module with focal optical lens

[0089] In order to reduce the movement of an asymmetric optical lens with optical power P>0 on an axis (e.g., the x-axis or the y-axis), one solution is to replace the asymmetric optical lens with two or more asymmetric optical lenses having a combined optical power P in the axis. Asymmetric optical lenses with a combined power P can be referred to as "power optical lenses". In this context, P is a power that can be used to achieve a desired focal length of a lens module in a dimension (e.g., in the x or y dimension). For example, P causes or contributes to a lens module having a different magnification along one of the axes (e.g., the x-axis). P can be derived based on a desired "squeezing" factor with respect to one of the axes. In some embodiments, the desired squeezing is approximately 20% to 50%. In these embodiments, the power P can be in the range of 10 diopters to 50 diopters. While many different combinations of powers of power optical lenses are possible (provided the combined power is P>0), having a power optical lens where the absolute value of the lens is greater than P results in reduced lens motion used to focus the lens module (compared to a single asymmetric optical lens with power P). To do this using two power optical lenses, one element can have positive power and the other element can have negative power so that the combined power is P (the combination of positive and negative power can also help reduce or correct chromatic aberration). For example, one power optical lens has a power of -nP and the other power optical lens has a power of +(n+1)P (where n is a real number greater than zero). Thus, in this example, the combined power of the two power lenses is P, where one power optical lens has a strong negative power and the other power optical lens can have a slightly stronger but positive power. That is, the power relationship of the power optical lenses can be different from the -nP and +(n+1)P examples described above (for example, if there are three or more power optical lenses). In another example, to achieve a lens module having a focal length of 20 mm along the x-axis and a focal length of 10 mm along the y-axis, the lens module may include a symmetric lens that achieves a focal length of 20 mm in the x-axis and the y-axis and two (or more) asymmetric lenses, wherein the asymmetric lenses have a combined optical power sufficient to magnify 1 / 2 times along the y-axis (thereby doubling the field of view). Among other advantages, the lens module results in an expanded field of view along the y-axis that can be captured by an image sensor (e.g., which has a shorter dimension along the y-axis).

[0090] As described above, one advantage of using power optical lenses (instead of a single asymmetric optical lens with a power of P) is that for focusing, one of the power optical lenses can be moved a smaller amount (compared to a single asymmetric optical lens with a power of P). This may be because each power optical lens has a relatively high optical power (e.g., -nP or +(n+1)P). For example, for an optical lens with a power of 100 along an axis and a second optical lens with a power of -80, a motor may move one of the elements 0.5 mm or less to correct for astigmatism at all distances along the axis.

[0091] (ii) Lens module with fixed and moving groups

[0092] In an embodiment where the lens module includes a power optical lens, the lens module may include two (or more) moving groups to focus the imaging system. The first group serves as the main focusing group (e.g., it includes one or more symmetrical lenses, one or more asymmetric lenses, or a combination thereof). In order to change the focusing distance, a motor (e.g., motor 112) can move the first group along an optical path (e.g., relative to the image sensor) so that one axis (e.g., the x-axis) is focused at a desired distance. The second group includes a first power optical lens having an optical power along a second axis and can also be moved along the optical path. The motor (or another motor) can move the second group so that the second axis (e.g., the y-axis) is focused at a desired distance. The third group includes a second power optical lens and can remain fixed relative to the image sensor (in this example, there are only two power optical lenses).

[0093] The positions of the first moving group and the second moving group may have some dependency on each other. For example, when the first group is at position T1, the second group is at position S1 (the position of one group may depend on the position of the other group). For any combination of positions Tn and Sn (where n is an integer), the lens module is focused at a certain distance, and for this distance, both axes are (e.g., correctly) focused. These positions and their corresponding focus distances can be known by the controller module (e.g., 113), so that the controller module (e.g., 113) can provide appropriate adjustment instructions to one or more motors (e.g., 112) to focus the imaging system at a given distance.

[0094] FIG. 6A to FIG. 6B An example of a lens module 607 having three groups (two moving groups and one fixed group) is shown. Specifically, Fig. 6A607 and an image sensor 609, wherein the lens module 607 includes a first group 621 (e.g., a symmetric lens (lens not shown)), a second group 622 having a power optical lens 601, and a third group 623 having (1) a power optical lens 602 and (2) another asymmetric optical lens. The power optical lens 601 and the power optical lens 602 may have an asymmetric lens 501 ( FIG. 5A to FIG. 5B In addition to group 621 and group 622 being moved closer to image sensor 609 (e.g., by a motor) to change the focusing distance of lens module 607, Figure 6B and Fig. 6A Similarly, the third group remains fixed. Among other advantages, since the focal optical lens 601 has a higher optical power than the lens 501, Figure 6B The focal length of the optical lens 601 moves less than Figure 5B , and therefore does not contact the optical lenses in the first group 621.

[0095] Other embodiments for achieving the same (or similar) results include a lens module having a single moving group for adjusting focus, thereby reducing the complexity of the imaging system. For example, only a single mechanism (e.g., a motor) changes the focus. This can be achieved by coupling the power optical lens (e.g., 601) to the first group (e.g., 621). Therefore, one of the power optical lenses moves with the first group (e.g., they are coupled to the same lens holder), while the other power optical lens (e.g., 602) remains fixed. In this case, the power of the power optical lens coupled to the first group can be selected so that for any focus position, the lens module focuses both axes (e.g., the x-axis and the y-axis) at the same distance. To achieve this, for example, one can (1) determine what motion distance (e.g., in mm) can be used to focus a first dimension of the field of view (e.g., the x-dimension), and then based on this, (2) design the powers of the power optical lenses so that (a) their combined power satisfies the lens design for the purpose of stretching (or compressing) a second dimension (e.g., the y-dimension), and so that (b) the power of one of the power optical lenses is such that, for objects at any distance, to correct for astigmatism in the second dimension, the power optical lens uses the same motion distance (e.g., in mm) as used to focus the first dimension (e.g., the x-dimension).

[0096] More generally, first, the focus range (e.g., 1m to 5m) and focal length of each of the first and second axes (e.g., x-axis and y-axis) of the lens module can be determined. For example, the desired lens module should have a y-axis focal length that is 1.5 times the x-axis focal length. The desired focal length can be based on the desired focus range and the size of the imaging system (e.g., the size of the image sensor and the amount of space available to move the optical lens to achieve the desired focus range). Next, determine (e.g., select or design) (e.g., symmetrical) optical lenses for the first group to achieve the desired focal length along the first axis (e.g., x-axis). This may result in the second axis (e.g., y-axis) having the same focal length as the first axis (e.g., because the optical lenses in the first group are symmetrical). Since the focal length in the second axis should be different from the focal length in the first axis (e.g., 1.5 times), one or more asymmetric optical lenses are determined to achieve the desired focal length along the second axis. (In this example, the asymmetric lenses have optical power only along the second axis. In other example embodiments, the asymmetric lenses may also have optical power along the first axis, and the lenses in the first group may be determined to account for this.) Therefore, in order to change the focus of the lens module along a focusing range (e.g., from 1m to 5m), the first group may be moved along the optical axis to adjust the focus along both axes, and the asymmetric group may also be moved to correct for astigmatism along the second axis.

[0097] However, as described above, depending on the focus range, the focal power of the lens, and the available movement of the optical lens, it may be desirable to reduce the movement of one or more of the asymmetric optical lenses. To this end, as described above, one of the asymmetric optical lenses having an optical focal power P>0 along the second axis can be replaced by two or more focal power optical lenses (which are also asymmetric) having a combined optical focal power P along the second axis. Although the combined optical focal power of the focal power optical lenses along the second axis should be P, there is a degree of freedom for each focal power of the focal power optical lenses. This degree of freedom can be used to determine the focal power of one of the focal power optical lenses so that it can move the same distance as the first group to achieve the same focus change (for example, the first group and one of the focal power optical lenses are both moved 5mm to change the focus distance from 1m to 5m). In other words, the focal power of the focal power optical lens can be selected based on the movement range of the first group (to achieve the desired focus range). More specifically, the focal power of each focal power optical lens is selected so that one of the focal power optical lenses can move along the same movement range with the symmetrical group. In some embodiments, this can be achieved by allowing the power to be variable during optimization, and then limiting the range of motion, thus causing the system to converge to a certain power of the lens during optimization. Thus, if the first group moves X mm, and the power optical lens also moves X mm to achieve the same focusing action, then both can be mechanically coupled to the same structure, and moved together (e.g., using a single motor) to change the focus of the lens module (while keeping the astigmatism below a threshold along the second axis).

[0098] FIG. 7A to FIG. 7B An example embodiment is shown in which the focal optical lens is part of the first group and moves with the lenses in the first group. Specifically, Fig. 7A A cross section of a lens module 707 and an image sensor 709 is shown, wherein the lens module 707 includes a movable group 720 and a fixed group 721 of optical lenses. The movable group 720 includes a first group of one or more (e.g., symmetrical) optical lenses (individual optical lenses are not shown) and a focal length optical lens 701. As previously described, the first group can be configured so that the two axes have the same focal length. In addition, in some embodiments, the first group 721 includes an asymmetric optical lens that is configured to correct optical aberrations (e.g., chromatic aberration, higher-order aberrations, spherical aberration, and coma). The movable group 720 is configured to move (e.g., via a controller module and a motor) along an optical axis that is at Fig. 7AIn the example of the optical axis, the z-axis is the same. For example, the lenses of the movable group 720 are mechanically coupled to the same lens holder. The fixed group 721 is configured to remain fixed along the optical axis and includes an asymmetric group including the power optical lens 702. The power optical lens 702 corresponds to the power optical lens 701. For example, the combined optical power of 701 and 702 along the axis is the same as the optical power of the lens 501 along the same axis.

[0099] In addition to the movable group 720 being moved to double the focusing distance of the lens module 707 (after the movement, both the x-axis and the y-axis are focused on the same plane (eg, the sensing plane of the image sensor)), Figure 7B and Fig. 7A Similarly. Specifically, the power optical lens 701 and the symmetric group move the same distance (e.g., within manufacturing standards or capabilities) because the power optical lens 701 has a specific optical power designed, so it moves the same amount. The various positions of the movable group 720 in the lens module 707 and the corresponding focus distances can be known by the controller module (e.g., 113), so that the controller module (e.g., 113) can provide appropriate adjustment instructions to the motor (e.g., 112) to focus the imaging system at a given distance.

[0100] (iii) Simulation

[0101] 8A to 9D is a ray tracing simulation that shows the advantages of using a focal optical lens. Fig. 8A 801 to 803 and an image sensor 800. Lens 803 is symmetrical, and lenses 801 and 802 are cylindrical lenses (which are examples of asymmetric optical lenses). Fig. 8A In the example of , the lens module has a focal length of 48 mm in the horizontal direction and a focal length of 32 mm in the vertical direction. Currently, the lens module focuses at 5000 mm.

[0102] Figure 8B It is due to Fig. 8A FIG. 8 is a graph of the point spread function of light at sensor 800 with the lens module in FIG. 8 . Overall, the light is focused on a point with little or no astigmatism. In this example, the point spread function is not circular because the lens module has an elliptical aperture of 10 mm and 5 mm (for the major and minor axes of the ellipse). Among other advantages, having an elliptical aperture means that the F / # is different for each axis, and more diffraction occurs in the direction with a smaller aperture (higher F / #).

[0103] Except that lens 803 is moved 1.12 mm to change the focus of the lens module from 5000 mm to 1500 mm (lenses 801 and 802 remain fixed), Figure 8C and Fig. 8A similar. Fig.8D It is due to Figure 8C 800. As shown, because asymmetric lens 801 and asymmetric lens 802 remain fixed, the point spread function is focused in the vertical dimension, but extremely blurred in the horizontal dimension (in other words, there is astigmatism in the horizontal dimension). The astigmatism can be corrected by moving lens 802, however the distance moved will be substantially different from the distance lens 803 moves (e.g., a difference of more than 1%, or resulting in separate motors for each lens), and the movement may cause lens 802 and lens 803 to contact each other.

[0104] Fig. 9A 901 to 904 and an image sensor 900. Lens 904 is symmetrical, and lenses 901 to 903 are cylindrical lenses. Fig. 8A Compared with the lens module in Fig. 9A The lens module in includes a focal optical lens. Specifically, lens 902 and lens 903 are focal optical lenses relative to lens 802 (the combined optical focal length of lens 902 and lens 903 is the same as the optical focal length of lens 802). Fig. 8A Similar to the lens module in Fig. 9A The example lens module in has a focal length of 48 mm in the horizontal direction and a focal length of 32 mm in the vertical direction. Currently, the lens module focuses at 5000 mm.

[0105] Fig. 9B It is due to Fig. 9A FIG. 9 is a graph of the point spread function of light at sensor 900 for a lens module in FIG. 9 . Overall, the light is focused on a point with little or no astigmatism. Figure 8B The description is similar because the lens module has an elliptical aperture of 10 mm and 5 mm (for the major and minor axes of the ellipse), so Fig. 9B The point spread function in is not circular.

[0106] Except that lens 903 and lens 904 are each moved 1.12 mm to change the focus of the lens module from 5000 mm to 1500 mm (lens 901 and lens 902 remain fixed), Fig. 9C and Fig. 9A similar. Fig.9D It is due to Fig. 9CFIG. 9 is a graph of the point spread function of light at sensor 900 for a lens module in FIG. 9. As shown, the point spread function is focused in the vertical and horizontal directions. In other words, there is little or no astigmatism in the horizontal direction. The lack of such astigmatism in the horizontal direction is because asymmetric lens 903 has optical power such that lens 903 moves the same amount as symmetric lens 904, causing lens 903 to correct for any astigmatism created by the movement of lens 904.

[0107] If there is some astigmatism at the focus position (e.g. due to imperfect manufacturing), it may be astigmatism that reduces the MTF (modulus transfer function) by no more than a factor of 2 (e.g. 1.5) compared to a focus position without astigmatism. Blur should not increase by a factor greater than 2 in any dimension over the focus range.

[0108] (iv) Lens module with field lens

[0109] The following description describes lens modules having a field lens on or near an image sensor (e.g., a field lens is an optical lens that is closest to the image sensor along an optical axis). These lens modules may include one or more movable groups of optical lenses (as described above), or these lens modules may be separate from the lens modules described above.

[0110] A field lens can be a type of asymmetric optical lens. Specifically, a field lens can have the same focal length in the x-axis and y-axis in a first portion of the lens (e.g., the center of the lens), but also have an asymmetric distortion behavior in a second portion of the lens (e.g., a portion surrounding the center portion) that allows a large field of view on one axis to be squeezed onto the image sensor. The axis can be aligned with the smaller side of the image sensor and can be aligned with the smaller dimension of the asymmetric aperture. For example, a field lens produces an asymmetric distortion that squeezes a larger vertical field of view into a specific sensor height.

[0111] A field lens intentionally introduces strong distortion in either the x-direction or the y-direction so that a larger field of view can be captured by the sensor. Thus, for example, a field lens allows a sensor with a 4:2 aspect ratio to capture a scene with a 4:3 aspect ratio. In some embodiments, a field lens distorts one dimension by a factor of 2 or more compared to the other dimension to squeeze the dimensions of the field of view to a specific sensor size.

[0112] As previously described, the distortion introduced by a field lens can be asymmetric. In other words, the distortion can stretch or compress the image along one axis (e.g., the x-axis) without distorting the image along another axis (e.g., the y-axis). Additionally or alternatively, the distortion can grow with distance from the center point of the optical lens. For example, light from the center portion of the scene passing through the field lens may not be distorted. However, light from portions of the scene passing through outside the center may be slightly distorted, and light from edge portions of the scene passing through the field lens may be strongly distorted. Among other advantages, this type of field lens can provide higher angular resolution in the center portion (because that portion does not have different magnifications along the axis).

[0113] exist Fig.10 An example of a lens module 1007 having a field lens 1020 is shown in FIG. Fig.11 An example "string" distortion of a field lens (e.g., 1020) in the vertical direction is shown. The crosses show where light rays from a particular field of view intersect the sensor after passing through the field lens. The black grid lines show where light rays from the field of view would intersect the sensor after passing through an ideal distortion-free lens. Among other advantages, Fig.11 The distortion in causes the image sensor to capture a larger vertical field of view (e.g., compared to a lens module without a field lens).

[0114] Asymmetric distortion of the field lens can be achieved by placing a cylindrical or anamorphic lens (or lens group) element on the sensor or close to the sensor (e.g., within 1 mm of the sensor). During focusing, the optical lens (or lens group) can remain stationary relative to the sensor. In some embodiments, the field lens includes a material with a high refractive index (e.g., at least 1.9) to produce distortion. For example, the field lens includes one or more high refractive index glass or crystalline materials, such as zinc sulfide (Zinc Sulfide), zinc sulfide (Cleartran), or zinc selenide. In some embodiments, the field lens is located on top of the sensor and a 45 degree mirror is placed between the lens and the other optical lenses.

[0115] Although the above description of Figures 3 to 11 The lens modules are described in the context of imaging system 101 , but the lens modules may be applied to other types of imaging systems.

[0116] Example machine architecture

[0117] Now refer to Fig.12 , Fig.12 is a block diagram showing components of an example machine capable of reading instructions from a machine-readable medium and executing the instructions in a processor (or controller). Specifically, Fig.12A diagrammatic representation of a mobile device 103 in the example form of a computer system 1300 is shown. The computer system 1300 may be configured to execute instructions 1324 (e.g., program code or software) to cause the machine to perform any one or more of the methods (or processes) described herein. In alternative embodiments, the machine operates as a standalone device or as a connected (e.g., networked) device connected to other machines. In a networked deployment, the machine may operate in the capacity of a server or a client user machine in server-client user network environment, or as a peer machine in a peer-to-peer (or distributed) network environment.

[0118] The machine may be a stand-alone camera having a processing component with a processor and memory as described below. The machine may also be part of a system that includes a camera coupled to a server computer, a client computer, a personal computer (PC), a tablet PC, a set-top box (STB), a smart phone, an Internet of Things (IoT) application, or any machine that is capable of executing instructions 1324 (sequentially or otherwise) specifying actions to be taken by the machine and that may have a small volume area (within which the imaging system described herein is incorporated). In addition, while only a single machine is shown, the term "machine" should also be considered to include any collection of machines that individually or collectively execute instructions 1324 to perform any one or more of the methods discussed herein. For example, the instructions may be for controlling the instructions 1324 for performing any one or more of the methods discussed herein with respect to FIGS. 1 to 13. Fig.11 Imaging system components and / or image processing system instructions are described.

[0119] The example computer system 1300 includes one or more processing units, typically processors 1302. The processor 1302 is, for example, a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), a controller, a state machine, one or more application specific integrated circuits (ASICs), one or more radio frequency integrated circuits (RFICs), or any combination of these. The computer system 1300 also includes a main memory 1304. The computer system may include a storage unit 1316. The processor 1302, the memory 1304, and the storage unit 1316 communicate via a bus 1308.

[0120] In addition, the computer system 10106 may include a static memory 1306, a display driver 1310 (e.g., for driving a plasma display panel (PDP), a liquid crystal display (LCD), or a projector). The computer system 1300 may also include an alphanumeric input device 1312 (e.g., a keyboard), a cursor control device 1314 (e.g., a mouse, a trackball, a joystick, a motion sensor, or other pointing tool), a signal generating device 1318 (e.g., a speaker), and a network interface device 1320, which are also configured to communicate via the bus 1308.

[0121] The storage unit 1316 includes a machine-readable medium 1322 on which are stored instructions 1324 (e.g., software) embodying any one or more of the methods or functions described herein. The instructions 1324 may also reside, completely or at least partially, within the main memory 1304 or within the processor 1302 (e.g., within a cache memory of the processor) during execution thereof by the computer system 1300, the main memory 1304 and the processor 1302 also constituting machine-readable media. The instructions 1324 may be sent or received over a network 1326 via the network interface device 1320.

[0122] Although the machine-readable medium 1322 is shown as a single medium in the example implementation, the term "machine-readable medium" should be taken to include a single medium or multiple media (e.g., a centralized database or distributed database, or associated caches and servers) that can store the instructions 1324. The term "machine-readable medium" should also be taken to include any medium that can store the instructions 1324 for execution by a machine and cause the machine to perform any one or more of the methods disclosed herein. The term "machine-readable medium" includes, but is not limited to, data repositories in the form of solid-state memories, optical media, and magnetic media.

[0123] Additional considerations

[0124] The imaging systems and associated algorithms described herein have several advantages over conventional cameras in terms of size, volume, shape, and performance. For example, among other advantages, the imaging systems described herein can produce images with greater resolution than conventional systems of comparable size.

[0125] Throughout the specification, multiple instances can be implemented as parts, operations or structures described as single instances. Although the individual operations of one or more methods are shown and described as separate operations, one or more of the individual operations can be performed simultaneously and do not require the operations to be performed in the order shown. The structure and function presented as separate parts in the example configuration can be implemented as a combined structure or parts. Similarly, the structure and function presented as a single part can be implemented as a separate part. These and other variations, modifications, additions and improvements fall within the scope of this paper theme.

[0126] Certain embodiments are described herein as including logic or multiple components, modules, or mechanisms, such as controller module 113. Modules may constitute software modules (e.g., code implemented on a machine-readable medium or in a transmission signal) or hardware modules. A hardware module is a tangible unit that is capable of performing certain operations and may be configured or arranged in a certain manner. In example embodiments, one or more computer systems (e.g., independent client or server computer systems) or one or more hardware modules (e.g., processors or groups of processors) of a computer system may be configured by software (e.g., an application or application portion) as hardware modules that operate to perform certain operations as described herein.

[0127] In various embodiments, hardware modules can be implemented mechanically or electronically. For example, a hardware module may include dedicated circuitry or logic that is permanently configured to perform certain operations (e.g., as a dedicated processor, such as a field programmable gate array (FPGA) or an application-specific integrated circuit (ASIC)). A hardware module may also include programmable logic or circuitry that is temporarily configured by software to perform certain operations (e.g., as contained in a general-purpose processor or other programmable processor). It will be understood that the decision to mechanically implement a hardware module in a dedicated and permanently configured circuitry or in a temporarily configured circuitry (e.g., configured by software) may be driven by cost and time considerations.

[0128] The various operations of the example methods described herein may be performed, at least in part, by one or more processors, such as processor 1302, that are temporarily configured (e.g., by software) or permanently configured to perform the relevant operations. Such processors, whether temporarily configured or permanently configured, may constitute processor-implemented modules that operate to perform one or more operations or functions. In some example embodiments, the modules mentioned herein may include processor-implemented modules.

[0129] The one or more processors may also operate to support execution of related operations in a "cloud computing" environment or as "software as a service" (SaaS). For example, at least some operations may be performed by a group of computers (as an example of a machine including a processor), which can be accessed via a network (e.g., the Internet) and via one or more appropriate interfaces (e.g., an application program interface (API)).

[0130] The execution of certain operations in the operation can be distributed among one or more processors, not resident only in a single machine, but deployed across multiple machines. In some example embodiments, one or more processors or processor-implemented modules can be located in a single geographic location (e.g., in a home environment, an office environment, or a server farm). In other example embodiments, one or more processors or processor-implemented modules can be distributed across many geographic locations.

[0131] Some parts of this specification are presented in terms of algorithms or symbolic representations of operations on data stored as bits or binary digital signals in machine memory (e.g., computer memory). These algorithms or symbolic representations are technical examples used by ordinary technicians in the field of data processing to convey the essence of their work to other technicians in the field. As used in this article, an "algorithm" is a self-consistent sequence of operations or similar processes that obtain a desired result. In this context, algorithms and operations involve physical manipulation of physical quantities. Typically, but not necessarily, such quantities can take the form of electrical, magnetic, or optical signals that can be stored, accessed, transmitted, combined, compared, or otherwise manipulated by a machine. Sometimes, mainly for general reasons, it is more convenient to use words such as "data", "content", "bit", "value", "element", "symbol", "character", "item", "number", "digital", etc. to refer to such signals. However, these words are only convenient labels and will be associated with appropriate physical quantities.

[0132] Unless expressly stated otherwise, discussions herein using words such as "process," "compute," "calculate," "determine," "present," "display," and the like may refer to the actions or processes of a machine (e.g., a computer) that manipulates or transforms data represented as physical (e.g., electronic, magnetic, or optical) quantities within one or more memories (e.g., volatile memory, non-volatile memory, or any suitable combination thereof), registers, or other machine components that receive, store, send, or display information.

[0133] As used herein, "one embodiment", "some embodiments" or "embodiments" means that a particular element, feature, structure or characteristic described in conjunction with the embodiment is included in at least one embodiment. The phrase "in one embodiment" appearing in various places in this specification does not necessarily all refer to the same embodiment.

[0134] Some embodiments may be described using the expressions "coupled" and "connected" and their derivatives. For example, some embodiments may be described using the term "coupled" to indicate that two or more elements are in direct physical or electrical contact. However, the term "coupled" may also mean that two or more elements are not in direct contact with each other, but still cooperate or interact with each other. The embodiments are not limited to this context.

[0135] As used herein, the terms "comprises," "comprising," "includes," "including," "has," "having," or any other variation thereof, are intended to cover a non-exclusive inclusion. For example, a process, method, article, or apparatus that includes a list of elements is not necessarily limited to only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. In addition, unless expressly stated to the contrary, "or" refers to an inclusive or rather than an exclusive or. For example, conditions A or B satisfy any of the following conditions: A is true (or exists) and B is false (or does not exist); A is false (or does not exist) and B is true (or exists); and both A and B are true (or exist).

[0136] In addition, "a" or "an" is used to describe the elements and components of the embodiments herein. This is only for convenience and to give a general sense of the invention. Unless otherwise clearly indicated, the description should be interpreted as including one or at least one, and the singular also includes the plural.

[0137] After reading this disclosure, those skilled in the art will understand additional alternative structural and functional designs for systems and processes for forming combined images through the principles disclosed herein. Therefore, although specific embodiments and applications have been shown and described, it should be understood that the disclosed embodiments are not limited to the exact constructions and components disclosed herein. Various modifications, changes and variations that are obvious to those skilled in the art may be made to the arrangement, operation and details of the methods and apparatus disclosed herein without departing from the spirit and scope defined in the appended claims.

Claims

1. An imaging system, comprising: An image sensor in the xy plane of a Cartesian coordinate system; A lens module, the lens module being configured to direct light onto the image sensor, the lens module comprising: a fixed set of optical lenses, including a first set of asymmetric optical lenses; and a movable group of optical lenses, including a symmetric optical lens and a second group of asymmetric optical lenses; and a controller coupled to the movable group and configured to focus the imaging system within a focus range by generating an adjustment signal to move the movable group along an optical axis, the lens module focusing light on the image sensor along both an x-axis and a y-axis of the image sensor and within the focus range, and the first group of asymmetric optical lenses and the second group of asymmetric optical lenses being positioned such that astigmatism is below a threshold within the focus range.

2. The imaging system according to claim 1, wherein: A combined optical power P of the first group of asymmetric optical lenses and the second group of asymmetric optical lenses along the x-axis or the y-axis has a positive value.

3. The imaging system according to claim 2, wherein: One of the groups of asymmetric optical lenses has a negative combined optical power value along the x-axis or the y-axis, and the other group of asymmetric optical lenses has a positive combined optical power value along the x-axis or the y-axis.

4. The imaging system according to claim 3, wherein: One of the sets of asymmetric optical lenses has a combined optical power value of -nP, and the other set of asymmetric optical lenses has a combined optical power value of +(n+1)P, where n is a real number greater than zero.

5. The imaging system according to claim 1, wherein: A combined optical power P of the first group of asymmetric optical lenses and the second group of asymmetric optical lenses along the x-axis or the y-axis is 10 diopters to 50 diopters.

6. The imaging system according to claim 1, wherein: Due to the first group of asymmetric optical lenses and the second group of asymmetric optical lenses, the lens module has different magnifications along the x-axis and the y-axis.

7. The imaging system of claim 1, wherein: The fixed group is configured to be fixed relative to the image sensor.

8. The imaging system of claim 1, wherein: The lens module is configured to move the movable group along a z-axis.

9. The imaging system of claim 1, wherein: The fixed group includes only the first asymmetric optical lens.

10. The imaging system of claim 1, wherein: The fixed group does not include a symmetrical optical lens.

11. The imaging system of claim 1, wherein: The movable group includes one or more additional asymmetric optical lenses configured to correct optical aberrations.

12. The imaging system of claim 1, wherein: The symmetric optical lens is closer to the image sensor along the optical axis than the first group of asymmetric optical lenses and the second group of asymmetric optical lenses.

13. The imaging system of claim 1, wherein: The movable group is closer to the image sensor along the optical axis than the fixed group.

14. The imaging system of claim 1, wherein: A distance along the optical axis from the first group of asymmetric optical lenses to an aperture stop of the imaging system is less than half of a total length of the lens module along the optical axis.

15. The imaging system of claim 1, wherein: At least one of the following: The first group of asymmetric optical lenses includes two or more asymmetric optical lenses; or The second group of asymmetric optical lenses includes two or more asymmetric optical lenses.

16. The imaging system of claim 1, wherein: The controller is configured to generate the adjustment signal to move the movable group of optical lenses together.

17. The imaging system of claim 1, wherein: The controller is configured to generate the adjustment signal to move each optical lens in the movable group by substantially the same amount.

18. The imaging system of claim 1, wherein: The controller is configured to generate the adjustment signal to move the symmetric optical lens and the second set of asymmetric optical lenses separately.

19. The imaging system of claim 1, wherein: The lens module also includes a field lens coupled to the image sensor, the field lens introducing asymmetric distortion to light passing through the field lens.

20. The imaging system of claim 1, further comprising an asymmetric aperture having a major axis and a minor axis.

21. An imaging system comprising: Image sensor; A lens module configured to direct light onto the image sensor, the lens module having different magnifications along a substantially orthogonal axis parallel to a sensing plane of the image sensor, the lens module comprising: a first set of one or more optical lenses including symmetric optical lenses; a second set of one or more optical lenses including a first asymmetric optical lens having a first optical power along a first axis parallel to the plane; and a third group of one or more optical lenses including a second asymmetric optical lens having a second optical power along a second axis parallel to the plane; and A controller is configured to generate an adjustment signal to focus an image onto the plane of the image sensor by adjusting a distance between the first group and the second group along an optical axis of the imaging system.

22. The imaging system of claim 21, wherein: The controller is configured to generate the adjustment signal to move the first group and the second group by substantially the same amount to focus the image onto the plane.

23. The imaging system of claim 21, wherein: The controller is configured to generate the adjustment signal to move the first group and the second group by substantially different amounts to focus the image onto the plane.

Images