Apparatus, computer-readable storage medium and method for providing autofocus capability based on object distance information

Abstract

Device comprising: a lens system (110) for receiving light from an environment external to the device; Distance evaluation circuits (150) configured to identify corresponding distances to each of a plurality of objects located within a field of view (320) of the lens system (110), including a first object and a second object; Selection circuits (160) coupled to the distance evaluation circuits (150) for determining a focus of the lens system (110), the selection circuits (160) comprising logic that, when executed, causes the device to perform operations comprising: Determining a first count of each of the plurality of objects that would appear in focus while the first object is at a first near depth of field based on a first focal configuration and a first aperture (122), the first count representing a total number of the plurality of objects that would appear in focus while the first object is at the first near depth of field; Determining a second count of each of the plurality of objects that would appear in focus while the second object is at a second near depth of field based on a second focal configuration and the first aperture (122), the second count representing a total number of the plurality of objects that would appear in focus while the second object is at the second near depth of field; Comparing a first score based on the first count and a second score based on the second count; and providing, based on the comparing, a signal (162) indicating a preference between the first focal configuration or the second focal configuration; a focus control (170) coupled to adjust the lens system (110) based on the signal (162); and an image sensor (124) optically coupled to capture an image received with the lens system (110), where either: the first score is calculated based on weighting values assigned to the plurality of objects that would appear in focus while the first object is at the first near depth of field based on a location of the respective object in the field of view (320), and the second score is calculated based on weighting values assigned to the plurality of objects that would appear in focus while the second object is at the second near depth of field based on a location of the respective object in the field of view (320), and / or the first score is calculated based on weight values assigned to the multiple objects that would appear in focus while the first object is at the first near depth of field based on an object type of the object, and the second score is calculated based on weight values assigned to the multiple objects that would appear in focus while the second object is at the second near depth of field based on an object type of the object.

Description

BACKGROUND1. Technical FieldThis disclosure relates generally to the field of optics, and more particularly, but not exclusively, to the operation of an image sensor having a variable focus lens system.2. General State of the ArtIn optics, the depth of field ("DOF") is the region in a scene between a closer distance and a farther distance, and objects in the image may appear to be acceptable sharp between these distances. A fixed focus lens can focus only precisely to a single depth within a scene and as such, sharpness gradually decreases on either side of this focus distance. Objects falling within the depth of field are considered to have acceptable sharpness.Digital imaging devices such as digital cameras often include a lens assembly that focuses image light onto an image sensor that measures the image light and generates an image based on the measurements. A variable focus lens may adjust its focus distance so that it may be focused to different distances at different times. This allows the imaging device to translate the depth of field to focus on objects at a plurality of distances. Conventional imaging devices often support autofocus functionality to facilitate changing focus distance. As the number and variety of form factors for imaging devices continue to grow over time, there is expected an increased demand for solutions that provide responsive and / or otherwise efficient autofocus functionality.US 2009 / 0 047 010 A1 describes an imaging apparatus including an autofocus section that moves at least a part of an imaging lens as a focusing lens and focuses the imaging apparatus on an object, a shot range setting section that sets a predetermined range including an object distance to the object obtained by the autofocus section as a shot range for photographing, a series shot section that continuously photographs at a plurality of focused positions in the shot range while moving and stopping the focusing lens according to input of a shot instruction, and a display section that displays an image of the object focused by the autofocus section and the shot range set by the shot range setting section.JP 2011-237 713 A relates to an image pickup apparatus that captures a plurality of photographic images in which the focal position of a lens is different in each capturing range set by dividing a capturing range.US 2009 / 0 015 681 A1 describes an apparatus that may include logic for capturing an image, logic for recognizing multiple faces in the image, logic for calculating a distance associated with each face, logic for calculating a depth of field based on the distance associated with each face, and logic for calculating focus and exposure settings for capturing the image based on the depth of field associated with the plurality of faces.BRIEF DESCRIPTIONThere is disclosed an apparatus according to claims 1 to 8, a non-transitory computer readable storage medium according to claims 9 to 13 and a method according to claims 14 to 15.BRIEF DESCRIPTION OF THE DRAWINGSThe various embodiments of the present invention are illustrated by way of example and not by way of limitation in the figures of the accompanying drawings, in which: FIG. 1 is a functional diagram illustrating elements of a system for determining a focal configuration according to an embodiment. FIG. 2 is a flow diagram illustrating elements of a method of operating an image sensor device according to an embodiment. FIG. 3A is a top view of an environment including a device to provide autofocus capability, according to an embodiment. FIG. 3B is a diagram illustrating processes performed to determine a focus configuration, according to an embodiment. FIG. 4 is a flow diagram illustrating elements of a process to determine a focus configuration according to an embodiment. FIG. 5 shows various views illustrating features of an environment in which an image sensor device is to provide autofocus capability, according to an embodiment.DETAILED DESCRIPTIONEmbodiments described herein provide various techniques and mechanisms for determining a focus type to be provided with a lens system. Such a lens system may be configured in some embodiments based on a determination as to whether objects observable by the lens system could be different in focus or out of focus, such as given a particular aperture and / or other operating characteristic of an image sensor optically coupled to the lens system. Where it is determined that one lens system configuration results (or would result) in more objects being in focus compared to another lens system configuration, according to one embodiment, a signal may be generated to indicate a relative preference of the one lens system configuration over the other lens system configuration.In the following description, numerous specific details are set forth in order to provide a thorough understanding of the embodiments. However, one skilled in the art will recognize that the techniques described herein may be practiced without one or more of the specific details or with other methods, components, materials, etc. In other instances, well-known structures, materials, or operations are not shown or described in detail to prevent certain aspects from backing into the background. References throughout this specification to "one embodiment" or "an embodiment" mean that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. Thus, the use of the phrase "in one embodiment" in various places throughout this specification does not necessarily always refer to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.In conventional digital imaging, if there is only one subject in a scene, an autofocus (AF) algorithm typically adjusts a lens position to adjust a focus distance with respect to that one subject. Some embodiments are based on a discovery by the inventors that under certain circumstances, this approach may be less than optimal for scenes with multiple objects, such as human faces located at different distances. Such embodiments improve differently in conventional autofocus techniques by providing mechanisms that recognize that one focus field may result in a greater number and / or better arrangement of objects in focus compared to another focus field.As used herein, "field of view" refers to the portion of an environment that is observable by a lens system. A field of view can relate, for example, to that part of a three-dimensional environment whose image can be captured as a two-dimensional image via a specific lens system which is directed at the section. The term "focus field" (also "focal field" or "focal field") refers to the part of the field of view in which an object or objects will be sufficiently in focus according to some predetermined criteria as observed by the lens system. A given focus field-which may for example partially depend on a given aperture of the imaging system-comprises a corresponding focus distance and a corresponding depth of field.A "focal distance" is a distance from a reference point (e.g., a center of a lens of the lens system) to the center of the focal field. A "depth of field" is a total depth of the focus field (e.g., as measured along a line of sight extending to / from the reference point). The depth of field, also known as the "focus distance range", extends between a near depth of field and a far depth of field. The term "near depth of field" refers to a distance to a nearest edge of the depth of field as measured from a reference point, such as a center of a lens of the lens system. Similarly, "far depth of field" herein refers to a distance from the reference point to a farthest edge of the depth of field.For many optical systems, for example, the relationship between focus distances and near depth of field (Dn) can be generally represented by the following equation: where the hyperfocus distance (H) is a closest distance at which a lens system can be focused while objects at infinity are held reasonably sharp. When the lens is focused at the hyperfocus distance, usually all objects will be reasonably sharp at distances from half the hyperfocus distance to infinity. The hyperfocal distance of an imaging system may be different for different settings (e.g., different diaphragms) of that imaging system.The relationship between Dn, s and H can also be generally represented by the following equations: and the relationship between the far n depth of field (Df), s and H can be generally represented by the following equations:However, a variety of additional or alternative criteria, such as including one or more equations adjusted by conventional imaging techniques, may be used to identify relationships between different ones of Dn, Df, H, and s.The term "focal configuration" as used herein refers to a given configuration of a lens system, such as one of several possible configurations, to facilitate the provision of a corresponding focus field. The focus field may be that of the lens system itself. Alternatively, the focus field may be an entire focus field provided by the lens system in combination with one or more other devices (such as including other one or more lenses, a particular aperture structure, circuitry for executing image focus software, and / or the like).Embodiments described herein determine, e.g., automatically, a relative preference of one focal configuration over another focal configuration, such determination based on an evaluation of an object or objects that could be located in a focus field. "Identified object" or "identified objects" herein refer differently to one or more objects located in a field of view of a lens system and each identified (e.g., including differing from each other) as having a corresponding distance from any reference such as a point located in or on a lens system, unless otherwise indicated. Such one or more objects may include only a subset of a larger plurality of objects observable by the lens system (e.g., where the subset includes only objects that each occupy at least a minimum portion of the field of view).As used herein, an "object set in focus" refers to a set of those one or more identified objects that are or would be in focus as observed during a particular focal configuration with the lens system. Different focal configurations of the lens system may thus correspond to different corresponding object rows in focus. Of the one or more objects in a given series of objects in focus, an object that is closest to the lens system may be referred to as a "closest object in focus", whereas an object that is furthest from the lens system is referred to as a "furthest object in focus. Accordingly, different object rows in focus may include different corresponding closest objects in focus and / or farthest objects in focus. A "count of objects in focus" (also referred to herein as an "object count" for brevity) herein refers to a total number of the one or more objects in an object row in focus.FIG. 1 illustrates elements of a system 100 according to an embodiment for determining a focal configuration to be implemented for use in an image acquisition operation. The system 100 is only one example of an embodiment configured to determine a preference of one focal configuration over another focal configuration based on corresponding distances of objects located within a field of view. Such a preference may be determined, for example, based on a corresponding calculated score of the focal configurations. The scores may each be equal to or otherwise be based on a corresponding count of objects in focus associated with a corresponding focus field.In the illustrated illustrative embodiment, the system 100 includes a lens system 110 including one or more lenses, such as the illustrated illustrative lens 112, to receive light 105 from an external environment. The lens system 110 may include any of a variety of optical devices having adjustable focus spacing capability. Such an optical device may be controlled based on the techniques described herein using one or more focus adjustment mechanisms adapted from conventional autofocus technology.The lens system 110 may be optically coupled to the direct light 105 from the external environment towards an image sensor 120 of the system 100, wherein, for example, light output by the lens system 110 is focused by a shutter 122 onto a pixel array 124. The pixel array of 124 may include complementary metal oxide semiconductor (CMOS) pixels and / or any of a variety of other pixels adapted from conventional image sensor techniques. Some embodiments are not limited to a particular pixel array architecture for use in generating image data based on the light 105. In some embodiments, a configuration of the lens system 110 is to be determined given a particular size of the aperture 122, for example, wherein the image sensor 120 is a fixed aperture device, or wherein a focal configuration is to be selected from a plurality of possible focal configurations for use in combination with a particular size of the aperture 122.For example, the system 100 may further include a distance sensor 140 configured to operate as a distance meter for detecting objects in a field of view observable with the lens system 110. Detection of object distances with the distance sensor 140 may include one or more operations adapted by conventional distance measurement techniques not described herein to avoid obscuring features of various embodiments. By way of illustration and not limitation, the proximity sensor 140 may provide the functionality of a laser range finder, ultrasonic range finder, or infrared range finder. Other means of ranging are possible, such as light detection and ranging (LIDAR), radio-based ranging and ranging (RADAR), microwave ranging, etc.The proximity sensor 140 may be coupled to output the signals 142 to the proximity evaluation circuits 150 of the system 100. The distance evaluation circuits 150 may include logic, such as an application specific integrated circuit (ASIC), processor circuits, state machine, and / or other semiconductor hardware, configured to detect that multiple objects are in a field of view observable by the lens system 110 based on the signals 142. Some or all of such objects may be distinguishable from each other by different corresponding distances from the system 100.Ranging with the ranging sensor 140 and the ranging circuits 150 may include active detection techniques, passive detection techniques (e.g., including phase demodulation, contrast measurement, and / or the like), or a combination thereof. In one embodiment, the distance evaluation circuits 150 may identify, for each object of a plurality of objects, a corresponding distance to that object relative to a reference location in or on the system 100. This identification may be based, for example, on a threshold response to a laser and / or other ranging signal output by ranging sensor 140. Such a minimum threshold response may limit the identified plurality of objects to those objects that each occupy at least a minimum threshold of the field of view. Alternatively or additionally, such a minimum threshold response may limit the identified plurality of objects to those objects in the field of view that are within a maximum threshold distance of system 100.Although some embodiments are not limited in this regard, the distance sensor 140 may provide a direction range measurement functionality that identifies a distance to a corresponding object for different corresponding portions of the field of view, at least a portion of which occupies the portion of the field of view. For example, the proximity sensor 140 may be operated to sweep the field of view sequentially (or otherwise), with respective response signals received by the proximity sensor 140 in sequence thus each associated with a different respective portion of the field of view. In such an embodiment, distance evaluation circuits 150 may correspond to different object distances, each of which has a different corresponding portion of the field of view.The system 100 may further include a selection circuit 160 coupled to receive, from the distance assessment circuits 150, information specifying or otherwise displaying the respective distances of the plurality of objects of system 100. The selection circuits 160 may include logic, such as including an ASIC, processor circuits, and / or the like, to determine a focal configuration to be implemented with the lens system 110 based on such object distances. Such determining may include the selection circuit 160 identifying a relative preference of a first focal configuration over a second focal configuration. This preference may be based on, for example, a determination that the first focal configuration would result in a greater number and / or better placement (indicated by a score or other metric) of objects in focus compared to the second focal configuration. Access thereto describing one or more relationships between focus distance, near depth of field (Dn), further depth of field (DF), hyperfocus distance (H), and / or any of various other optical characteristics to be provided with the lens system 100. By way of illustration and not limitation, the selection circuits 160 may include or be coupled to a memory 130 that is preprogrammed with such reference information - e.g., from a manufacturer, dealer, computer network service, or other agent. Based on object distance data from the distance evaluation circuits 150, the selection circuits 160 may access reference information in memory 130 to select, calculate, and / or otherwise determine, for a given distance from such an object, a total number of objects that are (or would) in focus during a corresponding focal configuration of the lens system 110. For example, the focal configuration may correspond to said object being at a near depth of field to be provided with the lens system 110.Based on an assessment of multiple possible focal configurations, the selection circuits 160 may output a signal 162 identifying or otherwise indicating a focal configuration determined to be preferred over at least one alternative focal configuration. In response to signal 162, a focus control (FC) 170 of system 100 may adjust or otherwise configure a focus field to be implemented with lens system 110. FC 170 may include any of a variety of one or more hardware and / or software mechanisms to change an effective focal distance provided with lens system 110. By way of illustration and not limitation, FC 170 may include a motor to move lenses of lens system 110 relative to each other and / or relative to pixel array 124. Alternatively or additionally, the FC 170 may include logic (e.g., including an ASIC, processor executing software, and / or the like) to be implemented, for example, at least partially into an effective focus field via image processing computations. Based on the signal 162, the FC 170 may implement a focal configuration that provides a particular depth of field with the lens system 110. During such a focal configuration, the image sensor 120 may be operated, e.g., in response to the selection circuits 160 and / or FC 170, to capture an image of the external environment.Although some embodiments are not limited in this regard, the distance evaluation circuits 150 may further include or be coupled to image recognition circuits (not shown) configured to receive and process image information generated by the pixel array 124 based on light received via the lens system 110. These image recognition circuits may be preprogrammed (or otherwise accessed) with other reference information describing one or more classes of objects, for example. Based on such other reference information, the image recognition circuitry may evaluate signals from the pixel array 124 to determine whether any region of the field of view includes a representation of an object that belongs to a predefined object class. Some examples of object classes include, but are not limited to, an eye class, mouth class, head class, motor vehicle class, building class, and / or the like. Identification of one or more objects of an object class may include operations adapted by conventional image recognition techniques. In one embodiment, one or more object distances differently indicated by signals 142 may each be associated with a corresponding object identified as belonging to a corresponding object class.In summary, the apparatus (system 100) shown in FIG. 1 may include a combination of at least some of the following elements: a lens system 110 to receive light from an environment external to the apparatus; distance assessment circuits 150 configured to identify corresponding distances to each of a plurality of objects including at least a first object and a second object, wherein the distance assessment circuits 150 may be coupled to a distance sensor 140 to process output signals 142 of the distance sensor 140 and determine the distance of a corresponding object based on the assessment of the signals 142; Selection circuits 160 coupled to the distance evaluation circuits 150 to determine a focus of the lens system 110, the selection circuits 160 comprising logic that, when executed, causes the apparatus (system 100) to perform operations comprising: determining a first count of each of the plurality of objects to be in focus while the first object is at a first near depth of focus Dn based on a first focal configuration (e.g., a given configuration of the lens system 110) and based on a first aperture 122 (e.g., a size of the first aperture 122 that optically couples light to a pixel array 124 of an image sensor 120 with the lens system 110 to capture an image received with the lens system 110); determining a second count of each of the plurality of objects to be in focus while the second object is at a second near depth of field Dn based on a second focal configuration and on the first aperture; comparing a first score based on the first count and a second score based on the second count; and providing, based on the comparing, a signal 162 indicative of a preference between the first focal configuration or the second focal configuration.Here, the corresponding first or second score of a first / second count may be based on a value of the first count or a value of the second count, respectively. For example, a corresponding score may be derived from an associated weighted count, wherein different weights may be assigned to the individual objects included in the count. If such weights are neglected, then the first score and the second score may be equal to the first count and the second count, respectively. Otherwise, a score may be based at least in part on a weighted value assigned to an object in the field of view.Furthermore, the apparatus may include a focus controller 170 coupled to adjust the lens system 110 based on the signal 162.The operations that may be performed by the device (system 100) also define a method for providing automatic focusing capability based on object distance information, which for one embodiment is described in more detail below, and which may be implemented by executing corresponding instructions stored on a non-transitory computer readable storage medium.FIG. 2 illustrates elements of a method 200 for determining a focal configuration of the lens system according to an embodiment. To illustrate certain features of various embodiments, the method 200 is described herein with reference to an example scenario illustrated in FIGS. 3A, 3B. FIG. 3A shows a top view of an environment 300 in which an image sensor device 310 is to be operated, according to an embodiment. FIG. 3B shows a view 350 of various distances as projected onto a single row 360 from the image sensor device 310 to corresponding objects in the environment 300. The method 200 may be performed with one or more components of the image sensor device 310 - e.g., wherein the image sensor device 310 includes some or all of the features of the system 100. However, other embodiments include method 200 by being performed by any of a variety of other image sensor devices having features described herein.In one embodiment, the method 200 includes, at 210, identifying corresponding distances to each of a plurality of objects in a field of view of a lens system. For example, as shown in FIG. 3A, the image sensor device 310 may be positioned (localized and aligned) such that some multiple objects, such as the illustrated six objects A through F, are each in the field of view 320 (e.g., between the lines of sight 322, 324) observable by a lens system 312 of the image sensor device 310. Objects A to F do not limit embodiments, and image sensor device 310 may be configured to execute method 200 based on more, fewer, and / or differently arranged objects.Positions of objects in the field of view 320 may be identified at least in part with reference to, for example, a polar coordinate system (e.g., part of a cylindrical or spherical coordinate system) that includes a distance dimension x and a radial dimension θ. In the illustrative scenario shown, the field of view 320 has located therein the object A at the location (X1, θ4), the object B at the location (x2, θ3), the object C at the location (x3, θ2), the object D at the location (x4, θ5), the object E at the location (x5, θ6), and the object F at the location (x6, θ1). In the exemplary scenario shown in FIG. 3A, objects A through F are each in a two-dimensional plane. However, it should be appreciated that some or all of these objects may be located differently at different vertical heights in a three-dimensional space-e.g., where the vertical height component of the location of an object may result in an additional distance between the object and the image sensor 310. The determination at 210 may include, for example, identifying the distances x1, x2, x3, x4, x5, x6- e.g., where such identifying is performed with distance evaluation circuits 150 based on the signals 142.The method 200 may further include performing a first determination of a focal configuration to be implemented (with the lens system 312 in the example of FIG. 3A ). Such determination, also referred to herein as a "first focus determination" for brevity, may determine a focus of the lens system-e.g., including operations to provide a comparable assessment of at least two focal configurations based on corresponding counts of objects in focus. For example, the first focus determination at 220 may include a first count of each of the plurality of objects to be in focus while a first object of the plurality of objects is at a first near depth of focus due to a first focal configuration of the lens system and a first aperture (i.e., a particular aperture size, which may be fixed or alternatively adjustable). The first count may represent a first total number of each of the plurality of objects that would appear in focus if observed with the lens system, while the first focus field is implemented with both the first focal configuration and the first aperture.The first focus determination may further include, at 230, determining a second count of each of the plurality of objects to be in focus while a second object of the plurality of objects is at a second near depth of field due to a second focal configuration and the first aperture. The second count may represent a second total number of each of the plurality of objects that would appear in focus if observed with the lens system, while the second focus field is implemented with a second focal configuration of the lens system and with the first aperture. The determining at 230 may include counting a total number of objects of a second in-focus object row corresponding to the second focal configuration.FIG. 3B illustrates an example of a focal configuration determination (such as that comprising determining at 220 and 230) that comprises counting, for each of a plurality of focal configurations, corresponding counts of objects of an in-focus object row corresponding to that focal configuration. Such a count (referred to herein as "object-in-focus count") may include setting a near depth-of-focus variable to be equal to a distance from a particular object distance, and calculating or otherwise determining a far depth-of-focus value corresponding to, e.g., to match, the near depth-of-focus value. The distances identified at 210 may then be evaluated to determine which objects are between the near depth of field and the corresponding far depth of field.For example, as shown in view 350, a focal configuration determination may perform a first object-in-focus count for a focus field D 1, where a near depth of field of focus of D 1 is to be at the same distance (x1-x0) from image sensor 310 as object A. The first object-in-focus count may determine that only one of objects A through F, i.e., object A, is (or would be) in focus when lens system 312 has a first focal configuration to facilitate D 1. A second object in focus count may be performed for a focus field D 2, where a near depth of field of focus of D 2 is to be at the same distance (x2-x0) from the image sensor 310 as object B. The second object in focus count may determine that a total of three of the objects - i.e., objects B, C, and D - are or would be in focus if the lens system 312 has a second focal configuration to facilitate D 2.The focal configuration determination may further perform a third object-in-focus count for a focus field D 3, where a near depth of focus of D 3 is to be at the same distance (x3-x0) from the image sensor 310 as object C. The third object-in-focus count may determine that a total of two of the objects, i.e., objects C and D, are or would be in focus if the lens system 312 has a third focal configuration to facilitate D 3. A fourth object in focus count may be performed for a focus field D4, where a near depth of field of focus of D4 is to be at the same distance (x4-x0) from the image sensor 310 as object D. The fourth object in focus count may determine that a total of two of the objects - i.e., objects D and E - are or would be in focus if the lens system 312 has a fourth focal configuration to facilitate D4.The focal configuration determination may further perform a fifth object-in-focus count for a focus field D 5, where a near depth of focus of D 5 is to be at the same distance (x5-x0) from the image sensor 310 as object E. The fifth object-in-focus count may determine that only one object - i.e., object E - is or would be in focus if the lens system 312 has a fifth focal configuration to facilitate D 5. A sixth object in focus count may be performed for a focus field D 6, where a near depth of field of focus of D 6 is to be at the same distance (x6-x0) from the image sensor 310 as object F. The sixth object in focus count may determine that only one object - i.e., object F - is or would be in focus if the lens system 312 has a sixth focal configuration to facilitate D 6. The corresponding depth of field of D1-D6may be substantially equal - e.g., within 10% of each other and, in some embodiments, within 5% of each other.The focal configuration determination performed by the method 200 may further include performing, at 240, a comparison of a first score based on the first count determined at 220 and a second score based on the second count determined at 230. For example, the first score and the second score may be equal to the first count and the second count, respectively. In another embodiment, a score may be based at least in part on a weighted value assigned to an object in the field of view. The assignment of such a weighted value can be based, for example, on a location of said object in the field of view. By way of illustration and not limitation, a weight value may be assigned to an object based at least in part on a location of the object relative to a reference point or line of reference (e.g., a center, centerline, edge, and / or corner) of the field of view. Alternatively or additionally, such a weight value may be assigned to the object based at least in part on its position in the field of view relative to one or more other objects that are also in the field of view. Such a weight value may additionally or alternatively be assigned based at least in part on an object class type identified by image recognition processing as corresponding to the object.Based on a result of the comparison performed at 240, the method 200 may further include, at 250, providing a signal indicative of a preference between the first focal configuration or the second focal configuration. For example, the signal may specify or otherwise indicate that the first focal configuration is preferable than the second focal configuration. The signal provided at 250 may indicate that the first focus field to be provided with the first focal configuration is to result in a greater number of objects - and / or a better weighted score for objects in focus compared to a second focus field that could otherwise be provided by the second focal configuration.With continued reference to the example scenario shown in FIG. 3B, at 250, the signal may indicate a preference for the focal configuration that facilitates D 2 over the focal configuration that facilitates D 1. Such a signal may identify the focal configuration that is to result from multiple such configurations in the largest number of objects in focus. In some embodiments, the signal provided at 250 may be generated independently of any count of objects to be in focus while each of the plurality of objects is offset from a near depth of field - e.g., independently of any determination of an object in focus count corresponding to a different focal configuration than one to place one of the plurality of objects at a near depth of field. For example, a focal configuration determination performed for objects A-F in the field of view 320 may include performing only six object in focus counts -- i.e., each for a corresponding one of the focus fields D1-D6shown.Conventional techniques for determining lens focusing cover a range of focus distances and perform corresponding computations for each of a large series of focus fields. This larger series typically includes many focus fields for which no identified object is (or would be) at the near depth of field. In comparison, some embodiments calculate scores for a relatively smaller, more particular set of focal fields-e.g., the total number may be no greater than a total number of the multiple objects. The comparable assessment of only D 1 to D 6-e.g., without assessing also many other intermediate focus fields each located between corresponding ones of D 1 to D 6-illustrates an efficiency obtained by many such embodiments. Compared to conventional techniques, such embodiments are more efficient by providing relatively simpler and therefore faster processing for evaluating focal fields.Although some embodiments are not limited in this regard, the method 200 may include one or more additional operations (not shown) to operate an image sensor device based on the signal provided at 250. For example, the method 200 may further comprise configuring the lens system based on the signal provided at 250 - e.g. wherein the lens system is to implement the first configuration to locate the first object at a near depth of field. In such an embodiment, an object other than the first object may be a closest object (of all multiple objects) to the lens array. After configuring the lens system, method 200 may further operate a pixel array to capture an image received with the lens system. Although some embodiments are not limited in this regard, the method 200 may be repeated one or more times, e.g., including the selection circuitry 160 (for example) performing one or more additional focus determinations of multiple focus determinations including the first focus determination. Some or all of such multiple focus determinations may each correspond, for example, to a different respective aperture that is to operate with the lens system.FIG. 4 illustrates elements of a method 400 for determining a focal configuration, according to an embodiment. The method 200 may be carried out, for example, using the system 100 or the image sensor device 310. In one embodiment, the method 200 includes some or all of the features of the method 200.In the illustrative embodiment shown, method 400 includes operations to initialize variables used in determining a preferred focal configuration. By way of illustration and not limitation, such operations may include, at 405, setting to zero each of a variable Dmaxthat represents a currently preferred near focus field and another variable Nmaxthat represents an object-in-focus count corresponding to Dmax. The operations at 405 may additionally or alternatively include setting a counter variable x to an initial value, e.g., one (1)The method 400 may further include, at 410, determining a distance dx of the xth object (where xth is an ordinal corresponding to a current value of the variable x) of a plurality of objects determined to be within a field of view observable via a lens system. The distance dx can be determined with respect to a reference location such as a center point in or on a lens of the lens system. At 415, method 400 may determine a value Nx representing an object in focus count-i.e., a count of objects that are (or would be) in focus while the lens system has a focal configuration that sets the xth object to a near focus field. The determining at 415 may include, for example, one or more operations such as determining at 220 or determining at 230.The method 400 may further include determining at 420 whether the value Nx last determined at 415 is greater than a present value of Nmax. Where it is determined at 420 that Nmax is greater than the current Nmax value, at 425, method 400 may perform operations including setting Nmax to be equal to the most recently determined value Nx. The operations at 425 may further include setting Dmax equal to the most recently determined value of Dn. A determination may then be made at 430 as to whether any other object of the plurality of objects remains to be addressed by the method 400. Where, instead, it is determined at 420 that Nmax is less than (or equal to, in some embodiments) the current Nmax value, the method 400 may precede an instance of the operations at 425 and proceed to the determination at 430.In response to a determination at 430 that each of the plurality of objects has been addressed, the method 400 may proceed to or follow subsequent operations (not shown) to implement, at the lens system, a focal configuration that provides a near depth of field equal to the most current value of Dmax. Where, instead, it is determined at 430 that at least one of the plurality of objects has not been addressed, method 400 may increment counter x at 435 and proceed to perform (for the newly incremented value of x) another instance of the determination at 410.In one embodiment, the initial xth object - i.e., a first object - of the plurality of objects to be addressed by the method 400 is a closest object of the plurality of objects to the lens system. For example, each next xth object to be addressed by the method 400 may be a next object further away from the lens system. In such an embodiment, one or more additional test conditions (not shown) may be evaluated to determine whether to execute an exit from method 400.By way of illustration and not limitation, an early exit from method 400 may be performed in response to a determination, e.g., at 420, that Nx represents the xth object and all other objects (of the plurality of objects) that are further away from the lens system than the xth object. Alternatively or additionally, an early exit from the method 400 may be performed in response to a determination, e.g., at 420, that each subsequent score Nx may not be greater than the current value of Nmax (e.g., where the method 400 is to address objects in sequence according to the increasing order of their respective distances from the lens system).FIG. 5 illustrates features of an embodiment in which a focal configuration of a lens system is determined based on corresponding scores for objects in a field of view, the scores in turn being determined based on different weight values assigned to different ones of the objects. Such determination may be performed, for example, by a system 100 or an image sensor device 310-e.g., according to one of the methods 200, 400.FIG. 5 shows a top view of an environment 500 in which an image sensor device 510 is to be operated, according to an embodiment. As shown in FIG. 5, the image sensor device 510 may be positioned (localized and aligned) such that objects, e.g., the illustrative plurality of objects A through F shown, are each in the field of view 520 between the lines of sight 522, 524 observable by a lens system 512 of the image sensor device 510. The image sensor device 510, in various embodiments, may be positioned to additionally or alternatively image more, fewer, and / or differently arranged objects.FIG. 5 shows an example view 550 of objects A-F (which are persons in the illustrative scenario) located within the field of view 520, the view 550 as viewed through the lens system 512. A focal configuration for lens system 512 may be determined, for example, based in part on the respective distances x 1 to x 6 of objects A to F from lens system 512. In some embodiments, such determination of a focal configuration may be further based on corresponding locations of some or all of objects A-F in view 550.For example, preprogrammed reference information - e.g., stored in memory 130 - may correspond to different regions of view 550 each with a corresponding value indicating a degree of value placed on objects in that region. In the illustrative embodiment shown, such regions include a region 554 in which a center of view 550 is located (the center aligned with central axis 526 of field of view 520). The regions may further include a region 556 adjacent to and extending around the region 554 and another region 558 adjacent to and extending around the region 556. Yet another region around 558 may extend to an edge 552 of the view 550.For a given one of such regions, an object identified as being in the region may be assigned a weight equal to or otherwise based on the predefined preference value associated with that region. In the illustrative embodiment shown, object A may be assigned a first weight corresponding to region 554, and objects B, C, and D may each be assigned a second weight corresponding to region 558. Objects E and F may each be assigned a third weight corresponding to the region adjacent to and surrounding region 558.A score Sxmay be calculated for a given focal configuration Cxof the lens system 512- e.g., where Cxprovide a near depth of field equal to the distance of the xth object of the plurality of objects from the lens system 512. In one embodiment, a value Sxmay be calculated according to the following: where I is an integer equal to a total number of the multiple objects, B ix is a Boolean value that is equal to "1" when the ite object is or were in focus during CX (and otherwise equal to "0"), and W i is a weight value associated with the region of view 550 in which the ite object is located. In some other embodiments, a given weight value W i may additionally or alternatively be determined based on an object type to which a corresponding its object belongs. By way of illustration and not limitation, a weight value w i may be relatively more significant where image recognition processing has identified the corresponding ith object as an instance of a human face (or portion thereof) object type. The assignment of particular weights to corresponding objects may be based on any of a wide variety of possible object type preferences that are preprogrammed or otherwise predetermined - e.g., from a manufacturer, dealer, user, or other agent. In an illustrative embodiment, a relatively more significant (e.g., larger value) weight may be assigned to objects of a human face object type compared to one or more alternative object types. The techniques by which such preferences are to be determined, however, may depend on application specific details and may not be limiting in some embodiments. For example, unlike object-in-focus counting, an S x- value (or a W i- value) may be a number other than any integer. Equation (6) is merely an example calculation to determine S x for a given xth object. Any of a variety of other computations to determine an S x- value may be performed according to various embodiments.Some embodiments may correspondingly calculate S x- values for two or more - e.g., each of - the / objects identified as being in the field of view 520. Lens system 512 may then be configured based on such a score of such scores. For example, a focal configuration of the lens system 512 may be implemented based on this focal configuration with a largest S x- value. In some scenarios, two or more focal configurations may each have the same S x- value - e.g., where that S x- value is greater than any of the other calculated S x- values. In such an embodiment, one of these two of more focal configurations may be selected for implementation based on this focal configuration with a shortest focus distance compared to others of the two of more focal configurations.Techniques and architectures for operating an optical device are described herein. Some portions of the detailed description herein are presented in terms of algorithms and symbolic representations of operations on data bits within a computer memory. These algorithmic descriptions and representations are the means used by one skilled in the data processing arts to effectively convey the substance of their work to others skilled in the art. An algorithm is here and is generally considered a self-consistent sequence of steps leading to a desired result. These steps are those requiring physical manipulation of physical quantities. Usually, though not necessarily, these quantities take the form of electrical or magnetic signals which can be stored, transmitted, combined, compared, and otherwise manipulated. It has been found suitable, in principle for reasons of common usage, to refer to these signals as bits, values, elements, symbols, characters, terms, numbers or the like.It should be noted, however, that all of these and similar terms are to be associated with the appropriate physical quantities and are merely appropriate terms applied to these quantities. Unless specifically stated otherwise, as apparent from the discussion herein, it is understood that in the specification discussions utilizing terms such as "processing" or "calculating" or "calculating" or "calculating" or "determining" or "presenting", and the like, refer to the acts and processes of a computer system or similar electronic computing device that transforms and manipulates data represented as physical (electronic) quantities within the registers and memories of the computer system into other data similarly represented as physical quantities within the computer system memories or registers or within other such information storage, transmission, or display devices.Certain embodiments also relate to an apparatus for performing the operations herein. This apparatus may be specially designed for the required purposes or it may comprise a general purpose computer selectively activated or reconfigured by a computer program stored in the computer. Such a computer program may be stored in a computer readable storage medium, such as, but not limited to, any type of disk including floppy disks, optical disks, CD-ROMs, and magneto-optical disks, read-only memories (ROMs), random access memories (RAMs) such as dynamic RAM (DRAM), EPROMs, EEPROMs, magnetic or optical cards, or any type of media suitable for storing electronic instructions, and each coupled to a computer system bus.The algorithms and displays presented herein are not inherently associated with any particular computer or other device. Various general purpose systems may be used with programs in accordance with the teachings herein, or it may prove convenient to design a more specialized apparatus to perform the required process steps. The required structure for many of these systems will appear in the description below. In addition, certain embodiments are not described with reference to a particular programming language. It will be appreciated that a variety of programming languages may be used to implement the teachings of these embodiments as described herein.In addition to the descriptions herein, various modifications may be made to the disclosed embodiments and implementations thereof without departing from the scope thereof. The illustrations and examples herein are therefore intended to be interpreted in an illustrative and non-limiting sense. The scope of the invention is to be measured with reference only to the following claims.

Claims

An apparatus comprising: a lens system (110) for receiving light from an environment external to the apparatus; distance estimation circuitry (150) configured to identify corresponding distances to each of a plurality of objects located in a field of view (320) of the lens system (110) comprising a first object and a second object; Selection circuitry (160) coupled to the distance evaluation circuitry (150) to determine a focus of the lens system (110), the selection circuitry (160) comprising logic that, when executed, causes the apparatus to perform operations comprising: determining a first count of each of the plurality of objects that would appear in focus while the first object is at a first near depth of focus based on a first focal configuration and a first aperture (122), wherein the first count represents a total number of the plurality of objects that would appear in focus while the first object is at the first near depth of focus; determining a second count of each of the plurality of objects that would appear in focus while the second object is at a second near depth of focus based on a second focal configuration and the first aperture (122), the second count representing a total number of the plurality of objects that would appear in focus while the second object is at the second near depth of focus; comparing a first score based on the first count and a second score based on the second count; and providing, based on the comparing, a signal (162) indicative of a preference between the first focal configuration or the second focal configuration; a focus controller (170) coupled to adjust the lens system (110) based on the signal (162); and an image sensor (124) optically coupled, to capture an image received with the lens system (110), wherein either: the first score is calculated based on weighting values assigned to the plurality of objects that would appear in focus while the first object is at the first near depth of field based on a location of the respective object in the field of view (320), and the second score is calculated based on weighting values assigned to the plurality of objects that would appear in focus while the second object is at the second near depth of field based on a location of the respective object in the field of view (320), and / or the first score is calculated based on weighting values assigned to the plurality of objects that would appear in focus while the first object is at the first near depth of field based on an object type of the object, and the second score is calculated based on weight values assigned to the plurality of objects that would appear in focus while the second object is at the second near depth of field based on an object type of the object.The apparatus of claim 1, wherein the selection circuitry (160) further determines one or more other focuses of the lens system (110) and partially determines each of the one or more of the other focuses based on a corresponding different aperture (122) than the first aperture (122).The apparatus of claim 1, wherein the signal (162) identifies the first focal configuration, wherein of the plurality of objects, an object other than the first object is the closest object to the lens system (110).The apparatus of claim 1, wherein the location of the object in the field of view (320) is relative to a reference point or a reference line of the field of view (320).The apparatus of claim 1, wherein the reference point is a center of the field of view (320).The apparatus of claim 1, wherein the location of the object in the field of view (320) is relative to another object of the plurality of objects.The apparatus of claim 1, wherein the first score comprises a different number than any integer.The apparatus of claim 1, wherein the object type comprises a human face object type.A non-transitory computer readable storage medium having instructions stored thereon that, when executed by one or more processing units, cause the one or more processing units to perform a method comprising: identifying corresponding distances to each of a plurality of objects in a field of view (320) of a lens system (110), the plurality of objects comprising a first object and a second object; determining a focus of the lens system (110) comprising: determining a first count of each of the plurality of objects that would appear in focus while the first object is at a first near depth of focus based on a first focal configuration and a first aperture (122), the first count representing a total number of the plurality of objects that would appear in focus while the first object is at the first near depth of focus; determining a second count of each of the plurality of objects that would appear in focus while the second object is at a second near depth of focus based on a second focal configuration and the first aperture (122), the second count representing a total number of the plurality of objects that would appear in focus while the second object is at the second near depth of focus; comparing a first score based on the first count and a second score based on the second count; and providing, based on the comparing, a signal (162) indicative of a preference between the first focal configuration or the second focal configuration; adjusting the lens system (110) based on the signal (162); and capturing an image received with the lens system (110); wherein either: the first score is calculated based on weight values, which are assigned to the plurality of objects which would appear in focus while the first object is at the first near depth of focus based on a location of the respective object in the field of view (320), and the second score is calculated based on weighting values assigned to the plurality of objects which would appear in focus while the second object is at the second near depth of focus based on a location of the respective object in the field of view (320), and / or the first score is calculated based on weighting values assigned to the plurality of objects which would appear in focus while the first object is at the first near depth of focus based on an object type of the object, and the second score is calculated based on weighting values assigned to the plurality of objects which would appear in focus, while the second object is at the second near depth of field, they are assigned based on an object type of the object.The computer readable storage medium of claim 9, wherein the method further comprises determining one or more other focuses of the lens system (110), and each of the one or more of the other focuses in part based on a corresponding different aperture (122) than the first aperture (122).The computer readable storage medium of claim 9, wherein the signal (162) identifies the first focal configuration, wherein, of the plurality of objects, an object other than the first object is the closest object to the lens system (110).The computer readable storage medium of claim 9, wherein the location of the object in the field of view (320) is relative to a reference point or a reference line of the field of view (320).The computer readable storage medium of claim 9, wherein the location of the object in the field of view (320) is relative to another object of the plurality of objects.A method comprising: identifying corresponding distances to each of a plurality of objects in a field of view (320) of a lens system (110), the plurality of objects comprising a first object and a second object; determining a focus of the lens system (110) comprising: determining a first count of each of the plurality of objects that would appear in focus while the first object is at a first near depth of focus based on a first focal configuration and a first aperture (122), wherein the first count represents a total number of the plurality of objects that would appear in focus while the first object is at the first near depth of focus; determining a second count of each of the plurality of objects that would appear in focus while the second object is at a second near depth of focus based on a second focal configuration and the first aperture (122), the second count representing a total number of the plurality of objects that would appear in focus while the second object is at the second near depth of focus; comparing a first score based on the first count and a second score based on the second count; and providing, based on the comparing, a signal (162) indicative of a preference between the first focal configuration or the second focal configuration; adjusting the lens system (110) based on the signal (162); and capturing an image received with the lens system (110); wherein either: the first score is calculated based on weight values, which are assigned to the plurality of objects which would appear in focus while the first object is at the first near depth of focus based on a location of the respective object in the field of view (320), and the second score is calculated based on weighting values assigned to the plurality of objects which would appear in focus while the second object is at the second near depth of focus based on a location of the respective object in the field of view (320), and / or the first score is calculated based on weighting values assigned to the plurality of objects which would appear in focus while the first object is at the first near depth of focus based on an object type of the object, and the second score is calculated based on weighting values assigned to the plurality of objects which would appear in focus, while the second object is at the second near depth of field, they are assigned based on an object type of the object.The method of claim 14, wherein the signal (162) identifies the first focal configuration, wherein of the plurality of objects, an object other than the first object is the closest object to the lens array.

Images