Imaging system and method

Abstract

A method of generating an image can include obtaining a profile of a combination of a full-frame camera and a lens; obtaining image information from an electronic sensor of a mobile electronic device; and / or generating a final digital image via an electronic control unit of the mobile electronic device by applying a profile of the one or more profiles to the image information.

Description

[0001] Cross-reference of related applications

[0002] This application claims the benefit of U.S. Provisional Patent Application Serial No. 63 / 168,768, filed March 31, 2021, the disclosure of which is incorporated herein by reference in its entirety, as fully set forth herein. Technical Field

[0003] This disclosure generally relates to imaging systems and methods, including imaging systems and methods that can be used in conjunction with smartphone cameras, such as to simulate full-frame cameras and lenses. Background Technology

[0004] The background description described below is provided for context only. Therefore, nothing in this background description is expressly or implicitly acknowledged as prior art to this disclosure without otherwise conforming to the principles of the art.

[0005] Integrating a full-frame camera and lens into a smartphone may be impractical or unfeasible, but it is desirable to emulate one or more full-frame cameras and lenses for use with images captured via the smartphone's camera.

[0006] Solutions / options that aim to minimize or eliminate one or more challenges or drawbacks of imaging systems and methods. The above discussion is intended to illustrate examples in the art only and is not a denial of scope. Summary of the Invention

[0007] In an exemplary illustrative embodiment, a method for generating an image may include: obtaining a profile of a combination of a full-frame camera and lens; obtaining image information from an electronic sensor of a mobile electronic device; and / or generating a final digital image by applying the profile from one or more profiles to the image information via an electronic control unit of the mobile electronic device.

[0008] Using some example embodiments, an imaging system may include: multiple electronic profiles of a combination of a full-frame camera and lens stored in electronic memory, and a mobile electronic device configured to acquire image information via an electronic sensor of the mobile electronic device. The mobile electronic device may be configured to generate a final digital image by applying profiles from the multiple electronic profiles to the image information via an electronic control unit of the mobile electronic device.

[0009] The above and other potential aspects, features, details, utility and / or advantages of the examples / embodiments of this disclosure will become apparent from reading the following description and viewing the accompanying drawings. Attached Figure Description

[0010] While the claims are not limited to the specific illustrations, an understanding of the aspects can be gained through discussion of various examples. The drawings are not necessarily drawn to scale, and some features may be exaggerated or omitted to better illustrate and explain the innovative aspects of the examples. Furthermore, the exemplary illustrations described herein are not exhaustive or otherwise limiting, and are not limited to the precise forms and configurations shown in the drawings or disclosed in the detailed description below. The exemplary illustrations are described in detail with reference to the following drawings:

[0011] Figure 1 This is a front perspective view generally showing an embodiment of a full-frame camera.

[0012] Figure 2 This is a front view that generally illustrates an embodiment of a mobile electronic device.

[0013] Figure 3 This is a front view that generally shows an embodiment of a full-frame camera.

[0014] Figure 3A This is a representation of an embodiment of the sensor of a full-frame camera.

[0015] Figure 4 This is a front view that generally illustrates an embodiment of a mobile electronic device.

[0016] Figure 4A This is a representation of an embodiment of a sensor for a mobile electronic device.

[0017] Figure 4B This is a perspective view that generally illustrates an embodiment of a mobile electronic device.

[0018] Figures 5A to 5C Images are generated / captured via an exemplary embodiment of another mobile electronic device, a full-frame camera and lens, and an exemplary embodiment of a mobile electronic device, respectively.

[0019] Figure 6 This is a diagram illustrating an exemplary embodiment of the imaging method.

[0020] Figure 7 This is a diagram illustrating an exemplary embodiment of the imaging method.

[0021] Figure 8 This is a diagram illustrating an exemplary embodiment of an imaging method that includes generating enhanced depth information / depth maps.

[0022] Figure 9 This is an illustrative representation showing a full-frame camera and an electronic display.

[0023] Figures 10A to 12 This is an illustrative representation of a point spread function (PSF) associated with a full-frame camera and lens.

[0024] Figure 13 This is a flowchart illustrating a portion of an exemplary embodiment of the imaging method.

[0025] Figure 14 This is a flowchart illustrating a portion of an exemplary embodiment of the imaging method. Detailed Implementation

[0026] Reference will now be made in detail to the illustrative embodiments of this disclosure, examples of which are described herein and illustrated in the accompanying drawings. While this disclosure will be described in conjunction with embodiments and / or examples, they are not intended to limit this disclosure to those embodiments and / or examples. Rather, this disclosure covers alternatives, modifications, and equivalents.

[0027] Full-frame camera 100 and lens 102 (see, for example) Figure 1 It can provide blurring to at least some images. For example, blurring can include the aesthetic effect of blurring out-of-focus parts of an image (see, for example, [link to image]). Figure 5B (Image 110 in the image). Various components of a full-frame camera and lens (such as aperture blades, flare, spherical aberration, and / or chromatic aberration) can affect how the camera / lens renders bokeh.

[0028] Smartphone cameras typically have much smaller image sensors, with a significantly smaller surface area than full-frame cameras to capture light (see, for example, [link to full-frame camera]). Figure 3A and Figure 4A (Comparison), and can produce a near-infinite depth of field, which renders the background image almost as sharp as the object / target itself, so that no part of the image is out of focus (see, for example, [link to example]). Figure 5A (Image 120 in the image). While some smartphone cameras have modes that blur the background of an image (e.g., portrait mode), this blur does not simulate the optical effects associated with full-frame camera and lens combinations, such as optical vignetting, chromatic aberration, spherical aberration, and / or various other properties of bokeh that photographers can use to achieve a particular look in their art. Instead, smartphone blur may be uniform and completely grain-free, which can give the blur an unnatural look and is hardly noticeable to the viewer's eye.

[0029] Using embodiments of this disclosure, a mobile electronic device 300 can be configured to simulate one or more effects of a full-frame camera 100, a lens 102, and / or a combination of full-frame cameras and lenses (see, for example, [link to relevant documentation]). Figure 5B Images from a combination of full-frame cameras and lenses (110), such as bokeh associated with this combination (see, for example, [image 110]). Figure 5C(Simulated effect in image 400). Furthermore, different simulation profiles can be used, each representing a unique full-frame camera and lens combination. For each simulation profile, effectively simulating the bokeh of the full-frame camera 100 and lens 102 can include: obtaining the original image, such as obtaining accurate depth information on a pixel-by-pixel basis (e.g., rather than applying uniform blur), and / or obtaining a digital / electronic profile of the full-frame camera and lens combination. Accurate depth information can be provided, for example, via a depth map. Simulated bokeh can include applying accurate / modified depth information and profiles to the original image.

[0030] In such as Figure 2 In the exemplary embodiments generally shown, the imaging system 200 may include a mobile electronic device 300 (e.g., a smartphone), which may include a first electronic sensor 320, a second electronic sensor 322, and / or an electronic control unit (ECU) 324. The mobile electronic device 300 may, for example, include a wireless / cellular transceiver / radio / modem 326, which may be connected, coupled to, and / or communicate with the ECU 324, and may be configured to communicate with a cellular network and / or another wireless network (e.g., the mobile electronic device may be configured as a cellular phone). The mobile electronic device 300 may include a memory 332, a display 334, and / or a processor 336 (see, for example, see...). Figure 8 Mobile electronic devices 300 can be relatively thin (see, for example, see below). Figure 4B For example, but not limited to, the mobile electronic device 300 may have a thickness 300T of approximately 1 inch (25.4 mm) or less, approximately 0.5 inches (12.7 mm) or less, or approximately 0.4 inches (10.6 mm) or less. In some examples, the mobile electronic device 300 may include a height 300H of at least 4 inches (101.6 mm) and / or a width of at least 2 inches (50.8 mm). For example, but not limited to, the width of the mobile electronic device 300 may be at least 5 times its thickness and / or the height may be at least 10 times its thickness.

[0031] Adopting, such as in Figure 1 , Figure 3 and Figure 3AIn the exemplary embodiments generally shown, the full-frame camera and lens combination may be relatively thick and / or may include a relatively large color sensor 104. For example, but not limited to, the full-frame camera 100 may include a depth / thickness of at least 1.5 inches (38 mm) or more, and / or the lens may include a depth / thickness of 2 inches (51 mm) or more (e.g., approximately 3 inches (76 mm)), which provides a combination of full-frame camera 100 and lens 102 with a combined depth / thickness of approximately 3.5 inches (89 mm) or more. Additionally or alternatively, the area of ​​the color sensor 104 of the full-frame camera 100 may, for example, but not limited to, be at least 1.24 inches. 2 (800mm 2 ), such as approximately 1.34 in 2 (864mm 2 ).

[0032] In such as Figure 4 and Figure 4A In the exemplary embodiments generally shown, the color sensor 330 of the first electronic sensor 320 of the mobile electronic device 300 can be significantly smaller than the color sensor of the full-frame camera 100. For example, but not limited to, the area of ​​the color sensor 330 of the embodiment of the mobile electronic device 300 can be less than 1 inch. 2 (645.16mm 2 (less than 0.5in) 2 (322.58mm 2 ), and / or approximately 0.16 in 2 (100mm 2 Up to approximately 0.23 in 2 (150mm 2 ), such as approximately 0.19 in 2 (123mm 2 The color sensor 330 of at least some embodiments of the mobile electronic device 300 may be significantly larger than that of other mobile electronic devices (e.g., other smartphones). For example, but not limited to, the Apple iPhone 12 Pro Max may include approximately 0.05 inches. 2 (35mm 2 The sensor area is smaller. A smaller color sensor is able to capture less light (e.g., proportionally less) at the same time as a larger color sensor.

[0033] In some cases, one or more chromatic aberration factors of the full-frame camera / lens 100 / 102 can be evaluated via a multi-channel PSF (e.g., simulated across several wavelengths). Chromatic aberration intensity can vary with the distance between the camera 100 and the display 430. A PSF not centered on the principal ray may be out of focus. A focused PSF may include chromatic aberration.

[0034] Adopting, such as in Figure 6 and Figure 7 In the exemplary embodiments generally shown, the imaging system 200 (e.g., ECU 324 of mobile electronic device 300) can be configured to acquire an initial image 402, acquire depth information 404, acquire a profile 406 of the full-frame camera 100, lens 102, and / or a combination of the full-frame camera 100 and lens 102, enhance the depth information 404 to obtain a modified depth map / information 422 (box 340), simulate the camera 100 and / or lens 102 (box 342), and / or generate a final digital image 400 that includes the simulated effects of the full-frame camera 100 and / or lens 102. Profile 406 may include one or more PSFs 408 (e.g., PSF information).

[0035] In such as Figure 8 In the exemplary embodiments generally shown, depth information 404 may be obtained via a second electronic sensor 322, which may include a time-of-flight (TOF) sensor. Depth information 404 may include depth data 410 and / or depth confidence information 412. A plurality of images 420 may be captured via a first electronic sensor 320 of the mobile electronic device 300 (e.g., via a color sensor 330). ECU 324 may select two or more of the plurality of images 420 for parallax calculation. ECU 324 may utilize depth information 404 (which may include depth data 410 and / or depth confidence information 412) and / or parallax calculation to generate modified / enhanced depth information and / or depth map 422 (block 340).

[0036] In an exemplary embodiment, the profile 406 of the full-frame camera 100 and / or lens 102 may include one or more point spread functions (PSFs) 408, such as in Figures 10A to 11B As generally illustrated. Obtaining a profile 406 may include operating the full-frame camera 100 and / or lens 102 to generate one or more PSFs 408. For example, the full-frame camera 100 and lens 102 may be operated to capture light and / or one or more contour images of the electronic display 430, such as at multiple distances (see, for example, [link to image]). Figure 9 The electronic display 430 may include, for example but not limited to, an organic light-emitting diode (OLED) display. The electronic display 430 may display a plurality of white circles 432 on a black background 434. Each PSF 408 may include characteristics of the corresponding circle 432, such as shape, color difference, fringing, and / or non-uniform intensity distribution (see, for example). Figures 10A to 11BThe characteristics may vary across the field of view (e.g., for a specific depth / distance between camera / lens 100 / 102 and display 430) and / or may vary depending on the depth / distance between camera / lens 100 / 102 and display 430. Profile 406 may include PSF information, which may be stored in a grid / matrix (X,Y) format. The PSF information may include PSF 408 for each corresponding depth and focal length (e.g., layers of the PSF), and / or the PSF information may be stored in a three-dimensional matrix (X,Y,Z / depth). PSF 408 (and its characteristics) may be utilized (e.g., by mobile electronic device 300) to simulate the behavior of camera / lens 100 / 102. The information in PSF 408 may include light intensity information, which may include, for example, a single value or multiple values ​​(e.g., three color values ​​RGB).

[0037] Adopting, such as in Figures 10A to 11B The exemplary embodiments generally shown herein may allow the initial PSF information to be modified, for example, by a mobile electronic device 300 and / or a remote computer server 440. For example, but not limited to, the initial PSF information (such as...) Figure 10A As shown (focal point 1.2m, depth 1.4m) and Figure 11A The image shown (focus 1.2m, depth 2.5m) can be normalized and / or white balanced. Information such as PSF (Power Seer Function) is simulated to generate modified PSF information, such as... Figure 10B As shown (focal point 1.2m, depth 1.4m) and Figure 11B As shown (focal point 1.2m, depth 2.5m).

[0038] In some examples, a library 444 (e.g., an electronic library) can be created for profiles and / or PSFs 408 for various cameras 100, lenses 102, and / or camera-lens combinations, and this library can be made accessible to the mobile electronic device 300. For example, but not limited to, a portion or all of the library 444 can be uploaded to and / or stored on the memory 332 of the mobile electronic device 300. Additionally or alternatively, a portion or all of the library 444 can be stored on the memory 442 of a remote server 440, and the mobile electronic device 300 can be configured to communicate with the remote server 440 (e.g., wirelessly via a global computer and / or cellular network) to obtain profile and / or PSF information. The library 440 may include PSFs 408 for some or all positions (X, Y), depth (Z), and / or focal length. The remote server 440 may include a remote server ECU 466 and / or a processor 448. The ECU 446 may include and / or be connected to the memory 442 and / or the processor 448.

[0039] In some illustrative embodiments, configuration file 406 and / or PSF 408 can be generated via simulation. For example, the physical dimensions and characteristics of camera 100 and / or lens 102, lighting information and / or depth information, etc., can be provided only to PSF simulator 450, such as Zemax OpticStudio provided by Zemax LLC. PSF simulator 450 can use the input information to generate simulation information to generate PSF 408, which can simulate the PSF generated directly via camera / lens 100 / 102. In some examples, PSF simulator 450 can be included, communicate with, and / or connected to a remote server 440 (e.g., via a wired connection, wireless connection, removable media, etc.) (see, for example, see...). Figure 8 ).

[0040] In such as Figure 12 In the exemplary embodiments generally shown, profile 406 may include PSF information that can be stored as a one-dimensional matrix (e.g., for each depth and focal length) on mobile electronic device 300 and / or remote server 440. This allows for more efficient application of PSF information and / or allows denser / higher quality PSF information to be utilized by and / or stored on mobile electronic device 300. For example, PSF information may be stored as a function of radial position from the center (e.g., rather than X, Y position), thereby effectively reducing a two-dimensional grid to a one-dimensional strip / array / matrix 452. Strip 452 can then be “rotated” around the center to provide PSF information covering a circular area on sensor 320, which approximates a complete set of PSF 408 across that area. For rotationally symmetric lens 102, this approximation is effectively identical to the full PSF information. PSF information may still include three values ​​(for R, G, B), even when stored in a one-dimensional strip / array / matrix 452.

[0041] In an exemplary embodiment, a first-order approximation of the PSF can be used for depth sampling and / or scaling using nearest-neighbor (NN) PSF. A first refinement may include ECU 324 utilizing bilinear spatial sampling and / or normalized spatial scaling. A second refinement may include ECU 324 performing trilinear spatial and / or depth sampling and scaling / rotation correction. A third refinement may include ECU 324 performing approximate basis decomposition and / or scaling / rotation correction. Separable approximations can provide additional refinement. A fourth refinement that can improve processing speed may include ECU 324 performing downsampling techniques (e.g., downsampling random sampling techniques). A fifth refinement may include ECU 324 performing multiple scaling.

[0042] Figure 13Exemplary embodiments of an imaging method are generally illustrated. This imaging method may include simulating a camera 100 and / or lens 102 using depth information 404 (e.g., depth maps 410, 422), a profile 406, one or more PSFs 408 (e.g., PSF information), and / or an initial / raw image 402, which may include an ECU 324. The ECU 324 may obtain / identify a focal region / focus 460 in the depth maps 410, 422. For example, but not limited to, the focal region / focus 460 may be identified by a sensor (e.g., a first electronic sensor 320 and / or a second electronic sensor 322), may be the center of the depth maps 410, 422, and / or may be user-specific. The ECU 324 may, for example, use a lens equation (the sum of 1 divided by the object distance and 1 divided by the image distance equals 1 divided by the focal length) to convert the depth maps 410, 422 into a blur radius map 462. This conversion may include converting depth values ​​(e.g., in millimeters) into a blur radius on a color / light sensor. In some embodiments, ECU324 may apply guided magnification to the blur radius map 462. Guided magnification may include edge thinning, which may align the blurred edges with the image edges.

[0043] In an illustrative embodiment, ECU 324 may generate quantized, magnified blur radius maps 464, 466, such as nearest indices in PSF information (e.g., a PSF array). In some cases, blur radius map 462 may include continuous blur, but obtaining, accessing, and / or utilizing continuous PSF information (e.g., for continuous depths) may not be feasible, and ECU 324 may identify PSFs 408 (box 472) available for depths closest to the corresponding pixel (before and after that pixel). For example, if the depth of a pixel is 1.9m, but a PSF is not available for 1.9m, ECU 324 may select a first PSF 408 (e.g., for a PSF of 1.8m) before the pixel and a second PSF 408 (e.g., for a PSF of 2.1m) after the pixel. ECU 324 may rotate, scale, and / or interpolate (box 476) the PSFs of adjacent pixels (for example, to generate modified PSF information 478). For example, ECU 324 can interpolate the PSF information of a pixel located 1.9m from the first PSF 408 and the second PSF 408, and can assign a larger weight to the first PSF 408 because it is closer to the pixel's depth than the second PSF 408. The number of available PSF 408s can, for example, but not limited to, correspond to a logarithmic function relative to depth, where a larger number of PSF 408s are available for smaller depths, and the number of available PSF 408s decreases at an increasing rate as depth increases. In some cases, ECU 324 can preprocess the PSF information (box 470), such as before identifying neighboring PSFs. Quantization can be applied to each pixel. In some examples, ECU 324 can, for example, compute residual scaling and / or interpolation factors before box 476 (box 474).

[0044] In an exemplary embodiment, ECU 324 may convert an initial / raw image 402 into a linear tone curve / gamma to generate a converted initial image 480, and / or ECU 324 may apply a per-pixel rendering process (box 482) that may be at least partially based on the converted initial image 480 and modified PSF information 478 to provide an intermediate image 484. The per-pixel rendering (box 482) may include accumulation, normalization, and / or blending. After the per-pixel rendering process (box 482), ECU 324 may recover the tone curve / gamma for the intermediate image 484 to generate a recovered image 486. ECU 324 may blend sharp pixels back into the foreground of the intermediate image 484, and / or output the intermediate image as a modified (e.g., recovered and / or blended back) output image (e.g., the final digital image 400).

[0045] Two illustrative methods for simulating out-of-focus images can be scattering and clustering. Scattering methods can produce optimal quality via layer-based rendering and alpha blending. Scattering methods can be relatively slow but accurate. For example, scattering methods may involve simulating diffusion from the source point to the output at each layer. Clustering methods may be less accurate but can be faster and / or provide better graphics processing unit (GPU) shader compatibility. When appropriate sampling and weighting strategies are used, clustering methods can provide a good approximation. Clustering methods can simulate each output pixel and can be (e.g., directly) mapped to the final image as a weighted sum of input pixels. Clustering methods can automatically account for foreground occlusion issues.

[0046] In such as Figure 14 In the exemplary embodiments generally shown, the pixel-by-pixel rendering process (e.g., block 482) may include: for each output pixel 490(x,y), ECU 324 initializing one or more accumulators, such as accumulators A(x,y) and W(x,y), traversing nearby pixels, and / or determining an output value (e.g., a modified version 494 of output pixel 490). Initializing the accumulators may include specifying / allocating memory or other electronic storage devices for information for each pixel (e.g., a blank image).

[0047] In an exemplary embodiment, traversing nearby pixels may include identifying one or more pixels 492(i,j) (e.g., neighboring pixels) at positions (x,y) close to / near the output pixel 490. The positions (i,j) of pixel 492 may be relative to the positions (x,y) of the output pixel 490. ECU 324 may obtain depth information z(i,j) and / or light / color information c(i,j) for each neighboring pixel 492. The depth information z(i,j) may be obtained from depth maps 410, 422 and / or blur radius map 462. The light / color information may be obtained from a light / color sensor, such as one that may be included with the first electronic sensor 320. ECU 324 may also obtain PSF information for each neighboring pixel 492, such as from a library 444 of PSF 408. The depth associated with a neighboring pixel 492 may determine which PSF 408 to use, and the position of the neighboring pixel 492 may determine what information to obtain from that PSF 408. The obtained PSF information may include light intensity information. In some cases, PSF information may include color (e.g., RGB) intensity; therefore, PSF information may include three values. This color information can simulate the color difference associated with a camera-lens combination of 100 / 102.

[0048] In an illustrative embodiment, ECU 324 can use the obtained PSF information to create a weighting factor w(i,j). For example, if the obtained PSF information is a single value (e.g., an intensity value), the weighting factor w(i,j) can be equal to that single value. If the obtained PSF includes three values ​​(e.g., RGB intensity), the weighting factor can include three values. ECU 324 can apply blending during rendering, which may include adjusting the weights based on depth. For example, the weights determined from closer PSFs can be increased and / or the weights determined from more distant PSFs can be decreased.

[0049] In an exemplary embodiment, ECU 324 can determine the cumulative value A(x,y) of a pixel based on the sum of the products of the corresponding weights of the pixel and its neighboring pixels and the light / color information of the pixel and its neighboring pixels: for all i,j (including (0,0), which corresponds to the pixel itself), and for each pixel x,y, A(x,y) = ∑w(i,j)*c(i,j). If both the weights and the light / color information include three values, the corresponding value (RGB) of the weights can be multiplied by the corresponding value (RGB) of the light / color information. If the weights include one value, that value can be multiplied by each value of the light / color information. ECU 324 can determine the total weight W(x,y) as the sum of all weights w(i,j). ECU 324 can determine the output / modified color information of the pixel by dividing A(x,y) by W(x,y). The output / modified color information can, for example, include a weighted average of the color information of pixel 490 and its neighboring pixel 492. A pixel-by-pixel rendering process (box 482) can be applied to each pixel of the image, such as sequentially and / or simultaneously.

[0050] In some configurations, system 200 may include a default profile that may not correspond to a specific camera / lens combination 100 / 102. If other profiles (e.g., for camera 100, lens 102, or combinations thereof) are unavailable, system 200 may utilize the default profile, and / or ignore or delete the default profile when other profiles become available. The default profile may be stored, at least initially, on the mobile electronic device 300, such as on memory 332.

[0051] One or more functions of system 200 may be performed at least in part via ECU 324 of mobile electronic device 300 and / or ECU 446 of remote server 400 (or associated processors 336, 448). One or more functions of ECU 324 may be performed at least in part via ECU 446, and vice versa.

[0052] In the example, an ECU (e.g., ECU 324) may include an electronic controller and / or an electronic processor, such as a programmable microprocessor and / or a microcontroller. In embodiments, the ECU may include, for example, an application-specific integrated circuit (ASIC). An ECU may include a central processing unit (CPU), memory (e.g., a non-transitory computer-readable storage medium), and / or input / output (I / O) interfaces. An ECU may be configured to perform various functions, including those described in more detail herein, and may embody appropriate programming instructions and / or code in software, hardware, and / or other media. In embodiments, an ECU may include multiple controllers. In embodiments, an ECU may be connected to a display, such as a touchscreen display.

[0053] Various examples / embodiments of various apparatuses, systems, and / or methods are described herein. Numerous specific details are set forth to provide a thorough understanding of the overall structure, function, manufacture, and use of the examples / embodiments described in the specification and shown in the accompanying drawings. However, those skilled in the art will understand that the examples / embodiments can be practiced without such specific details. In other instances, well-known operations, components, and elements have not been described in detail so as not to obscure the examples / embodiments described in the specification. Those skilled in the art will understand that the examples / embodiments described and illustrated herein are non-limiting examples, and therefore it can be understood that the specific structural and functional details disclosed herein may be representative and do not necessarily limit the scope of the embodiments.

[0054] Throughout this specification, references to “example,” “in an example,” “using an example,” “various embodiments,” “using an embodiment,” “in an embodiment,” or “embodiment,” etc., mean that a particular feature, structure, or characteristic described in connection with an example / embodiment is included in at least one embodiment. Therefore, the appearance of the phrases “example,” “in an example,” “using an example,” “in various embodiments,” “using an embodiment,” “in an embodiment,” or “embodiment,” etc., in their appropriate places throughout this specification does not necessarily refer to the same embodiment. Furthermore, a particular feature, structure, or characteristic can be combined in any suitable manner in one or more examples / embodiments. Therefore, a particular feature, structure, or characteristic illustrated or described in connection with one embodiment / example can be combined, in whole or in part, with features, structures, functions, and / or characteristics of one or more other embodiments / examples, without limitation, provided that such combination is not illogical or nonfunctional. Moreover, many modifications can be made to adapt a particular situation or material to the teachings of this disclosure without departing from the scope of this disclosure.

[0055] It should be understood that references to a single element are not necessarily so limiting and may include one or more such elements. Any directional references (e.g., addition, subtraction, up, down, upward, downward, left, right, left to right, top, bottom, above, below, vertical, horizontal, clockwise, and counterclockwise) are used for identification purposes only to aid the reader's understanding of this disclosure and do not impose limitations, particularly with respect to the location, orientation, or use of the examples / embodiments.

[0056] The term "attachment" (e.g., attachment, coupling, connection, etc.) should be interpreted broadly and may include intermediate components between connected elements, relative movement between elements, direct connection, indirect connection, fixed connection, movable connection, operative connection, indirect contact, and / or direct contact. Therefore, an engagement reference does not necessarily imply that two elements are directly connected / coupled and in a fixed relationship with each other. Connections of electrical components (if any) may include mechanical connections, electrical connections, wired connections, and / or wireless connections, etc. The use of "e.g." and "such as" in the specification should be interpreted broadly and is intended to provide non-limiting examples of embodiments of this disclosure, and this disclosure is not limited to such examples. The use of "and" and "or" should be interpreted broadly (e.g., should be considered as "and / or"). For example, but not limited to, the use of "and" does not necessarily require listing all elements or features, and the use of "or" is inclusive unless such a construction is illogical.

[0057] While processes, systems, and methods may be described herein by combining one or more steps in a particular order, it should be understood that this approach may be practiced with steps in different orders, with some steps performed simultaneously, with additional steps, and / or with some steps omitted.

[0058] All matters contained in the foregoing description or shown in the accompanying drawings should be interpreted as illustrative only, not restrictive. Changes may be made to the details or structure without departing from this disclosure.

[0059] It should be understood that the computer / computing device, electronic control unit (ECU), system, and / or processor described herein may include conventional processing devices known in the art, capable of executing pre-programmed instructions stored in associated memory, all of which perform according to the functions described herein. With regard to the methods described herein being embodied in software, the resulting software may be stored in associated memory and may also constitute a means for performing such methods. Such a system or processor may further be of the type of ROM, RAM, RAM and ROM, and / or a combination of non-volatile and volatile memory, so that any software can be stored, and also allowing the storage and processing of dynamically generated data and / or signals.

[0060] It should be further understood that the article of manufacture according to this disclosure may include a non-transitory computer-readable storage medium on which a computer program is encoded for implementing the logic and other functions described herein. The computer program may include code to perform one or more of the methods disclosed herein. Such embodiments may be configured to execute via one or more processors (such as multiple processors integrated into a single system, or multiple processors distributed across and connected together on a communication network), and the communication network may be wired and / or wireless. Code for implementing one or more of the features described in conjunction with one or more embodiments causes multiple transistors to change from a first state to a second state when executed by a processor. Specific changing patterns (e.g., which transistors change state and which do not change state) may be specified at least in part by logic and / or code.

Claims

1. A method for generating a digital image, the method comprising: Obtain the configuration file for the combination of full-frame camera and lens; Image information is obtained from the electronic sensors of mobile electronic devices; as well as The final digital image is generated by applying the configuration file to the image information via the electronic control unit of the mobile electronic device; The image information includes color information and depth information, wherein the depth information includes a depth map; Generating the final digital image includes: Identify the focal region in the depth map; Convert the depth map into a blurred radius map; Guided magnification is applied to the fuzzy radius map; The magnified blur radius map is quantized to the nearest index in the point spread function (PSF) information of the configuration file; Preprocess the PSF information; For the quantized and magnified fuzzy radius map, find the neighboring PSFs in the preprocessed PSF information; Calculate the interpolation factor; The interpolation factor is used to modify the PSFs of the preceding and following neighbors to generate modified PSF information; The image information is converted into a linear tone curve; A pixel-by-pixel rendering process is applied, at least in part, based on the converted image information and the modified PSF information, to provide an intermediate image; To recover the tone curve of the intermediate image; and The recovered intermediate image is output as the final digital image.

2. The method according to claim 1, wherein, The configuration file includes a point spread function.

3. The method according to claim 1, wherein, Obtaining the configuration file includes operating the full-frame camera and the lens to generate a point spread function.

4. The method according to claim 3, wherein, Operating the full-frame camera and the lens to generate the point spread function includes capturing one or more contour images of an electronic display.

5. The method according to claim 4, wherein, The electronic display shows multiple white circles against a black background; and The electronic display includes an OLED display.

6. The method according to claim 1, wherein, The configuration file is obtained from an electronic library of configuration files for each combination of full-frame cameras and lenses.

7. The method according to claim 6, wherein, The electronic library is stored in the memory of the mobile electronic device.

8. The method according to claim 6, wherein, The electronic library is stored on a remote server; as well as Obtaining the configuration file includes: the mobile electronic device wirelessly accessing the configuration file from the electronic library via a global computer network.

9. The method according to claim 1, wherein, The mobile electronic device includes a cellular transceiver.

10. The method according to claim 1, wherein, The full-frame camera includes a color sensor, the area of ​​which is larger than the area of ​​the electronic sensor in the mobile electronic device.

11. The method according to claim 1, comprising: Delete or ignore the default configuration file information of the mobile electronic device.

12. An imaging system, comprising: Multiple electronic profiles of the combination of full-frame camera and lens are stored in electronic memory; as well as A mobile electronic device configured to acquire image information via electronic sensors of the mobile electronic device; The mobile electronic device is configured to generate a final digital image by applying a configuration file from a plurality of electronic configuration files to the image information via the electronic control unit of the mobile electronic device. The image information includes color information and depth information; The depth information includes a depth map; Generating the final digital image includes: Identify the focal region in the depth map; Convert the depth map into a blurred radius map; Guided magnification is applied to the fuzzy radius map; The magnified blur radius map is quantized to the nearest index in the point spread function (PSF) information of the configuration file; Preprocess the PSF information; For the quantized and magnified fuzzy radius map, find the neighboring PSFs in the preprocessed PSF information; Calculate the interpolation factor; The interpolation factor is used to modify the PSFs of the preceding and following neighbors to generate modified PSF information; The image information is converted into a linear tone curve; A pixel-by-pixel rendering process is applied, at least in part, based on the converted image information and the modified PSF information, to provide an intermediate image; Recover the tone curve from the intermediate image; and The recovered intermediate image is output as the final digital image.

13. The system according to claim 12, wherein, The mobile electronic device includes the electronic memory.

14. The system of claim 12, comprising: A remote computer server, wherein the remote computer server includes the electronic storage; The mobile electronic device is configured to wirelessly communicate with the remote computer server to access the plurality of electronic profiles.

15. The system according to claim 12, wherein, The configuration file includes point spread function information stored in the electronic memory as a one-dimensional array; as well as The application of the configuration file includes: applying the point spread function information as a function of radial distance.

16. The system according to claim 12, wherein, The multiple electronic profiles include: multiple PSFs for multiple depths and focal lengths for each combination of full-frame cameras and lenses.

Images