Method for aesthetic-based image capture and electronic device
The electronic device system addresses amateur users' challenges by generating and optimizing camera settings for aesthetically superior images, bridging the gap in image quality with professional standards.
Patent Information
- Application Number
- PCT/KR2025/003641
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2024-03-23
- Filing Date
- 2025-03-21
- Publication Date
- 2025-10-02
AI Technical Summary
Amateur users face difficulties in selecting the best camera and zoom settings to capture aesthetically pleasing images due to lack of knowledge and practice, leading to significant gaps in image quality compared to professional users.
An electronic device system that generates multiple representative images with region of interests (ROIs), determines aesthetic scores, adjusts camera settings, and captures the best frame based on these scores and deviations, using processors and modules to optimize image capture.
Enhances image capture quality by recommending and capturing the best frame, aligning camera settings to user-defined scenes, reducing the need for user expertise and improving aesthetic outcomes.
Smart Images

Figure KR2025003641_02102025_PF_FP_ABST
Abstract
Description
METHOD FOR AESTHETIC-BASED IMAGE CAPTURE AND ELECTRONIC DEVICE
[0001] The present disclosure relates to a field of applications related to camera systems, and more particularly relates to relates to an electronic device and a method for aesthetic-based image capture.
[0002] Conventionally, cameras were frequently used by users to capture the images of a scene. So, the users had to carry the camera everywhere to capture the scene, which was cumbersome for the user. Thus, in recent years, several developments in existing electronic devices have taken place such that the developments of the existing electronic devices have reduced the dependency of the user on the camera, have fulfilled the demand of users, and have made the life of the users hassle-free. For example, the existing electronic devices like smartphones have been developed in such a manner that smartphones have decreased the requirement of carrying bulky cameras to capture the scene by the user input. The smartphones are now provided with a plurality of cameras, for example, an ultra-wide camera, a pair of telephoto cameras, and a wide camera which provide feasibility to the user to capture the best frames of the scene as per their requirement. Each camera as provided has different specifications and is configured to provide the best frame of the scene to be captured by the user input. The best frame may include a representative frame, a suitable frame or a single frame.
[0003] Further, capturing the best frame with the multiple cameras of the smartphone involves various factors, for example, selecting the best camera to capture the frame, and the optimum zoom ratio of the camera to capture the frame. Thus, there is a requirement that the user must be aware to use the plurality of cameras efficiently to capture the best frame of the scene such that the best frame is captured by the best camera from among the multiple cameras of the smartphone with the optimum zoom. However, there are limitations faced by the user inputs to use the plurality of cameras efficiently. The users, especially, amateur users, unlike professional users, are unable to use the plurality of cameras efficiently as they are unaware of the ways to use the specifications of the plurality of cameras efficiently. The amateur users further face several difficulties as mentioned below:
[0004] a. Amateur users face difficulty while placing the subject in the captured best frame for an optimum picture.
[0005] b. Amateur users face difficulty in determining the optimum zoom value considering the distance of the subject from the user and the corresponding camera to use for capturing the scene with the subject.
[0006] c. Amateur users face difficulty in determining the surrounding scene around the subject that needs to be captured for the best frame and to determine whether to include any secondary subject present in the scene.
[0007] Thus, the users are unable to decide the best frame, and the best camera with the best zoom to capture the best frame of the scene. Particularly, the best frame selection with the best aesthetic requires subjects that need to be captured in the image and also, noises to be removed from the subject. Thus, this requires constant practice and a degree of intuition by the user inputs. However, it is inconvenient for the amateur users to spend time learning and practicing the process of capturing the best frame. Therefore, there is a considerable gap in the aesthetics and the perspective of the scenes (images) captured by the amateur users as compared to the professional users, thus leading to imperfect capturing of the best frame of the scene from the plurality of cameras. Further, multiple examples are explained in the subsequent paragraphs for a better understanding of the problem as explained above:
[0008] In one example, as shown in Figure 1(i) the amateur users select the camera, i.e., the wide camera, from the plurality of cameras, to capture the best frame of the scene having texts. However, the best frame as captured includes the noise along with the texts, and further, the resolution of the texts is also unclear unlike the best frame captured by the professional users using the tele camera as shown in Figure 1(ii). Thus, there is a considerable gap in the aesthetics and the perspective of the scenes (images) captured by the amateur users as compared to the professional users, leading to imperfect capturing of the best frame of the scene from the plurality of cameras.
[0009] In another example, as shown in Figure 2(i), the amateur users select the camera, i.e., the ultra-wide camera, from the plurality of cameras, to capture the best frame of the scene. However, the best frame as captured includes flares at an edge of the best frame thereby completely reducing the quality and aesthetics of the best frame unlike the best frame captured by the professional users using the wide camera which eliminates the flares from the best frame as shown in Figure 2(ii) . Thus, there is a considerable gap in the aesthetics and the perspective of the scenes (images) captured by the amateur users as compared to the professional users, leading to imperfect capturing of the best frame of the scene from the plurality of cameras.
[0010] In yet another example, as shown in Figure 3(i), the amateur users select the camera, i.e., the tele camera, from the plurality of cameras, to capture the best frame of the scene. However, the best frame as captured misses the aesthetics of the scene. The best frame only captures the standalone lighthouse with no perspective of surroundings unlike the best frame captured by the professional users using the ultra-wide camera. The ultra-wide camera captures the lighthouse with the surroundings in the best frame thus increasing the aesthetics of the scene as shown in Figure 3(ii). Thus, there is a considerable gap in the aesthetics and the perspective of the scenes (images) captured by the amateur users as compared to the professional users, thus leading to imperfect capturing of the best frame of the scene from the plurality of cameras.
[0011] In yet another example, as shown in Figure 4(i), the amateur users select the camera, i.e., the tele camera with extreme zoom ratio, from the plurality of cameras, to capture the best frame of the scene. However, the best frame as captured loses part of the subject of interest unlike the best frame captured by the professional users using the telecamera with an appropriate zoom ratio. The best frame captures the subject of interest as shown in Figure 4(ii). Thus, there is a considerable gap in the aesthetics and the perspective of the scenes (images) captured by the amateur users as compared to the professional users, leading to imperfect capturing of the best frame of the scene from the plurality of cameras.
[0012] In yet another example, as shown in Figure 5(i), the amateur users select the camera, i.e., the tele camera with the appropriate zoom ratio, from the plurality of cameras, to capture the best frame of the scene. However, the best frame as captured loses part of the subject of interest as the camera is not optimally positioned to capture the best frame, unlike the best frame captured by the professional users using the tele camera with appropriate zoom ratio and appropriate region of interest. The best frame captures the subject of interest as shown in the Figure 5(ii). Thus, there is a considerable gap in the aesthetics and the perspective of the scenes (images) captured by the amateur users as compared to the professional users, leading to imperfect capturing of the best frame of the scene from the plurality of cameras.
[0013] Thus, it is evident from the above-mentioned discussion, that the same scene captured by different users may result in different output based on the configuration of the plurality of cameras used to capture the same scene as shown in Figure 6. Therefore, to capture the best frame of the scene, the user must spend a considerable amount of time learning to use the plurality of cameras to get the best frame of the scene which will be cumbersome for the user.
[0014] Many technological solutions have been developed to overcome the abovementioned problems. For instance, a known art discloses a method to crop and scale Region of Interest (ROI) from the captured best frame by an image editing tool may be used. However, this method has limitations, such as the cropping and scaling inherently reduce quality or size of the captured image / the captured best frame. Further, if the ROI of the best frame is small, then there is a significant reduction in the quality of the best frame while generating the best frame with the ROI. Further, during the cropping and scaling, hardware capabilities of the device cannot be used, for example, using the telephoto camera to produce a zoomed-in image as shown in Figure 7.
[0015] As per another known method, images may be edited with a generative AI model. The generative AI model is used to edit images where parts of the ROI are not captured in the images. However, this has a limitation such as the output of the editing of the image is based on a training model used by the software application, where in many cases the edited image loses the essence of the subject of interest which the user wanted to capture as shown in Figure 8.
[0016] As per another known method, image with an object may be captured by generating an aesthetic score with an aesthetic score generator 904. As seen in Figure 9, an object detection 902 may be employed to detect objects and the aesthetic score may be generated using the aesthetic score generator 904 after detecting the objects. However, this configuration has limitations, such that the aesthetic score generator 904 only considers a frame having object boundaries, thus, restricting the probability of capturing the best image by the camera. Further, the image having only the object of interest does not provide the best image to the user as shown in Figure 9.
[0017] Therefore, in view of the above-mentioned problems, it is advantageous to provide a system and a method that can overcome one or more above-mentioned problems.
[0018] This summary is provided to introduce a selection of concepts, in a simplified format, that are further described in the detailed description of the invention. This summary is neither intended to identify key or essential inventive concepts of the invention and nor is it intended for determining the scope of the invention.
[0019] The present disclosure discloses a system to capture an image with a camera of a device. The system includes a memory and at least one processor. The at least one processor is communicatively coupled with the memory. The at least one processor is configured to generate a plurality of representative images having a plurality of region of interests (ROIs). The at least one processor is configured to determine an aesthetic score of each representative image of the plurality of representative images. The at least one processor is configured to determine, for at least one representative image, a camera having a field of view (FOV) corresponding to an ROI of the at least one representative image. The at least one processor is configured to determine one or more crop parameters for the determined camera for aligning an FOV of a frame from the determined camera to the at least one representative image. The at least one processor is configured to determine a best frame based on the aesthetic score, a sensor deviation, and a crop deviation of the at least one representative image. The at least one processor is configured to capture the image corresponding to the determined best frame, by applying the determined one or more crop parameters to the frame of the determined camera.
[0020] In an embodiment, an electronic device for capturing an image is disclosed. The electronic device includes a memory and at least one processor. The at least one processor is communicatively coupled with the memory. The at least one processor is configured to generate a plurality of representative images having a plurality of ROIs. The at least one processor is configured to determine an aesthetic score for each representative image of the plurality of representative images. The aesthetic score of each of the representative image is greater than a pre-defined threshold aesthetic value and aesthetic value of a scene generated by a user by at least one camera from among a plurality of cameras of a device. The at least one processor is configured to determine for each of the plurality of representative images, a camera having a field of view corresponding to an ROI of the respective representative image and covers an FOV of the respective representative image. The at least one processor is configured to determine a sensor deviation for each of the plurality of representative images, by determining a deviation between the determined camera having the field of view (FOV) corresponding to the ROI of the respective representative image and the at least one camera from the plurality of cameras of the device used for selecting the scene by the user input. The at least one processor is configured to determine for each of the plurality of representative images, one or more crop parameters for a frame of the determined camera to align the frame of the determined camera to the respective representative image. The at least one processor is configured to determine a crop deviation for each of the plurality of representative images, by determining a deviation between a size of the ROI of the respective representative image and a size of the ROI of the scene selected by the user input. The at least one processor is configured to recommend a best frame from the plurality of representative images to the user based on the aesthetic score, the sensor deviation, and the crop deviation. The at least one processor is configured to capture the recommended best frame by applying the determined one or more crop parameters to the frame of the determined camera.
[0021] In an embodiment, also disclosed herein is a method to capture an image with a camera of a device. The method includes generating a plurality of representative images having a plurality of region of interests (ROIs). The method includes determining an aesthetic score for each representative image of the plurality of representative images. The method includes determining, for at least one representative image, a camera having a field of view (FOV) corresponding to an ROI of the at least one representative image. The method includes determining one or more crop parameters for the determined camera for aligning an FOV of a frame from the determined camera to the at least one representative image. The method includes determining a best frame based on the aesthetic score, a sensor deviation, and a crop deviation of the at least one representative image. Lastly, the method includes capturing the image corresponding to the determined best frame, by applying the determined one or more crop parameters to the frame of the determined camera.
[0022] In an embodiment, also disclosed herein is a method to capture an image. The method includes generating a plurality of representative images having a plurality of ROIs. The method includes determining an aesthetic score for each representative image of the plurality of representative images, where the aesthetic score of each of the representative image is greater than a pre-defined threshold aesthetic value and aesthetic value of a scene generated by a user by at least one camera from a plurality of cameras of a device. The method includes determining for each of the plurality of representative images, a camera having a field of view (FOV) corresponding to an ROI of the respective representative image and covers an FOV of the respective representative image. The method includes determining a sensor deviation for each of the plurality of representative images, by determining a deviation between the determined camera having the FOV corresponding to the ROI of the respective representative image and the at least one camera from the plurality of cameras of the device used for selecting the scene by the user input. The method includes determining for each of the plurality of representative images, one or more crop parameters for a frame of the determined camera to align the frame of the determined camera to the respective representative image. The method includes determining a crop deviation for each of the plurality of representative images, by determining a deviation between a size of the ROI of the respective representative image and a size of the ROI of the scene selected by the user input. The method includes recommending a best frame from the plurality of representative images to the user based on the aesthetic score, the sensor deviation, and the crop deviation. The method includes capturing the recommended best frame by applying the determined one or more crop parameters to the frame of the determined camera.
[0023] To further clarify the advantages and features of the present invention, a more particular description of the invention will be rendered by reference to specific embodiments thereof, which is illustrated in the appended drawings. It is appreciated that these drawings depict only typical embodiments of the invention and are therefore not to be considered limiting of its scope. The invention will be described and explained with additional specificity and detail with the accompanying drawings.
[0024] These and other features, aspects, and advantages of the present invention will become better understood when the following detailed description is read with reference to the accompanying drawings in which like characters represent like parts throughout the drawings, wherein:
[0025] Figures 1(i)- 9 illustrate images captured by a camera as example of prior art;
[0026] Figure 10 illustrates an environment of an electronic device communicably coupled with a device having a plurality of cameras, in accordance with an embodiment of the present disclosure;
[0027] Figure 11A illustrates a block diagram of the electronic device, in accordance with an embodiment of the present disclosure;
[0028] Figure 11B illustrates an exemplary embodiment of the electronic device, in accordance with an embodiment of the present disclosure;
[0029] Figures 12A-13F illustrate the generation of a plurality of test frames, in accordance with an embodiment of the present disclosure;
[0030] Figures 14A-14C illustrate an aesthetic score generation in the electronic device, in accordance with an embodiment of the present disclosure;
[0031] Figures 15A-15E illustrate a determined camera having a field of view (FOV) corresponding to an ROI of at least one representative image and a sensor deviation, in accordance with an embodiment of the present disclosure;
[0032] Figures 16A-16B illustrate one or more crop parameters of the determined camera, in accordance with an embodiment of the present disclosure;
[0033] Figures 17A-17C illustrate a sensor mode of the determined camera, in accordance with an embodiment of the present disclosure;
[0034] Figures 18A-18E illustrate a hardware override selection unit for determining a selection factor, in accordance with an embodiment of the present disclosure;
[0035] Figure 19 illustrates determining a best frame and capturing the image, in accordance with an embodiment of the present disclosure;
[0036] Figures 20A-20D illustrate an example of the electronic device, in accordance with an embodiment of the present disclosure;
[0037] Figures 21A-21B illustrate an alternate embodiment of the electronic device, in accordance with another embodiment of the present disclosure;
[0038] Figure 22 illustrates a flowchart depicting a method performed by the electronic device, in accordance with an embodiment of the present disclosure;
[0039] Figures 23A-23B illustrate a flowchart of a method performed by the electronic device, in accordance with another embodiment of the present disclosure; and
[0040] Figures 24A-24E illustrate multiple use cases of the electronic device, in accordance with an embodiment of the present disclosure.
[0041] Further, skilled artisans will appreciate that elements in the drawings are illustrated for simplicity and may not have necessarily been drawn to scale. Furthermore, in terms of the construction of the device, a plurality of components of the device may have been represented in the drawings by conventional symbols, and the drawings may show only those specific details that are pertinent to understanding the embodiments of the present invention so as not to obscure the drawings with details that will be readily apparent to those of ordinary skill in the art having the benefit of the description herein.
[0042] For the purpose of promoting an understanding of the principles of the invention, reference will now be made to the embodiment illustrated in the drawings and specific language will be used to describe the same. It will nevertheless be understood that no limitation of the scope of the invention is thereby intended, such alterations and further modifications in the illustrated electronic device, and such further applications of the principles of the invention as illustrated therein being contemplated as would normally occur to one skilled in the art to which the invention relates. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skilled in the art to which invention belongs. The electronic device and examples provided herein are illustrative only and not intended to be limiting.
[0043] For example, the term “some” as used herein may be understood as “none” or “one” or “more than one” or “all.” Therefore, the terms “none,” “one,” “more than one,” “more than one, but not all” or “all” would fall under the definition of “some.” It should be appreciated by a person skilled in the art that the terminology and structure employed herein are for describing, teaching, and illuminating some embodiments and their specific features and elements and therefore, should not be construed to limit, restrict, or reduce the spirit and scope of the present disclosure in any way.
[0044] For example, any terms used herein, such as “includes,” “comprises,” “has,” “consists,” and similar grammatical variants do not specify an exact limitation or restriction, and certainly do not exclude the possible addition of a plurality of features or elements, unless otherwise stated. Further, such terms must not be taken to exclude the possible removal of the plurality of the listed features and elements, unless otherwise stated, for example, by using the limiting language including, but not limited to, “must comprise” or “needs to include.”
[0045] Whether or not a certain feature or element was limited to being used only once, it may still be referred to as “plurality of features” or “plurality of elements” or “at least one feature” or “at least one element.” Furthermore, the use of the terms “plurality of” or “at least one” feature or element do not preclude there being none of that feature or element, unless otherwise specified by limiting language including, but not limited to, “there needs to be a plurality of…” or “plurality of elements is required.”
[0046] Unless otherwise defined, all terms, and especially any technical and / or scientific terms, used herein may be taken to have the same meaning as commonly understood by a person ordinarily skilled in the art.
[0047] Reference is made herein to some “embodiments.” It should be understood that an embodiment is an example of a possible implementation of any features and / or elements of the present disclosure. Some embodiments have been described for the purpose of explaining a plurality of the potential ways in which the specific features and / or elements of the proposed disclosure fulfill the requirements of uniqueness, utility, and non-obviousness.
[0048] Use of the phrases and / or terms including, but not limited to, “a first embodiment,” “a further embodiment,” “an alternate embodiment,” “one embodiment,” “an embodiment,” “multiple embodiments,” “some embodiments,” “other embodiments,” “further embodiment”, “furthermore embodiment”, “additional embodiment” or other variants thereof do not necessarily refer to the same embodiments. Unless otherwise specified, plurality of particular features and / or elements described in connection with plurality of embodiments may be found in one embodiment, or may be found in more than one embodiment, or may be found in all embodiments, or may be found in no embodiments. Although plurality of features and / or elements may be described herein in the context of only a single embodiment, or in the context of more than one embodiment, or in the context of all embodiments, the features and / or elements may instead be provided separately or in any appropriate combination or not at all. Conversely, any features and / or elements described in the context of separate embodiments may alternatively be realized as existing together in the context of a single embodiment.
[0049] Any particular and all details set forth herein are used in the context of some embodiments and therefore should not necessarily be taken as limiting factors to the proposed disclosure.
[0050] Embodiments of the present invention will be described below in detail with reference to the accompanying drawings.
[0051] Figure 10 illustrates an environment 1000 of an electronic device 1002 communicably coupled with a device 1004 having a plurality of cameras 1008, in accordance with an embodiment of the present disclosure. Figure 11A illustrates a block diagram of the electronic device 1002, in accordance with an embodiment of the present disclosure. Figure 11B illustrates an exemplary embodiment of the electronic device 1002, in accordance with an embodiment of the present disclosure. For the sake of brevity, Figures 10, 11A, and 11B are explained together.
[0052] In an embodiment, the electronic device 1002 and the device 1004 may be a smartphone, or any other electronic device having a plurality of cameras 1008 (1008a, 1008b, …, 1008n), without departing from the scope of the present disclosure. In an embodiment, the electronic device 1002 and the device 1004 may include the plurality of cameras 1008, without departing from the scope of the present disclosure. The electronic device 1002 may include the device 1004. The electronic device 1002 may include the plurality of cameras 1008.
[0053] In an embodiment, the user may be configured to select a scene with at least one camera from the plurality of cameras 1008. Further, the user may select one or more of a sensor ratio 1146, a zoom ratio 1148, and a capture size 1150 of the at least one camera, without departing from the scope of the present disclosure The plurality of cameras 1008 may include the ultra-wide camera, the wide camera, the tele camera etc., without departing from the scope of the present disclosure. Further, each of the camera may have different specifications, for example, sensor ratio, a zoom ratio, a capture size etc., without departing from the scope of the present disclosure. Further, simultaneously, the electronic device 1002 may receive an input from the device 1004 that the user has selected the scene with at least one camera. the electronic device 1002 may obtain an input from the device 1004 that the user has selected the scene with at least one camera. Thus, the electronic device 1002, for example, using a processor thereof, may operate on the input and generate a best frame to the user by cropping 1152 and scaling 1154, to capture an intelligent shot of the scene selected by the user input.
[0054] Particularly, the electronic device 1002 may be configured to capture an image with a camera from the plurality of cameras 1008, without departing from the scope of the present disclosure.
[0055] In an embodiment, the electronic device 1002 may include, but is not limited to, at least one processor (referred here as one or more processor and / or a processor) 1104, a memory 1108, and a plurality of modules 1112 among other examples which are explained in detail in subsequent paragraphs.
[0056] The electronic device 1002 may further include an Input / Output (I / O) interface 1010, and a communication circuitry 1106. In some embodiments, the electronic device 1002 may be integrated within the device 1004. In some embodiments, the electronic device 1002 may be remote from the device 1004 and communicably coupled with the device 1004. For instance, the electronic device 1002 may be integrated with as a standalone entity at a server / cloud architecture. In the embodiments where the electronic device 1002 is implemented as a standalone entity at a server / cloud architecture, the electronic device 1002 may be in communication with multiple devices to receive data from each of the multiple devices, and the details provided below with respect to the electronic device 1002 and the device 1004 is applicable for the electronic device 1002 and the multiple user devices as well.
[0057] In an exemplary embodiment, the processor 1104 may be operatively coupled to each of the I / O Interface 1110, the plurality of modules 1112, the communication circuitry 1106, and the memory 1108. In one embodiment, the processor 1104 may include a graphical processing unit (GPU) and / or an AI Engine (AIE). In one embodiment, the processor 1104 may include at least one data processor for executing processes in a virtual storage area network. The processor 1104 may include specialized processing units such as, integrated system (bus) controllers, memory management control units, floating point units, graphics processing units, digital signal processing units, etc. In one embodiment, the processor 1104 may include a central processing unit (CPU), a graphics processing unit (GPU), or both. The processor 1104 may be one or more general processors, digital signal processors, application-specific integrated circuits, field-programmable gate arrays, servers, networks, digital circuits, analog circuits, combinations thereof, or other now-known or later developed devices for analyzing and processing data. The processor 1104 may execute a software program, such as code generated manually (i.e., programmed) to perform the desired operation. The processor 1104 may include at least one processor. The processor 1104 may include at least one processor circuitry.
[0058] The processor 1104 may be disposed in communication with one or more input / output (I / O) devices via the I / O Interface 1110. In some embodiments, the processor 1104 may communicate with the device 1004 using the I / O Interface 1110. In some embodiments, the I / O Interface 1110 may be implemented within the device 1004. The I / O Interface 1110 may employ communication code-division multiple access (CDMA), high-speed packet access (HSPA+), global system for mobile communications (GSM), long-term evolution (LTE), New Radio(NR), Wi-Fi, Bluetooth, WiMax, or the like, etc. In an embodiment, the I / O Interface 1110 may enable input and output to and from the electronic device 1002 using suitable devices such as, but not limited to, display, keyboard, mouse, touch screen, microphone, speaker, and so forth.
[0059] Using the I / O Interface 1110, the electronic device 1002 may communicate with one or more I / O devices, specifically, the device 1004 having the plurality of cameras 1008, where the electronic device 1002 may capture the image with a camera from the plurality of cameras 1008 of the device 1004. The electronic device 1002 may obtain the image with a camera from the plurality of cameras 1008. For example, the input device may be an antenna, microphone, touch screen, touchpad, storage device, transceiver, video device / source, etc. The output devices may be a video display (e.g., cathode ray tube (CRT), liquid crystal display (LCD), light-emitting diode (LED), plasma, Plasma Display Panel (PDP), Organic light-emitting diode display (OLED) or the like), audio speaker, etc.
[0060] The processor 1104 may be disposed in communication with a communication network via a network interface. In an embodiment, the network interface may be the I / O Interface 1110. The network interface may connect to the communication network to enable the connection of the electronic device 1002 with the device 1004. The network interface may employ connection protocols including, without limitation, direct connect, Ethernet (e.g., twisted pair 10 / 100 / 1000 Base T), transmission control protocol / internet protocol (TCP / IP), token ring, IEEE 802.12A / b / g / n / x, etc. The communication network may include, without limitation, a direct interconnection, local area network (LAN), wide area network (WAN), wireless network (e.g., using Wireless Application Protocol), the Internet, etc. Using the network interface and the communication network, the electronic device 1002 may communicate with other devices. The network interface may employ connection protocols including, but not limited to, direct connect, Ethernet (e.g., twisted pair 10 / 100 / 1000 Base T), transmission control protocol / internet protocol (TCP / IP), token ring, IEEE 802.12A / b / g / n / x, etc.
[0061] The communication circuitry 1106 may be configured to receive and / or transmit signals to and from the device 1004. In one embodiment, the memory 1108 may be configured to store the information as required by the plurality of modules 1112 and the processor 1104 to perform one or more functions for capturing the image with the camera of the device 1004.
[0062] In an embodiments, the memory 1108 may be communicatively coupled to the processor 1004. The memory 1108 may be configured to store data, and instructions executable by the processor 1104. In one embodiment, the memory 1108 may be provided within the device 1004. In an embodiment, the memory 1108 may be provided within the electronic device 1002 being remote from the device 1004. In an embodiment, the memory 1108 may communicate with the processor 1104 via a bus within the electronic device 1002. In an embodiment, the memory 1108 may be located remote from the processor 1104 and may be in communication with the processor 1104 via a network. The memory 1108 may include, but is not limited to, a non-transitory computer-readable storage media, such as various types of volatile and non-volatile storage media including, but not limited to, random access memory, read-only memory, programmable read-only memory, electrically programmable read-only memory, electrically erasable read-only memory, flash memory, magnetic tape or disk, optical media and the like.
[0063] In one example, the memory 1108 may include a cache or random-access memory for the processor 1104. In alternative examples, the memory 1108 may be separate from the processor 1104, such as a cache memory of a processor, the system memory, or other memory. The memory 1108 may be an external storage device or database for storing data. The memory 1108 may be operable to store instructions executable by the processor 1104. The functions, acts, or tasks illustrated in the figures or described may be performed by the programmed processor 1104 for executing the instructions stored in the memory 1108. The functions, acts, or tasks are independent of the particular type of instruction set, storage media, processor, or processing strategy and may be performed by software, hardware, integrated circuits, firmware, micro-code, and the like, operating alone or in combination. Likewise, processing strategies may include multiprocessing, multitasking, parallel processing, and the like. The memory 1108 may store instructions. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to perform functions, acts, or tasks.
[0064] In some embodiments, the plurality of modules 1112 may be included within the memory 1108. The memory 1108 may further include a database to store data. The plurality of modules 1112 may include a set of instructions that may be executed to cause the electronic device 1002, in particular, the processor 1104 of the electronic device 1002, to perform any one or more of the methods / processes disclosed herein. The plurality of modules 1112 may be configured to perform the steps of the present disclosure using the data stored in the database. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to perform the steps of the present disclosure using the data stored in the database.
[0065] In an embodiment, each of the plurality of modules 1112 (modules 1112) may be a hardware unit that may be outside the memory 1108. Further, the memory 1108 may include an operating system for performing one or more tasks of the electronic device 1002, as performed by a generic operating system.
[0066] In one example, the modules 1112 may include an identifying module 1114, a generating module 1116, a determining module 1118, a comparing module 1120, a discarding module 1122, an updating module 1124, and a capturing module 1126. Each of the identifying module 1114, the generating module 1116, the determining module 1118, the comparing module 1120, the discarding module 1122, the updating module 1124, and the capturing module 1126 may be in communication with each other. Further, each of the identifying module 1114, the generating module 1116, the determining module 1118, the comparing module 1120, the discarding module 1122, the updating module 1124, and the capturing module 1126 may be in communication with the processor 1104.
[0067] Further, the present disclosure contemplates a computer-readable medium that includes instructions or receives and executes instructions responsive to a propagated signal. Further, the instructions may be transmitted or received over the network via a communication port or interface or using a bus (not shown). The communication port or interface may be a part of the processor 1104 or may be a separate component. The communication port may be created in software or may be a physical connection in hardware.
[0068] The communication port may be configured to connect with a network, external media, the display, or any other components in the system, or combinations thereof. The connection with the network may be a physical connection, such as a wired Ethernet connection, or may be established wirelessly. Likewise, the additional connections with other components of the electronic device 1002 may be physical or may be established wirelessly. The network may alternatively be directly connected to a bus. For the sake of brevity, the architecture and standard operations of the memory 1108, the processor 1104, the Communication circuitry 1106, and the I / O Interface 1110 are not discussed in detail.
[0069] Further, the working of the electronic device 1002 to capture the image using the camera of the device 1004 is explained subsequent paragraph. The working of the electronic device 1002 to obtain the image from the camera of the device 1004 is explained subsequent paragraph.
[0070] In an embodiment, a user may select a scene from at least one camera from the plurality of cameras 1008. The electronic device 1002 and the device 1004 may allow a user to select a scene from at least one camera among the plurality of cameras 1008. The user may have the option to select from the plurality of cameras 1008, for example, an ultra-wide camera, a wide camera, a tele camera, etc. The electronic device 1002 and the device 1004 may provide the user with an option to select from the plurality of cameras 1008, such as an ultra-wide camera, a wide camera, or a tele camera. Each camera has a different capture size, zoom ratio, and sensor value. In an embodiment, the at least one camera and optical zoom value of the at least one camera may be selected manually. Further, the processor 1104 may be configured to receive an input from the plurality of cameras 1008 of the device 1004. The processor 1104 may be configured to obtain an input from the plurality of cameras 1008 of the device 1004. Further, the processor 1104 may operate the input by performing the detection of one or more objects 1128, a face identification 1130, and a flare identification 1132. Further, the operated input may be sent to an aesthetic score generator 1134, where the processor 1104 may be configured to determine the aesthetic score of the operated input. Further, after determining the aesthetic score, the processor 1104 may be configured to determine a camera 1138 from the plurality of cameras 1008 matching with the operated input. Further, one or more crop parameters 1140 and a sensor mode from a plurality of sensor modes 1142 may be determined corresponding to the operated input. Lastly, the operated input may be received by a hardware override selection unit 1144 to determine a best frame which may be further cropped / scale to capture the image of the scene selected by at least one of the camera from the plurality of cameras 1008 of the device 1004. The operated input may be obtained by a hardware override selection unit 1144 to determine a best frame which may be further cropped / scale to capture the image of the scene selected by at least one of the camera from the plurality of cameras 1008 of the device 1004. Further, the detailed explanation of the operation performed by the electronic device 1002 is provided in the subsequent paragraphs.
[0071] The processor 1104, in conjunction with the identifying module 1114, the generating module 1116, the determining module 1118, the comparing module 1120, the discarding module 1122, the updating module 1124, and the capturing module 1126 may be configured to perform specific operations explained in subsequent paragraphs in conjunction with Figures 11A to 20D.
[0072] Figures 12A to 13F illustrate the generation of a plurality of test frames 1212 from a test frame generation 1136 process, in accordance with an embodiment of the present disclosure. For the sake of brevity, Figures 12A - 13F are explained together.
[0073] The generating module 1116 may be configured to generate a plurality of representative images having a plurality of region of interests (ROIs). The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to generate a plurality of representative images having a plurality of region of interests (ROIs). In an embodiment, elaborating further, the generating module 1116 may be configured to generate the plurality of representative images of the scene using a camera having a maximum field of view (FOV), for example, the ultra-wide camera, from the plurality of cameras 1008 in the device 1004. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to generate the plurality of representative images of the scene using a camera having a maximum field of view (FOV), for example, the ultra-wide camera, from the plurality of cameras 1008 in the device 1004. Further, the plurality of generated representative images may be independent of the scene selected by the user input.
[0074] Further, in an embodiment, the test frame generation unit 1136 may be configured to generate the plurality of test frames 1212 depending on the plurality of representative images. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to generate the plurality of test frames 1212 depending on the plurality of representative images. In an embodiment, the generating module 1116 may generate the plurality of test frames 1212 corresponding to the plurality of representative images. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to generate the plurality of test frames 1212 corresponding to the plurality of representative images.
[0075] In an embodiment, referring to Figure 12A, in the test frame generation unit 1136, prior to generating the plurality of test frames 1212, the comparing module 1120 may be configured to compare an aspect ratio 1204 of the one or more objects 1128 detected by the object detection unit and the scene selected by the user input. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to compare an aspect ratio 1204 of the one or more objects 1128 detected by the object detection unit and the scene selected by the user input. In an embodiment, the comparing module 1120 may be configured to compare the aspect ratio of the selected scene by the user input with the aspect ratio of ROI of the one or more objects 1128 for generating an initial frame / test frame. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to compare the aspect ratio of the selected scene by the user input with the aspect ratio of ROI of the one or more objects 1128 for generating an initial frame / test frame. Particularly, the user may capture images with several aspect ratios such as 4:3, 16:9, 1:1, etc. The electronic device 1002 and the device 1004 may capture images with several aspect ratios such as 4:3, 16:9, 1:1, etc. Further, the ROI of the one or more objects 1128 detected by the object detection unit, may not be the same as the aspect ratio of the selected scene by the user input. Thus, the comparing module 1120, after receiving the aspect ratio of the ROI of the one or more objects 1128, may compare and adjust the ROI of the one or more objects 1128 to match the aspect ratio of the selected scene by the user input to generate the initial frame. After receiving the aspect ratio of the ROI of the one or more objects 1128, the instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to compare and adjust the ROI of the one or more objects 1128 to match the aspect ratio of the selected scene by the user input for generating the initial frame.
[0076] In one example, referring to Figure 12B(i), when the ROI of the one or more objects 1128 may be less than the aspect ratio of the selected scene by the user input, then a width of the ROI of the one or more objects 1128 may increase to include additional surrounding area in a left and right side of the ROI of the one or more objects 1128. when the ROI of the one or more objects 1128 may be less than the aspect ratio of the selected scene by the user input, the instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to increase a width of the ROI of the one or more objects 1128 to include additional surrounding area in a left and right side of the ROI of the one or more objects 1128. Thus, the initial test frame may be generated with a center-aligned object denoted by X1.0, where the aspect ratio of the scene selected by the user input may be W_U / H_U generated by the equation W+W” / H =W_U / H_U.
[0077] In an example, referring to Figure 12B(ii), when the ROI of the one or more objects 1128 may be more than the aspect ratio of the selected scene by the user input, then a height of the ROI of the one or more objects 1128 may increase to include additional surrounding area in a top and bottom side of the ROI of the one or more objects 1128. when the ROI of the one or more objects 1128 may be more than the aspect ratio of the selected scene by the user input, the instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to increase a height of the ROI of the one or more objects 1128 to include additional surrounding area in a top and bottom side of the ROI of the one or more objects 1128. Thus, the initial test frame may be generated with a center-aligned object denoted by X1.0, where the aspect ratio of the scene selected by the user input may be W_U / H_U generated by W+ / H +H” =W_U / H_U.
[0078] Further, the generating module 1116 may be configured to generate the remaining plurality of test frames based on a category of the one or more objects 1128 in the initial test frame, without departing from the scope of the present disclosure. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to generate the remaining plurality of test frames based on a category of the one or more objects 1128 in the initial test frame. A plurality of examples of the plurality of test frames may be provided below:
[0079] In one example, referring to Figure 12C(i), a center-aligned test frame may be generated, i.e., X1.5, X2.0 etc. Further, X1.5 refers to the test frame which may be generated with 1.5 times of the width (1.5*W) and height (1.5*H) of the initial frame (X1.0) as generated in Figure 12(B(ii)). The additional area / surrounding area from a scene around the one or more objects 1128 may be included in the test frame in all directions.
[0080] In another example, referring to Figure 12C(ii), a corner-aligned test frame e.g., X1.5, X2.0, etc., may be generated. The corner-aligned test frame may have the one or more objects 1128 in any corner based on image classification defined in the look-up table, where the test frame may have 1.5 times the width (1.5*W) and height (1.5*H) of the initial frame (X1.0) as generated in the Figures 12(B(i)). Thus, based on the required test frame configuration, additional scene around the one or more objects 1128 may be included in any direction to generate the remaining test frames.
[0081] Referring to Figure 12D, in an embodiment, the user may select the scene using the at least one camera from the plurality of cameras 1008 in the device 1004. The device 1004 may allow the selection of the scene using at least one camera from the plurality of cameras 1008 based on the user input. At step 1214, the electronic device 1002, specially, the processor 1104 may receive notification that the user has selected the scene to capture. The processor 1104 may receive a notification that the scene to capture has been selected based on user input. The processor 1104 may obtain the notification that the scene to capture has been selected based on user input. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to obtain the notification that the scene to capture has been selected based on user input. Further, at step 1216, the processor 1104 may determine whether the ultra-wide (UW) camera is selected for capturing the scene. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to determine whether the ultra-wide (UW) camera is selected for capturing the scene. Further, at step 1220, when the ultra-wide camera is selected for capturing the scene, the processor 1104 determines whether the ultra-wide camera is used with no zoom. when the ultra-wide camera is selected for capturing the scene, the instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to determine whether the ultra-wide camera is used with no zoom. Further, when the ultra-wide camera is not selected for capturing the scene and / or when the ultra-wide camera is used with the zoom, the processor 1104 may issue a command to start the ultra-wide camera with no zoom to capture frame for analysis at step 1218. when the ultra-wide camera is not selected for capturing the scene and / or when the ultra-wide camera is used with the zoom, the instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to generate a command to start the ultra-wide camera with no zoom to capture frame for analysis at step 1218. At step 1222, the processor 1104 may generate a command to the ultra-wide camera to take the same image / scene as selected by the user input for analysis, if the ultra-wide camera is used with no zoom. At step 1222, the instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to generate a command to the ultra-wide camera to take the same image / scene as selected by the user input for analysis, if the ultra-wide camera is used with no zoom. Thus, the instructions may be obtained by the device 1004. Further, a frame of the ultra-wide camera from the device 1004 is provided to the processor 1104 for further operation and finally generates either an intelligent shot of the scene or the user shot. Further, the processor 1104 in conjunction with the modules 1112 may be configured to perform the following operations, after receiving the frame, as mentioned in the subsequent paragraphs.
[0082] Further, in an embodiment, the identifying module 1114 may be configured to identify the one or more objects 1128 in the frame of the camera having the maximum FOV, for example, the ultra-wide camera. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to identify the one or more objects 1128 in the frame of the camera having the maximum FOV, for example, the ultra-wide camera. Further, the generating module 1116 may be configured to generate the plurality of test frames 1212 corresponding to the plurality of representative images. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to generate the plurality of test frames 1212 corresponding to the plurality of representative images. Each test frame from the plurality of test frames 1212 may include the one or more objects 1128 aligned at one of a plurality of positions. In an embodiment, the generation of the plurality of test frames 1212 with the plurality of positions of the one or more objects 1128 may be based on a category of the one or more objects 1128 as shown in Figures 13A-13D. Further, the plurality of positions of the one or more objects 1128 based on the category of the one or more objects 1128 may be determined as per a frame generation look-up table 1206 provided below in Table 1:
[0083] ObjectCategoryTest Frames GeneratedCenter AlignedRight-Top AlignedRight-Bottom AlignedLeft-Top AlignedLeft-Bottom AlignedTree, House, Light-HouseObject Boundary[X1, X1.5, X2.0, X3.0]Object Boundary[X.5]Object Boundary[X1, X1.5, X2.0, X3.0]Object Boundary[X.5]Object Boundary[X1, X1.5, X2.0, X3.0]Moon, Sun, BirdObject Boundary[X1, X1.5, X2.0, X3.0]Object Boundary[X1, X1.5, X2.0, X3.0]Object Boundary[X.5]Object Boundary[X1, X1.5, X2.0, X3.0]Object Boundary[X.5]....................................
[0084] In an embodiment, the frame generation look-up table 1206 may be used by the modules 1112 in generating the required number of the plurality of test frames 1212 based on the plurality of factors, i.e., the category of the one or more objects 1128 and the plurality of positions of the one or more objects 1128 based on the category of the one or more objects 1128. Particularly, the frame generation look-up table 1206 or the modules 1112 may include a pre-trained neural network configured to determine specific classes / category of the one or more objects 1128. The electronic device 1002 may include a pre-trained neural network configured to determine specific classes / category of the one or more objects 1128. This configuration may consider the frame to detect different classes / categories of the one or more objects 1128 detected in the frame and generates a list of the category of the one or more objects 1128 detected with the corresponding coordinates within the frame. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to analyze the frame to detect different classes or categories of one or more objects 1128 detected in the frame and generate the list of the categories of the detected objects 1128 along with the coordinates of the detected objects 1128 within the frame. Further, the generation of the plurality of test frames 1212 based on the category of the one or more objects 1128 and the plurality of positions of the one or more objects 1128 based on the category of the one or more objects 1128 may be explained with a plurality of examples as provided below:
[0085] In one example, referring to Figure 13A(i-v), the category of one or more objects 1128 may include a tree. Thus, the plurality of test frame 1212 as generated may include the one or more objects 1128 positioned in several positions, i.e., the center aligned position , the right-bottom position, and the left-bottom position. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to generate the plurality of test frame 1212, including one or more objects 1128 positioned in various positions, such as the center aligned position, the right-bottom position, and the left-bottom position. When the determined category of the one or more objects 1128 is a tree, as shown in Figure 13A(i) or Figure 13A(ii), the instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to generate a test frame 1212 with the center-aligned position. When generating the test frame 1212, the instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to make the tree in Figure 13A(i) appear visually closer than the tree in Figure 13A(ii). When the determined category of the one or more objects 1128 is a tree, as shown in Figure 13A(iii) or Figure 13A(iv), the instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to generate a test frame 1212 with the right-bottom position. When generating the test frame 1212, the instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to make the tree in Figure 13A(iii) appear visually closer than the tree in Figure 13A(iv). When the determined category of the one or more objects 1128 is a tree, as shown in Figure 13A(v), the instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to generate a test frame 1212 with the left-bottom position.
[0086] In an example, referring to Figure 13B(i-v), the category of one or more objects 1128 may include a moon. Thus, the plurality of test frames 1212 as generated may include the one or more objects 1128 positioned in several positions, i.e., the center aligned position, the right-top position, and the left-top position. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to generate the plurality of test frames 1212, including the one or more objects 1128 positioned in various positions, such as the center aligned position, the right-top position, and the left-top position. When the determined category of the one or more objects 1128 is the moon, as shown in Figure 13B(i), Figure 13B(ii) or Figure 13B(iii), the instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to generate a test frame 1212 with the center-aligned position. When generating the test frame 1212, the instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to make the moon in Figure 13B(i) appear visually closer than the moon in Figure 13B(ii) or Figure 13B(iii). When the determined category of the one or more objects 1128 is the moon, as shown in Figure 13b(iv), the instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to generate a test frame 1212 with the left-top position. When the determined category of the one or more objects 1128 is the moon, as shown in Figure 13B(v), the instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to generate a test frame 1212 with the right-top position.
[0087] In an example, referring to Figure 13CA(i), the ROI of each object may be determined from a complete image. Further, the ROI of a combination of one or more objects 1128 may be determined and accordingly, the plurality of test frames 1212 may be generated. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to determine the ROI of each object from the complete image. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to determine the ROI of the combination of one or more objects 1128 and generate the plurality of test frames 1212.
[0088] In an example, referring to Figure 13CA(ii) and Figure 13D, the one or more objects 1128 may be center aligned. Thus, the ROIs of the one or more objects 1128 may be increased in all directions and accordingly, the plurality of test frames 1212 may be generated. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to increase the ROIs of one or more objects 1128 in all directions and generate the plurality of test frames 1212.
[0089] In an example, referring to Figure 13CA(iii) and the Figure 13D, the one or more objects 1128 may be aligned in the corner. Thus, the ROIs of the one or more objects 1128 may be increased in one of all the directions and accordingly, the plurality of test frames 1212 may be generated. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to increase the ROIs of one or more objects 1128 in one of all directions and generate the plurality of test frames 1212.
[0090] In an example, referring to Figure 13CB(iv) and Figure 13D, the one or more objects 1128 may be combined, and thus, three sets of the plurality of test frames 1212 of each object may be generated. Further, ROI of the plurality of test frames 1212 may be overlapped to the one or more objects 1128. Further, the combination of the test frames as generated may be dependent on the category of the one or more objects 1128. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to overlap the ROI of the plurality of test frames 1212 with one or more objects 1128. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to generate the combination of test frames based on the category of one or more objects 1128.
[0091] In an example, referring to Figure 13CB(v), a frame selected by the user input for selecting the scene by the at least one camera may be provided.
[0092] Thus, by considering the abovementioned examples, the plurality of test frames 1212 may be dependent on the category of the one or more objects 1128. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to generate the plurality of test frames 1212 based on the category of one or more objects 1128.
[0093] Further, in an embodiment and referring to Figure 13E, the identifying module 1114 may be configured to identify one or more faces 1302 in the plurality of representative images from the camera having the maximum FOV, for example, the ultra-wide camera. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to identify one or more faces 1302 in the plurality of representative images from the camera having the maximum FOV such as the ultra-wide camera. The generating module 1116 may be configured to generate the plurality of test frames 1212 corresponding to the plurality of representative images. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to generate the plurality of test frames 1212 corresponding to the plurality of representative images. Further, each test frame may include at least one identified face, without departing from the scope of the present disclosure. This configuration provides face regions in the frame and provides the number and coordinates of the faces as detected. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to provide face regions in the frame and determine the number and coordinates of the faces as detected. In an embodiment, the face identification / detection may be also performed by a hardware accelerated module or a software module, without departing from the scope of the present disclosure. The identification of one or more faces 1302 ensures that all faces are captured in the final image captured by the device 1004. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to confirm that all faces are included in the final image captured by the device 1004.
[0094] Additionally, if the identification of the one or more faces 1302 may not be performed, then this results in the generation of the plurality of test frames 1212 with incomplete faces. If the identification of the one or more faces 1302 may not be performed, the instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to generate the plurality of test frames 1212 with incomplete faces. Further, the plurality of test frames 1212 as generated may include the highest aesthetic score value which may be given to the user, where faces of the individuals may not be present, thus compromising the experience of the user capturing the scene. Therefore, the identification of the one or more faces 1302 is performed by the electronic device 1002 to eliminate above-mentioned shortcomings.
[0095] In one example, referring to Figure 13E, during the generation of the plurality of test frames 1212 by the camera having the maximum FOV from the plurality of cameras 1008, the identification of the one or more faces 1302 may be performed on the plurality of test frames 1212. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to perform the identification of one or more faces 1302 on the plurality of test frames 1212. Subsequently, the plurality of test frames 1212 as generated may contain the one or more identified faces. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to generate the plurality of test frames 1212 containing one or more identified faces. Further, a test frame “T” as shown may be never generated and no further operations may happen for the test frame “T”, since frame T only covers two out of the four faces present in the plurality of test frames 1212.
[0096] In an embodiment, the identifying module 1114 may be configured to identify one or more flares 1304 in the plurality of representative images from the camera having the maximum FOV, for example, the ultra-wide camera. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to identify one or more flares 1304 in the plurality of representative images from the camera having the maximum FOV such as the ultra-wide camera. The generating module 1116 may be configured to generate the plurality of test frames 1212 corresponding to the plurality of representative images. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to generate the plurality of test frames 1212 corresponding to the plurality of representative images. Further, at least one test frame from the plurality of test frames 1212 including the identified at least one flare may be disregarded. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to disregard at least one test frame from the plurality of test frames 1212 that includes the identified at least one flare. This configuration detects the at least flare in the frame and determines the co-ordinates of the flare. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to detect the at least flare in the frame and determine the co-ordinates of the flare. In an embodiment, the flare detection / identification may be also performed by a hardware accelerated module or a software module, without departing from the scope of the present disclosure. The disregarding of the frame having the at least one flare results in a high-quality image suggested by the electronic device 1002. The disregarding of the frame may ensure that the plurality of test frames 1212 with high score may be considered for the final shot of the scene, thereby decreasing overall processing delay.
[0097] In one example, referring to Figure 13F, during the generation of the plurality of test frames 1212 by the camera having the maximum FOV from the plurality of cameras 1008, then the identification of the flares may be performed on the plurality of test frames 1212. During the generation of the plurality of test frames 1212 by the camera having the maximum FOV from the plurality of cameras 1008, the instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to perform the identification of the flares on the plurality of test frames 1212. Subsequently, if the flare may be detected in the test frame, thus the plurality of test frames 1212 may be generated by the next camera having the maximum FOV to avoid the creation of an unnecessary test frame having the flare. If the flare may be detected in the test frame, the instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to generate the plurality of test frames 1212 using the next camera with the maximum FOV to avoid the creating an unnecessary test frame having the flare. Further, a test frame “T” as shown may be never generated and no further operations may happen for the test frame “T”, since the frame T includes the flares.
[0098] Figures 14A to 14C illustrate aesthetic score generation in the electronic device 1002, in accordance with an embodiment of the present disclosure.
[0099] In an embodiment, an aesthetic score generator 1402 may be configured to identify the aesthetic score of the plurality of test frames 1212. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to identify the aesthetic score of the plurality of test frames 1212. The aesthetic score of the plurality of test frames 1212 may be dependent on several factors. The aesthetic score of the plurality of test frames 1212 may be based on a plurality of factors. The several factors may include global features and local features. The plurality of factors may include global features and local features. In an embodiment, the global features may include the type of the one or more objects 1128 and the position of the one or more objects 1128. Further, the local features may include colour combination, contrast, saturation etc. of the plurality of test frames 1212. In an embodiment, the aesthetic score generator 1402 may be associated with a convolution neural network model which may be used for determining the aesthetic score. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to extract features from the plurality of test frames 1212 and determine the aesthetic score based on the features. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to extract features from the plurality of test frames 1212 using the convolution neural network model or a deep learning model. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to determine the aesthetic score based on the one or more objects 1128 and color properties of the plurality of test frames 1212. The electronic device 1002 may be associated with a convolution neural network model which may be used for determining the aesthetic score. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to determine the aesthetic score using the convolution neural network based on following features. The aesthetic score generator 1402 using the convolution neural network may determine the aesthetic score by following features:
[0100] - Feature extraction 1408 from the image 1404 - Features may be parts or patterns of the one or more objects 1128 in the image 1404 that help to identify the image 1404. The identification may be performed by using an image processing techniques for feature extraction identification of foreground and background objects in the image 1404. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to extract feature from the image 1404. The features may be parts or patterns of the one or more objects 1128 in the image 1404 that contribute to identifying the image 1404.
[0101] - Determination of a colour histogram 1410 of the image 1404 - In image processing and photography, the colour histogram 1410 may be a representation of the distribution of colors in the image 1404. For digital images, the colour histogram may represent the number of pixels that have colours in each of a fixed list of colour ranges. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to determine a colour histogram 1410 of the image 1404.
[0102] - Assigning a score to the image 1404 based on the various features of the image frame. Further, in the illustrated embodiment, the score 1412 may be assigned between 1 to 10, without departing from the scope of the present disclosure. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to assign a score to the image 1404 based on the various features of the image frame.
[0103] Further, the generation of the aesthetic score is explained in subsequent paragraphs.
[0104] In an embodiment, referring to Figure 11 to 14C, the determining module 1118 may be configured to determine an aesthetic score for each representative image of the plurality of representative images. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to an aesthetic score for each representative image of the plurality of representative images. In an embodiment, the aesthetic score of the representative images may be defined as an optimum framing of the plurality of representative images.
[0105] In an embodiment, the determining module 1118 may be configured to determine an aesthetic score of each test frame corresponding to each representative image. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to determine the aesthetic score of each test frame corresponding to each representative image.
[0106] In an embodiment, the comparing module 1120 may be configured to compare the aesthetic score of each test frame with one or more of a pre-defined threshold score and an aesthetic score of a reference image or an aesthetic score of the scene selected by the user input. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to compare the aesthetic score of each test frame with one or more of a pre-defined threshold score and an aesthetic score of a reference image or an aesthetic score of the scene selected by the user input. In an embodiment, the pre-defined threshold score may be stored in an aesthetic cut-off filter 1416, without departing from the scope of the present disclosure. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to control the memory 1108 to store the pre-defined threshold score. In an embodiment, the aesthetic score of the reference image may be stored in a reference cut-off filter 1414, without departing from the scope of the present disclosure. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to control the memory 1108 to store the aesthetic score of the reference image. The identifying module 1114 may be configured to identify at least one test frame, corresponding to the at least one representative image, including the aesthetic score greater than the one or more of the pre-defined threshold aesthetic score and the aesthetic score of the reference image or the aesthetic score of the scene selected by the user input as shown in table M of Figure 14C. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to identify at least one test frame, corresponding to the at least one representative image, including the aesthetic score greater than the one or more of the pre-defined threshold aesthetic score and the aesthetic score of the reference image or the aesthetic score of the scene selected by the user input.
[0107] In an embodiment, the reference image may include a frame of the camera having the maximum FOV and aligned to an FOV of the scene selected by the user input, without departing from the scope of the present disclosure. Particularly, a reference cut-off may be updated from the aesthetic score of the scene selected by the user input. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to update the reference cut-off based on the aesthetic score of the scene selected by the user input. Further, the reference image may use the user configuration and static capability information of the device 1004. The reference image 1210 may be generated by the processor 1104 in conjunction with the modules 1112 performing the following operations:
[0108] Operation 1: determining the at least one camera selected by the user input and user-selected zoom from the device 1004.
[0109] Operation 2: getting the optical zoom ratio of the at least one camera and the ultra-wide camera from the device 1004. and
[0110] Operation 3: mapping the frame of the at least one camera to the ultra-wide camera frame (the camera having the maximum FOV).
[0111] The operation to determine the reference image may be explained with an example:
[0112] In one example, data as provided below may be derived from the device 1004. An optical zoom reference of the at least one camera w.r.t. the camera having the maximum FOV (ultra-wide camera) may be OZR_U. A digital zoom applied to the at least one camera by the user input may be DZ_USR. Further, the full size of the sensor frame of the camera having the maximum FOV i.e. ultra-wide camera may be [ W_UW, H_UW]. A user reference frame co-ordinates from the ultra-wide camera may be [ X_UR, Y_UR, W_UR, H_UR]. The digital zoom of the user reference frame w.r.t the ultra-wide camera may be DZ_UW and further the reference image may be determined as:
[0113] - DZ_UW = DZ_USR * OZR_U
[0114] - W_UR = W_UW / DZ_UW
[0115] - H_UR = H_UW / DZ_UW
[0116] - X_UR = (W_UW - W_UR) / 2
[0117] - Y_T2 = (H_UW - H_UR) / 2
[0118] The discarding module 1122 may be configured to discard one or more test frames from the plurality of test frames 1212 having the aesthetic score less than the one or more pre-defined threshold aesthetic score or the aesthetic score of the reference image of the aesthetic score of the scene selected by the user input as shown in Figure 14C. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to discard one or more test frames from the plurality of test frames 1212 having the aesthetic score less than the one or more pre-defined threshold aesthetic score or the aesthetic score of the reference image of the aesthetic score of the scene selected by the user input. In one example, the pre-defined threshold aesthetic score REF_AS may be 4, thus, the plurality of test frames less than 4 may be discarded. Further, a test Map M (as shown in Figure 14C) may be generated with the plurality of test frames greater than 4 (as shown in Figure 14A) that may be considered for further operation as shown in Figure 14C. This configuration ensures an optimum frame to capture and also increases the probability of capturing the object of interest in the scene as selected by the user input.
[0119] Figures 15A to 15E illustrate the determined camera 1138 having a field of view (FOV) corresponding to an ROI of the at least one representative image and a sensor deviation, in accordance with an embodiment of the present disclosure.
[0120] In an embodiment, the determined camera 1138 may be used for capturing the identified at least one test frame. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to use the determined camera 1138 to capture the identified at least one test frame.
[0121] In such embodiment, the determining module 1118 may be configured to determine, for at least one representative image, the camera 1138 having the field of view (FOV) corresponding to the ROI of the at least one representative image. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to determine, for at least one representative image, the camera 1138 having the field of view (FOV) corresponding to the ROI of the at least one representative image. The determination of the camera 1138 ensures optimum capturing of the image along with optimum zoom by the camera 1138. The electronic device 1002 may determine the camera 1138 to facilitate optimum capturing of the image along with optimum zoom by the camera 1138.
[0122] In an embodiment, to determine the camera 1138 from the plurality of cameras 1008, the identifying module 1114 may be configured to identify the camera 1138 from the plurality of cameras 1008 having a maximum optical zoom ratio that covers a complete FOV of the at least one representative image in the FOV of the identified camera 1138. to determine the camera 1138 from the plurality of cameras 1008, the instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to identify the camera 1138 from the plurality of cameras 1008 having a maximum optical zoom ratio that covers a complete FOV of the at least one representative image in the FOV of the identified camera 1138. Further, the concept of determining the camera 1138 may be explained with examples as provided below:
[0123] In one example, referring to Figure 15A, the generated plurality of test frames 1212 may be provided to the aesthetic score generator 1402. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to provide the generated plurality of test frames 1212 to the aesthetic score generator 1402. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to generate the aesthetic score based on the generated plurality of test frames 1212. The plurality of test frames 1212 may be generated from the camera, for example, the ultra-wide camera. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to generate the plurality of test frames 1212 using the camera, such as the ultra-wide camera. Further, a test frame T1 from the plurality of test frames 1212 may be selected as a best frame and may be within the field of view of the ultra-wide camera. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to select the test frame T1 from the plurality of test frames 1212 as the best frame, which may be within the field of view of the ultra-wide camera. The at least one camera as selected by the user input may be a wide camera. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to select the at least one camera based on the user input, which may be the wide camera. Thus, the determined camera 1138 may be the ultra-wide camera which may be optimum for capturing the scene, without departing from the scope of the present disclosure. Further, the determined camera 1138 ensures that the field of view of the plurality of test frames 1212 may be the same as the field of view of the at least one camera from the plurality of cameras 1008 and thus captures the best frame.
[0124] In another example, referring to Figure 15A, the generated plurality of test frames 1212 may be provided to the aesthetic score generator 1402. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to provide the generated plurality of test frames 1212 to the aesthetic score generator 1402. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to generate the aesthetic score based on the generated plurality of test frames 1212. The plurality of test frames 1212 may be generated from the camera. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to generate the plurality of test frames 1212 from the camera. Further, a test frame T2 from the plurality of test frames 1212 may be selected as a best frame and may be within the field of view of the wide camera. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to select the test frame T2 from the plurality of test frames 1212 as the best frame, which may be within the field of view of the wide camera. The at least one camera as selected by the user input may be the ultra-wide camera. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to select the at least one camera based on the user input, which may be the ultra-wide camera. Thus, the determined camera 1138 may be the wide camera which may be optimum for capturing the scene, without departing from the scope of the present disclosure. Further, the determined camera 1138 ensures that the field of view of the plurality of test frames 1212 may be the same as the field of view of the at least one camera from the plurality of cameras 1008, avoiding unnecessary cropping, scaling and thus captures the best frame.
[0125] In yet another example, referring to Figure 15B, the generated plurality of test frames 1212 may be provided to the aesthetic score generator 1402. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to provide the generated plurality of test frames 1212 to the aesthetic score generator 1402. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to generate the aesthetic score based on the generated plurality of test frames 1212. The plurality of test frames 1212 may be generated from the camera. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to generate the plurality of test frames 1212 from the camera. Further, a test frame T3 from the plurality of test frames 1212 may be selected as a best frame within the field of view of the wide camera. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to select the test frame T3 from the plurality of test frames 1212 as the best frame, which may be within the field of view of the wide camera. The at least one camera as selected by the user input may be a tele camera. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to select the at least one camera based on the user input, which may be the tele camera. Thus, the determined camera 1138 may be the wide camera which may be optimum for capturing the scene, without departing from the scope of the present disclosure. Further, the determined camera 1138 ensures that the field of view of the plurality of test frames 1212 may be the same as the field of view of the at least one camera from the plurality of cameras 1008 and thus captures the best frame.
[0126] In yet another example, the generated plurality of test frames 1212 may be provided to the aesthetic score generator 1402. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to provide the generated plurality of test frames 1212 to the aesthetic score generator 1402. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to generate the aesthetic score based on the generated plurality of test frames 1212. The plurality of test frames 1212 may be generated from the camera. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to generate the plurality of test frames 1212 from the camera. Further, a test frame T4 from the plurality of test frames 1212 may be selected as a best frame and may be within the field of view of the tele camera. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to select the test frame T4 from the plurality of test frames 1212 as the best frame, which may be within the field of view of the tele camera. The at least one camera as selected by the user input may be the wide camera. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to select the at least one camera based on the user input, which may be the wide camera. Thus, the determined camera 1138 may be the tele camera which may be optimum for capturing the scene, without departing from the scope of the present disclosure. Further, the determined camera 1138 ensures that the field of view of the plurality of test frames 1212 may be same as the field of view of the at least one camera from the plurality of cameras 1008, avoiding unnecessary cropping, and scaling and thus captures the best frame.
[0127] In an embodiment, the plurality of test frames 1212 may be always generated by the camera having the maximum FOV, for example, the ultra-wide camera. Further, the region of interest (ROI) of the representative images may be a sub-set of the field of view of the camera having the maximum FOV. Thus, the camera 1138 as determined may have the maximum optical zoom ratio, since the remaining test frames from the plurality of test frames 1212 may be zoomed in with respect to the plurality of test frames 1212 generated by the ultra-wide sensor having minimum optical zoom ratio.
[0128] In one example, referring to Figure 15C(i), the determined camera 1138 may have the maximum optical zoom. Subsequently, the determined camera 1138 may be matched with the tele frame. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to match the determined camera 1138 with the tele frame. However, the identified at least one test frame T 1502 may be more than the tele frame. The identified at least one test frame T 1502 may be larger than the tele frame. Thus, the determined camera 1138 may not be able to capture the entire region of the test frame T 1502. Therefore, again the determined camera 1138 may be matched with the test frame T 1502. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to match the determined camera 1138 with the test frame T 1502. Further, the determined camera 1138 matches with the test frame T 1502, thus the further operation of matching the determined camera 1138 may not be carried out.
[0129] In one example, referring to Figure 15C(ii), the determined camera 1138 may have the maximum optical zoom. Subsequently, the determined camera 1138 may be matched with the tele frame. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to match the determined camera 1138 with the tele frame. The determined camera 1138 matches with the test frame T 1502, thus further operation of matching the determined camera may not be carried out.
[0130] In one example, a method for determining camera 1138 may be explained below with reference to Figure 15D:
[0131] The plurality of cameras 1008 may include ultra-wide camera (0.5X), wide camera (1.0X), tele1 camera (3X), tele2 camera (10X). Further, the data provided below may be static capabilities of the plurality of cameras 1008. Therefore,
[0132] - Optical Zoom Reference of the wide camera with respect to (w.r.t.) the ultra-wide camera may be OZR_W (i.e. 1 / 0.5 =2). The Optical Zoom Reference of the wide camera with respect to the ultra-wide camera may be defined as OZR_W (i.e. 1 / 0.5 =2).
[0133] - Optical Zoom Reference of the tele1 camera w.r.t. the ultra-wide camera may be OZR_T1 (i.e. 3 / 0.5 =6). The Optical Zoom Reference of the tele1 camera with respect to the ultra-wide camera may be defined as OZR_T1 (i.e. 3 / 0.5 =6).
[0134] - At step 1504, Optical Zoom Reference of the tele2 camera with respect to (w.r.t.) the ultra-wide camera is determined which may be OZR_T2 (i.e. 10 / 0.5 =20). At step 1504, The Optical Zoom Reference of the tele2 camera with respect to the ultra-wide camera is determined and may be defined as OZR_T2 (i.e. 10 / 0.5 =20). At step 1504, the instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to determine the Optical Zoom Reference of the tele2 camera with respect to the ultra-wide camera is determined and may be defined as OZR_T2 (i.e. 10 / 0.5 =20).
[0135] - At step 1506, the full size of the frame of the ultra-wide camera is determined which may be [ W_UW, H_UW ]. At step 1506, the instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to determine the full size of the frame of the ultra-wide camera , which may be defined as [ W_UW, H_UW ].
[0136] - At step 1508, test Frame “B” coordinates from the ultra-wide camera = [ X_TB , Y_TB, W_TB, H_TB]. At step 1508, the instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to identify coordinates of the test Frame “B” from the ultra-wide camera. Thus,
[0137] If (Match Not Found) {
[0138] Tele2 Frame w.r.t UW :
[0139] W_T2 = W_UW / OZR_T2
[0140] H_T2 = H_UW / OZR_T2
[0141] X_T2 = (W_UW - W_T2) / 2
[0142] Y_T2 = (H_UW - H_T2) / 2
[0143] Match With Tele2 = (X_T2 < X_TB) && (Y_T2 < Y_TB)
[0144] && ((X_T2+W_T2)>(X_TB+W_TB)) && ((Y_T2+H_T2)>(Y_TB+H_TB))
[0145] }
[0146] If (Match Not Found) {
[0147] / Repeat calculation for T1
[0148] }
[0149] If (Match Not Found){
[0150] / Repeat calculation for Wide
[0151] }
[0152] Else Select ultra-wide camera.
[0153] Particularly, at step 1510, a full FOV of the tele 2 camera is determined as ROI of the ultra-wide camera. At step 1510, the instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to determine a full FOV of the tele 2 camera as ROI of the ultra-wide camera. At step 1512, full co-ordinate of tele 2 is determined with respect to the ultra-wide camera and given as X_T2, Y_T2, W_T2, H_T2. At step 1512, the instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to determine the full co-ordinate of tele 2 camera with respect to the ultra-wide camera and to provide them as X_T2, Y_T2, W_T2, H_T2. At step 1514, after receiving input from the step 1512 and the step 1508, the determining module 1118 determines whether the coordinate of the test frame is within the tele 2 camera. At step 1514, after receiving input from the step 1512 and the step 1508, the instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to determine whether the coordinate of the test frame is within the tele 2 camera. The determined camera 1138 may be the tele 2 camera, if the coordinate of the test frame is within the tele 2 camera. If the coordinate of the test frame is within the tele 2 camera, the instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to identify the tele 2 camera as the determined camera 1138. Further, at step 1516, the optical zoom reference of the tele 1 camera with respect to the ultra-wide camera may be determined, if the coordinate of the test frame is not within the tele 2 camera. If the coordinate of the test frame is not within the tele 2 camera, the instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to determine the optical zoom reference of the tele 1 camera with respect to the ultra-wide camera. Further, at step 1518, the full FOV of the tele 1 may be determined as ROI of the ultra-wide camera. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to determine the full FOV of the tele 1 as the ROI of the ultra-wide camera. At step 1520, the full co-ordinate of tele 1 is determined with respect to the ultra-wide camera and given as X_T1, Y_T1, W_T1, H_T1. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to determine the full co-ordinate of tele 1 camera the full co-ordinate of tele 1 and to provide them as X_T1, Y_T1, W_T1, H_T1. At step 1522, after receiving input from the step 1520 and step 1508, the determining module 1118 determines that whether the coordinate of the test frame is within the tele 1 camera. After receiving input from the step 1520 and step 1508, the instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to determine that whether the coordinate of the test frame is within the tele 1 camera. The determined camera 1138 may be the tele 1 camera, if the coordinate of the test frame is within the tele 1 camera otherwise the operation continues till the time the camera 1138 may not be determined for the test frame. The determined camera 1138 may be the tele1 camera if the coordinates of the test frame are within the tele1 camera. Otherwise, the operation continues until the camera 1138 is determined for the test frame. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to determine the camera 1138 as the tele1 camera if the coordinates of the test frame are within the tele1 camera. Otherwise The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to continue the operation until the camera 1138 is determined for the test frame.
[0154] For test Frame "T", the determined camera may be referred to as TF_T_SS as provided in the table M shown in Figure 15E, without departing form the scope of the present disclosure.
[0155] Further, in an embodiment, after determining the camera 1138, the sensor deviation may be performed. In an embodiment, the comparing module 1120 may be configured to compare the determined camera 1138 having the FOV corresponding to the ROI of the at least one representative image and the at least one camera from the plurality of cameras 1008 of the device 1004 used for selecting the scene by the user input. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to compare the determined camera 1138 having the FOV corresponding to the ROI of the at least one representative image and the at least one camera from the plurality of cameras 1008 of the device 1004 used for selecting the scene by the user input.
[0156] In an embodiment, the determining module 1118 or the electronic device 1002 may be configured to determine the sensor deviation between the determined camera 1138 having the FOV corresponding to the ROI of the at least one representative image and the at least one camera from the plurality of cameras 1008 of the device 1004 user for selecting the scene by the user input, based on the comparison. Further, the sensor deviation may be explained by an example provided below:
[0157] In one example, referring to Figure 15E, the plurality of test frames 1212 having the highest aesthetic score may be B. Further, the determining module 1118 may be configured to receive the plurality of test frames 1212 having the highest aesthetic score, static capabilities of the plurality of cameras 1008, and configuration of the at least one camera from the plurality of cameras 1008 configured by the user input to select the scene. The determining module 1118 may be configured to obtain the plurality of test frames 1212 with the highest aesthetic score, the static capabilities of the plurality of cameras 1008, and the configuration of at least one camera from the plurality of cameras 1008, which is configured by user input to select the scene. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to obtain the plurality of test frames 1212 with the highest aesthetic score, the static capabilities of the plurality of cameras 1008, and the configuration of at least one camera from the plurality of cameras 1008, which is configured by user input to select the scene. Further, the determining module 1118 may determine and generate output as the determined camera 1138 is compatible with the B test frames. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to determine and generate output as the determined camera 1138 is compatible with the B test frames. The determined camera 1138 for the Test frame B may be referred as the TF_B_SS. Subsequently, the sensor deviation may be performed between the determined camera 1138 having the FOV corresponding to the ROI of the at least one representative image (test frame) and the at least one camera from the plurality of cameras 1008 of the device 1004 user for selecting the scene by the user input. The sensor deviation for the Test frame B may be referred to as TF_T_SD and provided in Table M as shown in Figure 15E.
[0158] Thus, the sensor deviation may be provided by / determined by a table as provided below in Table 2:
[0159] Sensor DeviationFactorDetermined Camera 1138ultra-wideWideTele1Tele2UserSelectionultra-wide0123Wide1012Tele12101Tele23210
[0160] Figures 16A and 16B illustrate one or more crop parameters 1140 of the determined camera 1138, in accordance with an embodiment of the present disclosure.
[0161] In an embodiment, the determining module 1118 may be configured to determine one or more crop parameters 1140 for the determined camera 1138 for aligning a frame of the determined camera 1138 to the at least one representative image. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to determine one or more crop parameters 1140 for the determined camera 1138 for aligning a frame of the determined camera 1138 to the at least one representative image.
[0162] In such embodiment, the determined module 1118 may be configured to determine the one or more crop parameters 1140 based on a difference between the camera having the maximum FOV and the FOV of the determined camera 1138, for the identified at least one test frame, corresponding to the ROI of the at least one representative image. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to determine the one or more crop parameters 1140 based on a difference between the camera having the maximum FOV and the FOV of the determined camera 1138, for the identified at least one test frame, corresponding to the ROI of the at least one representative image. The determining module 1118 may be configured to determine the one or more crop parameters 1140 on the identified at least one test frame to align an FOV of an image from the determined camera 1138 with an FOV of the identified at least one test frame by using a plurality of parameters. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to determine the one or more crop parameters 1140 on the identified at least one test frame to align an FOV of an image from the determined camera 1138 with an FOV of the identified at least one test frame by using a plurality of parameters. The plurality of parameters may include optical zoom reference, sensor output size of the determined camera 1138, and the camera having the maximum FOV, without departing from the scope of the present disclosure.
[0163] In such embodiment, prior to aligning the FOV of the image from the determined camera 1138 with the FOV of the identified at least one test frame, an FOV of at least one of the plurality of test frames 1212 may be aligned with the FOV of the identified at least one test frame, without departing from the scope of the present disclosure. The determination of the one or more crop parameters 1140 may be important as mostly the FOV of the determined camera 1138 does not match with the at least one identified frame from the plurality of test frames 1212. So, the one or more crop parameters 1140 ensure matching of the FOV of the image from the determined camera 1138 with the FOV of the identified at least one test frame.
[0164] In an embodiment, the one or more crop parameters 1140 may be determined by an algorithm provided below and with reference to Figure 16B. Further, the data given below may be static capabilities of the plurality of camera 1008.
[0165] At step 1604, the Optical Zoom Reference of the determined camera 1138 i.e. TF_T_SS may be OZR.
[0166] At step 1606, full size of the frame of the camera having maximum FOV (i.e. ultra-wide camera) may be [ W_UW, H_UW]. Thus, after receiving the full size frame of the camera and the optical zoom reference of the determined camera 1138, the full FOV co-ordinates of the determined camera 1138 as ROI of the ultra-wide camera may be determined, at step 1612. At step 1612, after receiving the full size frame of the camera and the optical zoom reference of the determined camera 1138, the instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to determine the full FOV co-ordinates of the determined camera 1138 as ROI of the ultra-wide camera.
[0167] At step 1608, the full size of the frame of determined camera 1138 may be [ W_SEL, H_SEL]. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to determine the full size of the frame of determined camera 1138 as [ W_SEL, H_SEL].
[0168] Further, the Test Frame “T” Generated Co-ordinates from the ultra-wide camera which may be [ X_T, Y_T, W_T, H_T], and at step 1610, the Test Frame “B” co-ordinates w.r.t the ultra-wide camera may be X_TB , Y_TB, W_TB, H_TB. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to determine coordinates of the the Test Frame “T”.
[0169] Further, output from the step 1612, step 1608, and the step 1610 may be provided and thus, a crop may be selected after correction from Base frame (TF_T_CS):
[0170] - X_T_CS = (X_T - W_UW / (OZR*2) ) *OZR *( W_SEL / W_UW)
[0171] - Y_T_CS = (Y_T - H_UW / (OZR*2) ) *OZR *( H_SEL / H_UW)
[0172] - W_T_CS = W_T * OZR *( W_SEL / W_UW)
[0173] - H_T_CS = H_T * OZR *( H_SEL / H_UW)
[0174] - TF_T_CS =[[X_T_CS , Y_T_CS, W_T_CS, H_T_CS], at step 1614
[0175] Further, the determination of the one or more crop parameters 1140 may be explained by an example provided below:
[0176] In one example, the electronic device 1002 may receive the plurality of test frames 1212 having the highest score as B, static capabilities of the plurality of cameras 1008, configuration of the at least one camera configured by the user input to select the scene, determined camera 1138. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to obtain the plurality of test frames 1212 having the highest score as B, static capabilities of the plurality of cameras 1008, configuration of the at least one camera configured by the user input to select the scene, determined camera 1138. Further, the FOV of the at least one of the plurality of test frames 1212 may be an ultra-wide frame, and the FOV of the identified at least one test frame may be a tele frame. Further, the one or more crop parameters 1140 may be determined to align the FOV of the at least one of the plurality of test frames 1212 and the FOV of the identified at least one test frame. Subsequently, the one or more crop parameters 1140 may be determined on the identified at least one test frame to align the FOV of the image from the determined camera 1138, for example, the wide camera, with the FOV of the identified at least one test frame, tele frame, by using the plurality of parameters.
[0177] Figures 17A-17C illustrate a sensor mode of the determined camera 1138, in accordance with an embodiment of the present disclosure.
[0178] In an embodiment, after determining the one or more crop parameters 1140, the determining module 1118 may be configured to determine the sensor mode from the plurality of sensor modes 1142 of the determined camera 1138, for the identified at least one test frame. After determining the one or more crop parameters 1140, the instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to determine the sensor mode from the plurality of sensor modes 1142 of the determined camera 1138, for the identified at least one test frame. The plurality of sensor modes 1142 may include a remosaic mode and a binning mode, without departing from the scope of the present disclosure. In an embodiment, based on the remosaic capability of the determined camera 1138 and the one or more crop parameters 1140, the sensor mode as selected may be at least one of the remosaic mode and binning mode, without departing from the scope of the present disclosure.
[0179] In an embodiment, if the selected sensor mode may be the remosaic sensor and a crop-scale for the image may be too high, then a full-resolution remosaic mode may be selected. Further, the FOV of the optimum frame / best frame may be cropped from a larger frame and may be downscaled back to the required resolution of the best frame. Additionally, if the selected sensor mode may be different from the remosaic mode, then no further operation may be performed by the processor 1104. This configuration ensures that the quality of the image may not be compromised / degraded due to excessive cropping of the image.
[0180] In one example, referring to Figure 17B(i), the at least one identified test frame may be the tele frame, and thus the one or more crop parameters 1140 may be applied on the plurality of test frames 1212 to attain the at least one identified frame. The capacity of the determined camera 1138 may be 12MP. The application of the one or more crop parameters 1140 reduces size of the image from 12MP to 7MP. Then, further upscaling may be performed to reach the 12MP of the image as selected by the user input while selecting the scene. Thus, the absence of a determination of the sensor mode and upscaling degrades the quality of the final image as compared to the scene selected by the user input.
[0181] On the other hand, referring to Figure 17B(ii), when the determined camera 1138 has the remosaic mode (48MP Remosaic / 12MP Binning), the processor 1104 determines the full capacity of the frame of the determined camera 1138, i.e., 48MP. Thus, the one or more crop parameters 1140 may be applied on the plurality of test frames 1212 to attain the at least one identified frame, and that reduces the size of the image from 48MP to 28MP. Then, further downscaling may be performed to reach the 12MP of the image as selected by the user input while selecting the scene. Therefore, the quality of the final image may be improved or provided of the same quality as compared to the scene selected by the user input.
[0182] In an embodiment, once the sensor mode is determined, the updating module 1124 may update the determined one or more crop parameters 1140 based on the determined sensor mode from the plurality of sensor modes 1142 of the determined camera 1138. Further, the image may be captured by applying the updated one or more crop parameters 1140. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to update the determined one or more crop parameters 1140 based on the determined sensor mode from the plurality of sensor modes 1142 of the determined camera 1138. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to apply the updated one or more crop parameters 1140 to the captured image.
[0183] In such embodiment, the one or more crop parameters 1140 may be updated by an algorithm as provided below and also, the data as provided below may be static capabilities of the device 1004.
[0184] At step 1704, an Optical Zoom Reference of the determined camera 1138 with respect to the ultra-wide camera may be TF_B_SS may be OZR
[0185] At step 1706, the full size of the frame of the camera having maximum FOV i.e. ultra-wide camera may be [ W_UW , H_UW ]
[0186] At step 1708, the full frame size of the determined camera 1138 may be [ W_SEL , H_SEL ]. Further, the Remosaic Capability of the determined camera 1138 may be R_SEL, and the Binning Factor of the determined camera 1138 may be BF_SEL
[0187] At step 1710, coordinates of the Test Frame “B” generated from the ultra-wide camera may be [ X_TB , Y_TB, W_TB, H_TB ]
[0188] At step 1712, full FOV coordinates of the determined camera 1138 may be determined as ROI of the ultra-wide camera. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to identify the full FOV coordinates of the determined camera 1138 as ROI of the ultra-wide camera.
[0189] At step 1714, coordinates of the Test Frame “B” with respect to the determined camera 1138 may be [ X_TB_CS, Y_TB_CS, W_TB_CS, H_TB_CS]. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to identify coordinates of the Test Frame “B” with respect to the determined camera 1138.
[0190] At step 1716, when the determining module 1118 may determine whether the remosaic sensor mode may be available in the determined camera 1138 as received from a block 1718. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to determine whether the remosaic sensor mode may be available in the determined camera 1138 as received from a block 1718. If the remosaic sensor mode may not be available, then no further operation is performed and accordingly, at step 1722, the table M may be updated. If the remosaic sensor mode is available, then further operation may be performed which is explained below:
[0191] Further, the selected crop after correction from Base frame (TF_B_CS):
[0192] If (R_SEL == FALSE)
[0193] {
[0194] X_T_CS = (X_T - W_UW / (OZR*2) ) *OZR *( W_SEL / W_UW)
[0195] Y_T_CS = (Y_T - H_UW / (OZR*2) ) *OZR *( H_SEL / H_UW)
[0196] W_T_CS = W_T * OZR *( W_SEL / W_UW)
[0197] H_T_CS = H_B * OZR *( H_SEL / H_UW)
[0198] }
[0199] Else
[0200] {
[0201] X_T_CS = ((X_T - W_UW / (OZR*2) ) *OZR *( W_SEL / W_UW) ) * BF_SEL
[0202] Y_T_CS = ((Y_T - H_UW / (OZR*2) ) *OZR *( H_SEL / H_UW) ) * BF_SEL
[0203] W_T_CS = (W_T * OZR *( W_SEL / W_UW) ) * BF_SEL
[0204] H_T_CS = (H_T * OZR *( H_SEL / H_UW) ) * BF_SEL
[0205] }
[0206] TF_T_CS =[[X_T_CS , Y_T_CS, W_T_CS, H_T_CS]
[0207] Thus, at step 1720, the test frame B coordinates with respect to the determined camera 1138 and scaled to binning factor of the determined camera 1138 may be given as TF_T_CS = [[X_T_CS , Y_T_CS, W_T_CS, H_T_CS]. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to identify the test frame B coordinates with respect to the determined camera 1138 and scaled to binning factor of the determined camera 1138.
[0208] Additionally, Further, once the operation is performed then, at step 1724, the table 3 (related to Table M as shown in Figure 14C) as provided below may be updated accordingly:
[0209] Frame IndexAesthetic ScoreDetermined CameraSensor DeviationSensor Mode SelectedCrop Parameter(s)Test Frame 1TF_1_ASTF_1_SSTF_1_SDTF_1_SMSTF_1_CSTest Frame 2TF_2_ASTF_2_SSTF_2_SDTF_2_SMSTF_2_CSTest Frame 3TF_3_ASTF_3_SSTF_3_SDTF_3_SMSTF_3_CSTest Frame 4TF_4_ASTF_4_SSTF_4_SDTF_4_SMSTF_4_CS....................................Test Frame BTF_B_ASTF_B_SSTF_B_SDTF_B_SMSTF_B_CS
[0210] Thus, when the remosaic capability may be supported by the determined camera 1138 / remosaic sensor mode may be present in the determined camera 1138, then the TF_T_SMS may be 1. Further, when the remosaic capability may not be supported by the determined camera 1138 / remosaic sensor mode may not be present in the determined camera 1138, then the TF_T_SMS may be 0.
[0211] In an embodiment, after determining the sensor mode, the crop deviation may be determined. In an embodiment, the determining module 1118 may be configured to determine the crop deviation based on a difference between the determined camera 1138, for the identified at least one test frame, corresponding to the ROI of the at least one representative image and an FOV of the at least one camera from the plurality of cameras 1008 of the device 1004 used for selecting the scene by the user input. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to determine the crop deviation based on a difference between the determined camera 1138, for the identified at least one test frame, corresponding to the ROI of the at least one representative image and an FOV of the at least one camera from the plurality of cameras 1008 of the device 1004 used for selecting the scene by the user input. The crop deviation factor may be determined by following operations as provided below:
[0212] The data as provided below may be from the electronic device 1002. Further, the operation is as follows, where the data, for example, the full size of the frame of the camera, the optical zoom reference with respect to (w.r.t.) the ultra-wide camera, the user selected zoom ratio, etc., may be form the electronic device 1002:
[0213] The full size of the frame of the camera having the maximum FOV of Reference (i.e. ltra-wide camera) may be [ W_UW, H_UW ].
[0214] The optical zoom reference w.r.t the ultra-wide camera may be U_OZR.
[0215] The user selected zoom ratio may be U_ZOOM.
[0216] Further, the max zoom supported by the electronic device 1002 may be MAX_ZOOM
[0217] Test Frame “T” generated Co-ordinates from the ultra-wide camera = [ X_TB, Y_TB, W_TB, H_TB ]
[0218] ROI Size ratio of the Test Frame “T” ROI_B may be (((W_UW * H_UW) / (W_TB* H_TB) )) / 2
[0219] ROI size ratio of the user selected frame ROI_USR may be U_ZOOM * U_OZR
[0220] If (ROI_B > ROI_USR)
[0221] Crop Deviation Factor TF_T_CD = (ROI_B- ROI_USR) *(MAX_ZOOM)
[0222] Else
[0223] Crop Deviation Factor TF_T_CD = (ROI_USR-ROI_B) *(MAX_ZOOM)
[0224] Further, the crop deviation may be explained with an example. In one example, the identified at least one frame may be B test frames. Further, the crop deviation of the B test frames may be determined by the following operations:
[0225] Operation 1: ROI Size ratio of 1 to B of "B" test frames compared to the ROI size of the reference frame, i.e., the ultra-wide camera.
[0226] Operation 2: ROI size ratio of the frame selected by the user input for the scene compared to ROI size of the ultra-wide camera.
[0227] Further, the difference may be determined between the operation 1 and operation 2.
[0228] The difference between the operation 1 and the operation 2 may be determined as a ratio of MAX zoom supported by the electronic device 1002. Further, for test Frame “T”, TF_T_CS depicts the co-ordinates of the crop parameters and TF_T_CD denotes the crop deviation factor as mentioned in the table 4 (related to Table M as shown in Figure 14C) M provided below:
[0229] Frame IndexAesthetic ScoreDetermined CameraSensor DeviationCrop Parameter(s)Crop DeviationTest Frame 1TF_1_ASTF_1_SSTF_1_SDTF_1_CSTF_1_CDTest Frame 2TF_2_ASTF_2_SSTF_2_SDTF_2_CSTF_2_CDTest Frame 3TF_3_ASTF_3_SSTF_3_SDTF_3_CSTF_3_CDTest Frame 4TF_4_ASTF_4_SSTF_4_SDTF_4_CSTF_4_CD....................................Test Frame BTF_B_ASTF_B_SSTF_B_SDTF_B_CSTF_B_CD
[0230] Figures 18A-18E illustrate the hardware override selection unit 1144 for determining a selection factor, in accordance with an embodiment of the present disclosure.
[0231] At step 1802, the electronic device 1002, specially, the processor 1104, may be configured to receive the instruction from the device 1004 that the scene may be selected by the user input. The processor 1104, may be configured to obtain the instruction from the device 1004 that the scene may be selected by the user input. Further, at step 1804, the processor 1104 may determine whether the camera (determined camera 1138) for the best frame used for capturing the image. At step 1806, the processor 1104 may determine whether the user zoom region of the scene may be the same as determined by the processor 1104, if the camera for the best frame may be used to capture the image. At step 1808, the processor 1104 may be configured to issue a command to the configured ISP to require crop corresponding to the camera and sensor mode selected, if the user zoom region is not the same as determined by the processor 1104. Also, at step 1816, no further operation is required when the user zoom region may be the same as determined by the processor 1104.
[0232] At step 1812, the processor 1104 may determine whether the remosaic mode may be selected, if the camera is not for the best frame. Further, at step 1810, the processor 1104 may issue a command to start the determined camera 1138 to take an intelligent shot (best frame), if the remosaic mode is not selected. At step 1814, the processor 1104 may issue a command to start the determined camera 1138 in the remosaic mode to take the intelligent shot (best frame), if the remosaic mode is selected.
[0233] Further, all the instructions may be received or obtained by the device 1004 and accordingly capture the best frame / the intelligent shot at step 1818 / the user shot at step 1820. Further, the operation of the hardware override selection unit 1144 may be explained in detail in the subsequent paragraphs:
[0234] In an embodiment, after determining the sensor mode and updating the one or more crop parameters, the determining module 1118 may be configured to determine a difference between the aesthetic score of the identified at least one test frame and a total deviation. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to determine a difference between the aesthetic score of the identified at least one test frame and a total deviation. In an embodiment, the total deviation may be determined based on the sensor deviation and the crop deviation. In an embodiment, the determining module 1118 may be configured to determine the selection factor based on the difference. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to determine the selection factor based on the difference.
[0235] In an embodiment, the total deviation may be given as:
[0236] - Total Deviation TF_T_TD = TF_T_SD + TF_T_CD, where TF_B_SD may be between a scale of 0 to the total number of cameras in the electronic device 1002 of the device 1004.
[0237] - For example, if the electronic device 1002 may include 4 cameras (ultra-wide camera, wide camera, tele1 camera, tele2 camera)
[0238] - minimum sensor deviation may be 0 (the at least one camera and the determined camera 1138 may be same)
[0239] - maximum sensor deviation may be 3. (the at least one camera is ultra-wide and the determined camera 1138 may be tele2 or vice-versa).
[0240] In an embodiment, the selection factor may be depicted as:
[0241] - TF_T_SF=TF_T_AS-TF_T_TD
[0242] - Where TF_B_AS may be between scale of 4 to 10 (Since Test Frames with aesthetic scores less than 4 are already discarded by the aesthetic score generator 1402) and the aesthetic score generator 1402 may be trained to provide scores between 0-10. Thus, TF_B_SF may be on a scale of 0-10.
[0243] The test frame with the highest selection factor is referred to as the “Best Frame”. The selection factor may be important as the selection factor normalize the aesthetic score of the identified at least one test frame and thus avoid deviating too much from the scene selected by the user input. The selection factor may be determined to capture the user shot / best frame / or both. The selection factor may be provided in the table 5 (related to Table M as shown in Figure 14C) as provided below:
[0244] Frame IndexAesthetic ScoreIdentified CameraSensor DeviationSensorModeSelectedCrop SelectedCrop DeviationTotalDeviationSelection FactorTest Frame 1TF_1_ASTF_1_SSTF_1_SDTF_1_SMSTF_1_CSTF_1_CDTF_1_TDTF_1_SFTest Frame 2TF_2_ASTF_2_SSTF_2_SDTF_2_SMSTF_2_CSTF_2_CDTF_2_TDTF_2_SFTest Frame 3TF_3_ASTF_3_SSTF_3_SDTF_3_SMSTF_3_CSTF_3_CDTF_3_TDTF_3_SFTest Frame 4TF_4_ASTF_4_SSTF_4_SDTF_4_SMSTF_4_CSTF_4_CDTF_4_TDTF_4_SF......................................................Test Frame BTF_B_ASTF_B_SSTF_B_SDTF_B_SMSTF_B_CSTF_B_CDTF_B_TDTF_B_SF
[0245] In an embodiment, TF_B_AS may be the aesthetic score of the test Frame “B” Aesthetic Score. TF_B_SS may be the determined camera 1138 of the test Frame “B”. TF_B_SD may be the sensor deviation of the test frame “B”. TF_B_SMS may be selected sensor mode (Sensor mode - Remosaic on / off w.r.t determined camera 1138 i.e. TF_B_SS, to be used for frame capture) for the Test Frame “B”. TF_B_CS may be the crop selected (the crop co-ordinates w.r.t the determined camera 1138 i.e. TF_B_SS, to be used for frame capture) of the test Frame “B”. TF_B_CD may be the crop deviation value of the test frame “B”. TF_B_TD” i.e. TF_B_SD + TF_B_CD, may be the total deviation of the test Frame “B. TF_B_SF, i.e., TF_B_SF = TF_B_AS - TF_B_TD, may be the selection factor of the test Frame “B”.
[0246] Further, the selection factor may be explained with below examples:
[0247] In one example, the user may be intended to capture the scene 1822 shown in Figure 18B(i). However, the identified at least one frame as selected deviates too much from the user selected scene, depending on the determined camera 1138 and the one or more crop parameters 1140 to provide the frame with the highest aesthetic score as shown in Figure 18B(ii). Thus, the selection factor normalizes the aesthetic score of the identified at least one frame and generates the optimum frame / best frame for capturing the selected scene by the user input.
[0248] In another example, Test Frame “T1” may have an aesthetic score of 9.77, the sensor deviation of 2, and the crop deviation of 6. In this example, to achieve a good aesthetic score the deviation happens at a large extent as compared to the selected scene by the user input i.e. Low selection factor = (9.77-2-6) =1.77
[0249] In yet another example, Test Frame “T2” may have the aesthetic score of 7.50, the sensor deviation of 1, the crop deviation of 0.5. In this example, the deviation may be less as compared to the selected scene by the user input i.e. moderate selection factor = (7.50-1-0.5) =6.00.
[0250] Further, the determining module 1118 may be configured to determine the best frame based on the selection factor. The selection factor may include a plurality of modes. The plurality of modes may include an automatic mode and a manual mode. This configuration provides the best frame to the user.
[0251] In an embodiment, the selection factor may include the automatic mode. In such embodiment, the comparing module 1120 may be configured to compare the selection factor with a plurality of threshold selection factors. The plurality of threshold selection factors may include a low threshold factor, a mid-threshold selection factor, and a high threshold factor. The determining module 1118 may be configured to determine the best frame when the selection factor may be at least equal to or greater than the mid-threshold selection factor and lesser than the high-threshold selection factor. The determining module 1118 may be configured to provide the best frame to the user along with the initial frame of the scene selected by the user input to capture the image. Further, the determining module 1118 may be configured to determine the best frame when the selection factor may be at least equal to or greater than the high threshold selection factor and provide only the best frame to the user.
[0252] In an embodiment, the best frame in the automatic mode may be determined by the following operations:
[0253] Operation 1: to determine top ranked test frame from the B test frames with the best selection factor from the table 6 (related to Table M as shown in Figure 14C) as provided below:
[0254] Frame IndexAesthetic ScoreIdentified CameraSensor DeviationSensorModeSelectedCrop SelectedCrop DeviationTotalDeviationSelection FactorTest Frame 1TF_1_ASTF_1_SSTF_1_SDTF_1_SMSTF_1_CSTF_1_CDTF_1_TDTF_1_SFTest Frame 2TF_2_ASTF_2_SSTF_2_SDTF_2_SMSTF_2_CSTF_2_CDTF_2_TDTF_2_SFBest FrameTF_3_ASTF_3_SSTF_3_SDTF_3_SMSTF_3_CSTF_3_CDTF_3_TDTF_3_SFTest Frame 4TF_4_ASTF_4_SSTF_4_SDTF_4_SMSTF_4_CSTF_4_CDTF_4_TDTF_4_SF......................................................Test Frame BTF_B_ASTF_B_SSTF_B_SDTF_B_SMSTF_B_CSTF_B_CDTF_B_TDTF_B_SF
[0255] Operation 2: determine final image capturing conditions where threshold may be defined as SF_MIN = 0, SF_LOW =4, SF_MID =7, SF_MAX =10, shown in Figure 18C. Further, the conditions are as follows:
[0256] - Condition 1: when the selection factor of the best frame may be below SF_LOW, the best frame may be discarded as shown by 1826 in favor of the initial frame of the scene selected by the user input.
[0257] - Condition 2: when the selection factor of the best frame may be more than the SF_LOW and less than the SF_MID, the best frame may be captured along with the initial frame as shown by 1828 of the scene selected by the user input.
[0258] - Condition 3: when the selection factor of the best frame may be more than SF_MID, the initial frame of the scene selected by the user input may be discarded in favor of the Best Frame as shown by 1830.
[0259] In an embodiment, the SF_LOW and SF_MID values may be tuned / varied as per a tolerance required from a final output from the electronic device 1002. In one example, if the SF_LOW maybe 4 & SF_MID maybe 8, the electronic device 1002 may be more stringent in determining the final image capture conditions (only a frame with a very high aesthetic score may be selected for the third condition as described above). Further, the automatic mode may be explained with a plurality of examples:
[0260] In one example, referring to Figure 18D, the plurality of cameras 1008 may include the ultra-wide camera-0.5X, wide camera-1.0X, tele camera-3.0X, and max zoom-30X. Further, referring to Figure 18D(i(a)), the user selects the scene 1822 with the ultra-wide camera with a zoom ratio of the ultra-wide camera as 1.0X. The frame / camera 1832 selected by the electronic device 1002 may be the tele camera with a zoom ratio of the ultra-wide as 60.0X as shown in Figure 18D(i(b)). The best frame may be selected where the total deviation maybe 3.98, an aesthetic score may be 7.2, the selection factor maybe 3.02, the SF_LOW may be 4, and the SF_MID may be 7. Thus, the resultant best frame may be the initial frame of the scene 1822 selected by the user input as shown in Figure 18D(i(c)).
[0261] In another example, referring to Figure 18D(ii(a)), the user selects the scene 1822 with the tele camera with a zoom ratio of the ultra-wide camera as 6.0X. The frame / camera 1832 selected by the electronic device 1002 may be the ultra-wide camera with a zoom ratio of the ultra-wide as 1.4 X as shown in Figure 18D(ii(b)). The best frame may be selected where the total deviation maybe 2.15, an aesthetic score maybe 8.7, the selection factor may be 6.55, the SF_LOW maybe 4, and the SF_MID maybe 7. Thus, the resultant best frame may be the duplicate best frame as shown in Figure 18D(ii(c))-18D(ii(d)).
[0262] In yet another example, referring to Figure 18D(iii(a)), the user selects the scene 1822 with the tele camera with a zoom ratio of the ultra-wide camera as 14.0X. The frame / camera 1832 selected by the electronic device 1002 may be the tele camera with a zoom ratio of the ultra-wide as 10.0X as shown in Figure 18D(iii(b)). The best frame may be selected where the total deviation may be 0.13, aesthetic score may be 8.7, the selection factor may be 7.67, the SF_LOW may be 4, and the SF_MID may be 7. Thus, the resultant best frame may be the best frame 1832 as shown in Figure 18D(iii(c)).
[0263] In an embodiment, when the selection factor may include the manual mode. In such embodiment, the determining module 1118 may be configured to determine a first best frame, a second best frame, and a third best frame from the identified at least one test frame based on the selection factor. The determining module 1118 may be configured to determine the best frame from at least one of the first best frame, the second best frame, and the third best frame by the user input. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to determine the best frame from at least one of the first best frame, the second best frame, and the third best frame by the user input. This configuration aligns the best frame with the scene selected by the user input using the at least one camera from the plurality of camera 1008 of the device 1004, without departing from the scope of the present disclosure.
[0264] In an embodiment, the best frame in the manual mode may be determined by the following operations:
[0265] Operation 1: to determine top ranked test frame from the B test frames with the best selection factor from the table M as provided the Figure 18E.
[0266] Operation 2: to provide an option to the user to select the best frame among the frames with best selection factor. For example, the first best frame 1836 with the selection factor TF_1_SF, the third best frame 1840 with selection factor TF_3_SF, and the second best frame 1838 with the selection factor TF_B_SF may be assumed to have the best scores.
[0267] Thus, the hardware parameters may be updated in the device 1004 and the above-mentioned frames along with other factors as provided in the table M may be given as the option to the user to select any of the frames for final capture. Further, the option may change as per the requirement in the present subject matter.
[0268] Figure 19 illustrates determining the best frame and capturing the image, in accordance with an embodiment of the present disclosure.
[0269] In an embodiment, the determining module 1118 may be configured to determine the best frame based on the aesthetic score, the sensor deviation, and the crop deviation of the at least one representative image, as discussed above, without departing from the scope of the present disclosure. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to determine the best frame based on the aesthetic score, the sensor deviation, and the crop deviation of the at least one representative image.
[0270] In an embodiment, the capturing module 1126 may be configured to capture the image corresponding to the determined best frame, by applying the determined one or more crop parameters 1140 to the frame of the determined camera. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to capture the image corresponding to the determined best frame, by applying the determined one or more crop parameters 1140 to the frame of the determined camera. In such embodiment, the capturing module 1126 may be configured to capture the image of the scene with the determined camera 1138 corresponding to the best frame. The instructions, when executed by the processor 1104 individually or collectively, cause the electronic device 1002 to capture the image of the scene with the determined camera 1138 corresponding to the best frame. The best frame may be determined by the selection factor, by selecting the sensor mode, and then applying the determined one or more crop parameters 1140 for the corresponding representative image. The generation of the best frame may be explained with an example as provided below:
[0271] In one example, the hardware override selection unit 1144 may submit a request to the electronic device 1002 to generate the best frame with the determined camera 1138, the selected sensor mode, and the zoom value selected corresponding to the best frame. Further, the processor 1104 may be configured to generate the best frame through the following steps as provided below:
[0272] - determine the frame from the determined camera 1138 and selected sensor mode based on input received from the hardware override selection unit 1144,
[0273] - crop the frame from the determined camera 1138 with ISP Frame based on the input delivered from the hardware override selection unit 1144, and
[0274] - upscale / downscale the cropped image based on the final capture size configured by the user input in the device 1004, without departing from the scope of the present disclosure.
[0275] Therefore, the above-mentioned operations performed from Figures 12A to 19 may be summarised by one example as explained with reference to Figures 20A to 20D, without departing from the scope of the present disclosure.
[0276] In one example, the device 1004 and / or the electronic device 1002 may have the following configuration: 12 MP ultra-wide camera having 05.X optical zoom, 12 MP wide camera having 1.0X optical zoom, remosaic capability may be supported (48 MP Remosaic Mode), 12 MP tele camera having 3X optical zoom, maximum digital zoom supported by the device 1004 may be 30.0X.
[0277] Further, the user configured capture may be assumed to be of the following configuration: the camera selected may be the tele camera, zoom ratio selected may be 3.0X.
[0278] The aesthetic score may be calculated / generated for the test frame T as TF_T_AS=7.87. Further, the determined camera 1138 with the sensor deviation is performed and is shown in the table 7 below:
[0279] Frame IndexAesthetic ScoreDetermined cameraSensor DeviationTest Frame TTF_T_AS =7.87TF_T_SS= WideTF_T_SD =1
[0280] Thus, TF_T_SS may be the determined camera 1138 selected for the test Frame T which may be the wide camera, since the field of view of the wide camera covers the Test Frame T region completely. TF_T_SD sensor deviation where the determined camera 1138 may be the wide camera and the at least one camera may be the tele camera, therefore, the sensor deviation is calculated as 1 based on the below table 8:
[0281] Sensor DeviationFactorSystem SelectionUltra-wideWideTele1UserSelectionUltra-wide012Wide101Tele1210
[0282] Further, the sensor mode may be determined and provided in the below table 9:
[0283] Frame IndexAesthetic ScoreDetermined CameraSensor DeviationSensor Mode SelectedTest Frame TTF_T_AS =7.87TF_T_SS= WideTF_T_SD =1TF_1_SMS =1 (Remosaic)
[0284] The TF_1_SMS may be determined as 1 since the remosaic capability may be supported by the wide sensor.
[0285] Further, from Figure 20B, the one or more crop parameters 1140 and the crop deviation may be determined:
[0286] Further, the crop deviation may be determined and provided in the table 10 below:
[0287] Frame IndexAesthetic ScoreCamera DeterminedSensor DeviationSensor Mode SelectedCrop Parameter(s)Crop DeviationTest Frame TTF_T_AS =7.87TF_T_SS = WideTF_T_SD =1TF_1_SMS =1 (Remosaic)TF_1_CD =0.09
[0288] Further, the field of view and frame size of each camera along with the crop deviation in the electronic device 1002 may be provided in the below table 11:
[0289] Frame IndexAesthetic ScoreCamera DeterminedSensor DeviationSensor Mode SelectedCrop Parameter(s)Crop DeviationTest Frame TTF_T_AS =7.87TF_T_SS = WideTF_T_SD =1TF_1_SMS =1 (Remosaic)TF_T_CS = [700, 500, 6400,4800]TF_1_CD =0.09
[0290] Thus, the test frame T region with respect to the ultra-wide frame may be output 1, i.e., Left top X=1175, left top Y=875, width=1600, height=1200. Test Frame T Region with respect to the determined camera 1138: TF_T_SS may be wide. Sensor mode selected TF_1_SMS may be 1(Remosaic). TF_T_CS may be output 2, i.e., Left top X = 700, Left top Y=500, width=6400, height=4800.
[0291] Further, after the crop deviation and receiving input from the hardware override selection unit 1144, the electronic device 1002 may generate the best frame by following operations:
[0292] - Total Deviation TF_T_TD = TF_T_SD (sensor deviation) + TF_T_CD (crop deviation), where TF_T_TD =1+0.09 =1.09
[0293] - Selection factor TF_T_SF = TF_T_AS (aesthetic score) - TF_T_TD (total deviation) where TF_T_SF=7.87-1.09 =6.78
[0294] Thus, based on the selection factor, the best frame may be determined:
[0295] - By the automatic mode
[0296] - By the manual mode.
[0297] For capturing the image / best frame, the determined camera (TF_T_SS), sensor mode selected (TF_1_SMS) & the one or more crop parameters 1140 (TF_T_CS) may be updated in the device 1004 and / or the electronic device 1002.
[0298] In yet another embodiment, a electronic device 1002 may be configured to capture an image. In such embodiment, the electronic device 1002 may include a memory 1108 and at least one processor 1104. The at least one processor 1104 (referred to as a processor 1104) may be configured to be communicatively coupled with the memory 1108. The configuration of the processor 1104 may be the same as explained with reference to Figure 11. Thus, the same is not explained here for sake of the brevity.
[0299] Further, the processor 1104 may be configured to generate the plurality of representative images having the plurality of ROIs. The processor 1104 may be configured to determine the aesthetic score for each representative image of the plurality of representative images. The aesthetic score of each of the representative image may be greater than the pre-defined threshold aesthetic value and the aesthetic values of the scene generated by the user input by the at least one camera from the plurality of cameras 1008 of the device 1004 and / or the electronic device 1002. The processor 1104 may be configured to determine, for each of the plurality of representative images, the camera 1138 having the field of view corresponding to an ROI of the respective representative image and covers an FOV of the respective representative image. The processor 1104 may be configured to determine the sensor deviation for each of the plurality of representative images, by determining a deviation between the determined camera 1138 having the field of view corresponding to the ROI of the respective representative image and the at least one camera from the plurality of cameras 1008 used for selecting the scene by the user input. The processor 1104 may be configured to determine, for each of the plurality of representative images, the one or more crop parameters 1140 for the frame of the determined camera 1138 to align the frame of the determined camera to the respective representative image. The processor 1104 may be configured to determine the crop deviation for each of the plurality of representative images, by determining a deviation between the size of the ROI of the respective representative image and the size of the ROI of the selected scene by the user input. The processor 1104 may be configured to recommend the best frame from the plurality of representative images to the user based on the aesthetic score, the sensor deviation, the crop deviation. The processor 1104 captures the recommended best frame (image) by applying the determined one or more crop parameters to the frame of the determined camera 1138.
[0300] Figures 21A-21B illustrate an alternate embodiment of the electronic device 1002, in accordance with yet another embodiment of the present disclosure.
[0301] Referring to Figure 21A, when the user selects a multiple regions of interest, then accordingly the plurality of test frames 1212 with respect to the multiple regions of interest may be generated. Further, the best frame may be selected among the ROIs provided by the user input.
[0302] Referring to Figure 21B, when the user selects a region of interest, then accordingly the plurality of test frames 1212 with respect to the region of interest may be generated. Further, the best frame may be selected based on the frame selected by the user input.
[0303] Therefore, based on the operation mode of the overall electronic device 1002 which may be:
[0304] - Auto selection of frame as explained with reference to Figures 11 to 19
[0305] - Manual selection of multiple frames as explained with Figure 21A.
[0306] - Manual selection of single frame as explained in Figure 21B.
[0307] The individual blocks within the electronic device 1002 may be enabled / disabled as per the below table 12:
[0308] FunctionsAuto Selection of FrameManual Selectionof Multiple FrameManual Selection of Single FrameTest Frame GenerationRequiredNot requiredNot requiredReference Frame -to be used for generation of all input frames to Aesthetic Score Calculationultra-wide cameraser selected camera (ultra-wide / wide / Tele1 / tele2)ser selected camera (ultra-wide / wide / Tele1 / tele2)AestheticScore Calculation -For the region of interest / Test Frame(s)RequiredRequiredNot requiredDetermined Camera With Frame -For the region of interest / Test Frame(s)Sensor SelectionRequiredRequiredRequiredSensor deviation calculationRequiredRequiredNot requiredCrop Parameter(s)Crop SelectionRequiredRequiredRequiredCrop Deviation CalculationRequiredRequiredNot requiredSensor mode correctionRequiredRequiredRequiredHardware Override Selection unitSelection Factor CalculationRequiredRequiredNot requiredGeneration of (1) "Best Frame" and / or "User Capture“(2) Give the Option to a user to select among the top Frames with the best selection scoreRequiredRequiredNot requiredUpdate the system with the determined camera, Selected Sensor mode, Selected Crop region to generate capture frameRequiredRequiredRequired
[0309] Figure 22 illustrates a flowchart showing a method 2200 performed by the electronic device 1002, in accordance with an embodiment of the present disclosure.
[0310] The method 2200 can be performed by programmed computing devices, for example, based on instructions retrieved from non-transitory computer readable media. The computer-readable media can include machine-executable or computer-executable instructions to perform all or portions of the described method. The computer readable media may be, for example, digital memories, magnetic storage media, such as magnetic disks and magnetic tapes, hard drives, or optically readable data storage media.
[0311] The method 2200 includes a series of operations shown at step 2202 through step 2212 of Figure 22. The method 2200 may be performed by the electronic device 1002 in conjunction with the modules 1112, the details of which are explained in conjunction with Figures 10 to 19, and the same are not repeated here for the sake of brevity in the present disclosure. The method 2200 begins at step 2202.
[0312] At step 2202, the method 2200 includes generating the plurality of representative images having the plurality of region of interests (ROIs). For generating the plurality of representative images, the method includes generating the plurality of representative images of the scene using the camera having the maximum field of view (FOV) from the plurality of cameras 1008 in the device 1004 independent of the scene selected by the user input. The scene may be selected by the user input using the at least one camera from the plurality of cameras in the device 1004. At step 2202, the method 2200 includes generating the plurality of representative images having the plurality of region of interests (ROIs), after receiving a command to obtain an image using a user selected camera to acquire the image. Before the step 2202, the method 2200 includes obtaining or receiving a command to capture or obtain an image using a camera from the plurality of cameras, where the camera is selected by the user input.
[0313] The method 2200 includes identifying the one or more objects 1128 in the frame of the camera having the maximum FOV. The method includes generating the plurality of test frames 1212 corresponding to the plurality of representative images. Each test frame from the plurality of test frames 1212 may include the one or more objects 1128 aligned at one of the plurality of positions. Each test frame may be generated based on the camera having the maximum FOV from the plurality of cameras 1008 of the device 1004. The method 2200 includes generating the plurality of test frames 1212 with the plurality of positions of the one or more objects 1128 may be based on the category of the one or more objects 1128. Further, the method 2200 includes identifying the one or more faces 1302 in the plurality of representative images from the camera having the maximum FOV. The method 2200 includes generating the plurality of test frames 1212 corresponding to the plurality of representative images. Each test frame from the plurality of test frames 1212 may include the at least one identified faces.
[0314] Further, the method 2200 includes identifying the one or more flares 1304 in the plurality of representative images from the camera having the maximum FOV. The method 2200 includes generating the plurality of test frames 1212 corresponding to the plurality of representative images. At least one test frame from the plurality of test frames 1212 having the identified at least flare may be disregarded.
[0315] Prior to generating the plurality of test frames 1212, the method 2200 includes comparing the aspect ratio of the selected scene by the user input with the aspect ratio of the ROI of the one or more objects 1128 for generating the initial test frame. The method 2200 includes generating the remaining plurality of test frames 1212 based on the category of the one or more objects 1128 in the initial frame.
[0316] At step 2204, the method 2200 includes determining the aesthetic score for each representative image of the plurality of representative images. At step 2204, the method 2200 includes identifying the aesthetic score for each representative image of the plurality of representative images. The method 2200 includes determining the aesthetic score for each test frame, corresponding to each representative image. The method 2200 includes comparing the aesthetic score of each test frame with one or more of the pre-defined threshold aesthetic score and the aesthetic score of the reference image or the aesthetic score of the scene selected by the user input. The method 2200 includes identifying the at least one test frame, corresponding to the at least one representative image, having the aesthetic score greater than the one or more of the pre-defined threshold aesthetic score and the aesthetic score of the reference image or the aesthetic score of the scene selected by the user input. The method 2200 includes discarding the one or more test frames from the plurality of test frames 1212 having the aesthetic score less than the one or more pre-defined threshold aesthetic score or the aesthetic score of the reference image or the aesthetic score of the scene selected by the user input. In an embodiment, the reference image may include the frame of the camera having the maximum FOV and aligned to the FOV of the scene selected by the user input.
[0317] At step 2206, the method 2200 includes determining for the at least one representative image, the camera 1138 having the field of view (FOV) corresponding to the ROI of the at least representative image. To determine the camera 1138 from the plurality of cameras 1008, the method 2200 includes identifying the camera 1138 from the plurality of cameras 1008 having the maximum optical zoom ratio that covers the complete FOV of the at least representative image in the FOV of the identified camera 1138.
[0318] After determining, for the at least one representative image, the camera 1138, the method 2200 includes comparing the determined camera 1138 having the FOV corresponding to the ROI of the at least one representative image and the at least one camera from the plurality of cameras 1008 of the device 1004 used for selecting the scene by the user input. The method 2200 includes determining the sensor deviation between the determined camera 1138 having the FOV corresponding to the ROI of the at least one representative image and the at least one camera from the plurality of cameras 1008 of the device 1004 used for selecting the scene by the user input, based on the comparison.
[0319] At step 2208, the method 2200 includes determining the one or more crop parameters 1140 for the determined camera 1138 for aligning the FOV of the frame from the determined camera 1138 to the at least one representative image. At step 2208, the method 2200 includes identifying the one or more crop parameters 1140 for the determined camera 1138 to align the FOV of the frame from the determined camera 1138 with the at least one representative image. The method 2200 includes determining the one or more crop parameters 1140 based on the difference between the camera having the maximum FOV and the FOV of the determined camera 1138, for the identified at least one test frame, corresponding to the ROI of the at least one representative image. The method 2200 includes determining the one or more crop parameters 1140 on the identified at least one test frame, to align the FOV of the image from the determined camera 1138 with the FOV of the identified at least one test frame by using the plurality of parameters. The plurality of parameters includes optical zoom reference, sensor output size of the determined camera and the camera having the maximum FOV.
[0320] After determining the one or more crop parameters 1140, the method 2200 includes determining the sensor mode from the plurality of sensor modes 1142 of the determined camera 1138, for the identified at least one test frame. The plurality of sensor modes 1142 may include the remosaic mode and the binning mode. Further, the method 2200 includes updating the determined one or more crop parameters 1140 based on the determined sensor mode from the plurality of sensor modes 1142 of the determined camera 1138. The image may be captured by applying the updated one or more crop parameters 1140.
[0321] At step 2210, the method 2200 includes determining the best frame based on the aesthetic score, the sensor deviation, and the crop deviation of the at least one representative image
[0322] The method 2200 includes determining the crop deviation based on the difference between the determined camera 1138, for the identified at least one test frame, corresponding to the ROI of the at least one representative image, and the FOV of the at least one camera from the plurality of cameras 1008 of the device 1004 used for selecting the scene by the user input.
[0323] Further, after determining the sensor mode and updating the one or more crop parameters 1140, the method 2200 includes determining the difference between the aesthetic score of the identified at least one test frame and the total deviation. The total deviation may be determined based on the sensor deviation and the crop deviation. The method 2200 includes determining the selection factor based on the difference. The method 2200 includes determining the best frame based on the selection factor. Further, the selection factor may include the plurality of modes. The plurality of modes may include the automatic mode and the manual mode. The selection factor may include at least one of the aesthetic score, the sensor deviation, or the crop deviation of the at least one representative image. The method 2200 includes determining the best frame when the selection factor may be at least equal to or greater than the high threshold selection factor. The method 2200 includes determining the best frame when the aesthetic score of the at least one representative image may be at least equal to or greater than the pre-defined score. The method 2200 includes determining the best frame when the sensor deviation of the at least one representative image may be at least equal to or greater than the pre-defined sensor deviation. The method 2200 includes determining the best frame when the crop deviation of the at least one representative image may be at least equal to or greater than the pre-defined crop deviation.
[0324] When the selection factor includes the automatic mode, the method 2200 includes comparing the selection factor with the plurality of threshold selection factors. The plurality of threshold selection factors includes the low threshold selection factor, the mid-threshold selection factor, and the high threshold selection factor. The method 2200 includes determining the best frame when the selection factor may be at least equal to or greater than the mid-threshold selection factor and lesser than the high threshold selection factor. Thus, provides the best frame to the user along with the initial frame of the scene selected by the user input to capture the image. Further, the method 2200 includes determining the best frame when the selection factor may be at least equal to or greater than the high threshold selection factor and provides only the best frame to the user.
[0325] At step 2212, the method 2200 includes capturing the image corresponding to the determined best frame, by applying the determined one or more crop parameters 1140 to the frame of the determined camera 1138. At step 2212, the method 2200 includes obtaining the image corresponding to the determined best frame, by applying the identified one or more crop parameters 1140 to the frame of the determined camera 1138.For capturing the image corresponding to the determined best frame, the method 2200 includes capturing the image of the scene with the determined camera 1138, corresponding to the best frame as determined by the selection factor, by selecting the sensor mode and then applying the determined one or more crop parameters 1140 for the corresponding representative image.
[0326] Figures 23A-23B illustrate a flowchart depicting a method 2300 performed by the electronic device 1002, in accordance with another embodiment of the present disclosure.
[0327] The method 2300 can be performed by programmed computing devices, for example, based on instructions retrieved from non-transitory computer readable media. The computer readable media can include machine-executable or computer-executable instructions to perform all or portions of the described method. The computer readable media may be, for example, digital memories, magnetic storage media, such as magnetic disks and magnetic tapes, hard drives, or optically readable data storage media.
[0328] The method 2300 includes a series of operations shown at step 2302 through step 2316 of Figures 23A-23B. The method 2300 may be performed by the electronic device 1002 in conjunction with the modules 1112. The method 2300 begins at step 2302.
[0329] At step 2302, the method 2300 includes generating the plurality of representative images having the plurality of ROIs.
[0330] At step 2304, the method 2300 includes determining the aesthetic score for each representative image of the plurality of representative images. The aesthetic score of each of the representative image may be greater than the pre-defined threshold aesthetic value and aesthetic value of the scene generated by the user input by the at least one camera from the plurality of cameras 1008 of the device 1004.
[0331] At step 2306, the method 2300 includes determining for each of the plurality of representative images, the camera 1138 having the field of view FOV corresponding to the ROI of the respective representative image and covers FOV of the respective representative image.
[0332] At step 2308, the method 2300 includes determining the sensor deviation for each of the plurality of representative images, by determining the deviation between the determined camera 1138 having the field of view (FOV) corresponding to the ROI of the respective representative image and the at least one camera from the plurality of cameras 1008 of the device 1004 used for selecting the scene by the user input.
[0333] At step 2310, the method 2300 includes determining for each of the plurality of representative images, the one or more crop parameters 1140 for the frame of the determined camera 1138 to align the frame of the determined camera 1138 to the respective representative image.
[0334] At step 2312, the method 2300 includes determining the crop deviation for each of the plurality of representative images, by determining the deviation between the size of the ROI of the respective representative image and the size of the ROI of the scene selected by the user input.
[0335] At step 2314, the method 2300 includes recommending the best frame from the plurality of representative images to the user based on the aesthetic score, the sensor deviation, and the crop deviation.
[0336] At step 2316, the method 2300 includes capturing the recommended best frame by applying the determined one or more crop parameters 1140 to the frame of the determined camera.
[0337] Figures 24A-24E illustrate multiple use cases of the electronic device 1002, in accordance with an embodiment of the present disclosure.
[0338] Referring to Figure 24A, the electronic device 1002 provides Test Frame TF_X generated with all 3 road signs in the frame. Tele camera may be selected as the determined camera 1138 corresponding to the Test Frame TF_X. The one or more crop parameters 1140 applied the right crop on Tele camera. Further, the final selection factor calculated may be highest for TF_X (TF_SF) providing the best frame 2402. Lastly, the hardware override selection unit 1144 may generate a duplicate shot along with the user-selected shot which captures the rod signs clearly based on SF_LOW > TF_SF < SF_MID.
[0339] Referring to Figure 24B, the electronic device 1002 provides the wide camera determined with the discarding frame having the flare detection. The test frames TF_X may be generated with the wide frames. The one or more crop parameters 1140 may be applied the right crop on the wide camera. The final selection factor calculated may be highest for TF_X (TF_SF) which provides the best frame 2404. Lastly, the hardware override selection unit 1144 may generate a best frame shot & discard the user shot which has flare based on TF_SF > SF_MID.
[0340] Referring to Figure 24C, the electronic device 1002 provides Test Frame TF_X generated with full ROI of the ultra-wide frame. The ultra-wide camera may be selected as the determined camera 1138 corresponding to the Test Frame TF_X. The one or more crop parameters 1140 applied the right crop on the ultra-wide camera. Further, the final selection factor calculated may be highest for TF_X (TF_SF) providing the best frame 2406. Lastly, the hardware override selection unit 1144 may generate a duplicate shot along with the user-selected shot which captures the scene in the best possible way based on SF_LOW > TF_SF < SF_MID.
[0341] Referring to Figure 24D, the electronic device 1002 provides Test Frame TF_X generated with right crop covering the house completely. The tele camera may be selected as the determined camera 1138 corresponding to the Test Frame TF_X. The one or more crop parameters 1140 applied the right crop on the tele camera. Further, the final selection factor calculated may be highest for TF_X (TF_SF) providing the best frame 2408. Lastly, the hardware override selection unit 1144 may generate the best frame shot & discard the user shot that has missed part of the object of interest based on TF_SF > SF_MID.
[0342] Referring to Figure 24E, the electronic device 1002 provides Test Frame TF_X generated with right crop covering the house completely. The tele camera may be selected as the determined camera 1138 corresponding to the Test Frame TF_X. The one or more crop parameters 1140 applied the right crop on the tele camera. Further, the final selection factor calculated may be highest for TF_X (TF_SF) providing the best frame 2410. Lastly, the hardware override selection unit 1144 may generate a generate the best frame shot & discard the user shot that has missed part of the object of interest based on TF_SF > SF_MID.
[0343] According to an embodiment, a method for aesthetic-based image capture may comprise generating a preview using a camera with a field of view(FOV) from among a plurality of cameras in the device, identifying one or more objects from the generated preview, generating a plurality of test frames, each test frame having one or more objects aligned at a plurality of positions of the generated preview, generating an aesthetic score for each test frame, identifying the test frame having the highest aesthetic score from among the plurality of test frames, selecting a camera from among the cameras in the device, the selected camera having a capability to capture the scene matching the test frame with the highest aesthetic score, capturing an image using the selected camera by cropping the FOV so as to match the alignment of the objects in a preview of the selected camera with that of the test frame having the highest aesthetic score.
[0344] According to an embodiment, the camera may include a camera with maximum FOV.
[0345] According to an embodiment, the method may include determining a category of the one or more objects, and generating the plurality of test frames based on the determined category of the one or more objects.
[0346] According to an embodiment, the method may include determining a ROI of the combination of one or more objects, increasing the ROIs of one or more objects in all directions or in one of all directions, and overlapping the ROI of the plurality of test frames with one or more objects.
[0347] According to an embodiment, the method may include identifying one or more faces in the scene from the generated preview, determining the number and the coordinates of one or more faces, and generating the plurality of test frames, each test frame having one or more faces.
[0348] According to an embodiment, the method may include identifying one or more flares in the scene from the generated preview, and disregarding at least one test frame from the plurality of test frames having the identified at least one flare.
[0349] According to an embodiment, the method may include adjusting a width a ROI of the one or more objects for generating the plurality of test frames.
[0350] According to an embodiment, the method may include determining the aesthetic score based on the one or more objects and color properties of the plurality of test frames.
[0351] According to an embodiment, an electronic device 1002 for aesthetic-based image capture, the electronic device 1002 may include memory 1108 storing instructions, a plurality of cameras, at least one processor 1104.
[0352] According to an embodiment, the instructions, when executed by the at least one processor 1104 individually or collectively, cause the electronic device 1002 to, generate a preview using a camera with a field of view(FOV) from among a plurality of cameras in the device, identify one or more objects from the generated preview, generate a plurality of test frames, each test frame having one or more objects aligned at a plurality of positions of the generated preview, generate an aesthetic score for each test frame, identify the test frame having the highest aesthetic score from among the plurality of test frames, select a camera from among the cameras in the device, the selected camera having a capability to capture the scene matching the test frame with the highest aesthetic score, capture an image using the selected camera by cropping the FOV so as to match the alignment of the objects in a preview of the selected camera with that of the test frame having the highest aesthetic score.
[0353] According to an embodiment, the instructions, when executed by the at least one processor 1104 individually or collectively, cause the electronic device 1002 to, determine a category of the one or more objects, and generate the plurality of test frames based on the determined category of the one or more objects.
[0354] According to an embodiment, the instructions, when executed by the at least one processor 1104 individually or collectively, cause the electronic device 1002 to, determine a ROI of the combination of one or more objects, increase the ROIs of one or more objects in all directions or in one of all directions, and overlap the ROI of the plurality of test frames with one or more objects.
[0355] According to an embodiment, the instructions, when executed by the at least one processor 1104 individually or collectively, cause the electronic device 1002 to, identify one or more faces from the generated preview, determine the number and the coordinates of one or more faces, and generate the plurality of test frames, each test frame having one or more faces.
[0356] According to an embodiment, the instructions, when executed by the at least one processor 1104 individually or collectively, cause the electronic device 1002 to, identify one or more flares from the generated preview, and disregard at least one test frame from the plurality of test frames having the identified at least one flare.
[0357] According to an embodiment, the instructions, when executed by the at least one processor 1104 individually or collectively, cause the electronic device 1002 to, adjust a width a ROI of the one or more objects in the scene for generating the plurality of test frames, and determine the aesthetic score based on the one or more objects and color properties of the plurality of test frames.
[0358] As would be gathered, the electronic device 1002 disclosed provides a comprehensive approach to capture the best frame for the optimum shot of the image by the device 1004. The configuration as disclosed captures the object of interest clearly and efficiently in a less stipulated time without compromising on the quality of the image / best frame as captured. The configuration as disclosed ensures that the amateur users may also get the best shot of the scene selected by the user input without any hassle. This configuration also saves time and money of the user, thus being cost-effective solution.
[0359] While specific language has been used to describe the present disclosure, any limitations arising on account thereto, are not intended. As would be apparent to a person in the art, various working modifications may be made to the method 2200 in order to implement the inventive concept as taught herein. The drawings and the foregoing description give examples of embodiments. Those skilled in the art will appreciate that one or more of the described elements may well be combined into a single functional element. Alternatively, certain elements may be split into multiple functional elements. Elements from one embodiment may be added to another embodiment.
Claims
1.A method for aesthetic-based image capture, comprising:generating a preview using a camera with a field of view (FOV) from among a plurality of cameras in the device;identifying one or more objects from the generated preview;generating a plurality of test frames, each test frame having one or more objects aligned at a plurality of positions of the generated preview;generating an aesthetic score for each test frame;identifying the test frame having the highest aesthetic score from among the plurality of test frames;selecting a camera from among the cameras in the device, the selected camera having a capability to capture the scene matching the test frame with the highest aesthetic score;capturing an image using the selected camera by cropping the FOV so as to match the alignment of the objects in a preview of the selected camera with that of the test frame having the highest aesthetic score.2.The method of claim 1, wherein the camera includes a camera with maximum FOV.3.The method of claim 1, wherein the generating the plurality of test frames further comprises:determining a category of the one or more objects; andgenerating the plurality of test frames based on the determined category of the one or more objects.4.The method of claim 1, wherein the generating the plurality of test frames further comprises:determining a ROI of the combination of one or more objects;increasing the ROIs of one or more objects in all directions or in one of all directions; andoverlapping the ROI of the plurality of test frames with one or more objects.5.The method of claim 1, further comprising:identifying one or more faces in the scene from the generated preview;determining the number and the coordinates of one or more faces; andgenerating the plurality of test frames, each test frame having one or more faces.6.The method of claim 1, further comprising:identifying one or more flares in the scene from the generated preview; anddisregarding at least one test frame from the plurality of test frames having the identified at least one flare.7.The method of claim 1, wherein the generating the plurality of test frames further comprises:adjusting a width a ROI of the one or more objects for generating the plurality of test frames.8.The method of claim 1, wherein the generating an aesthetic score further comprising:determining the aesthetic score based on the one or more objects and color properties of the plurality of test frames.9.An electronic device for aesthetic-based image capture, the electronic device comprising:memory storing instructions;a plurality of cameras;at least one processor;wherein the instructions, when executed by the at least one processor individually or collectively, cause the electronic device to,generate a preview using a camera with a field of view(FOV) from among a plurality of cameras in the device;identify one or more objects from the generated preview;generate a plurality of test frames, each test frame having one or more objects aligned at a plurality of positions of the generated preview;generate an aesthetic score for each test frame;identify the test frame having the highest aesthetic score from among the plurality of test frames;select a camera from among the cameras in the device, the selected camera having a capability to capture the scene matching the test frame with the highest aesthetic score;capture an image using the selected camera by cropping the FOV so as to match the alignment of the objects in a preview of the selected camera with that of the test frame having the highest aesthetic score.10.The electronic device of claim 9, wherein the camera includes a camera with maximum FOV.11.The electronic device of claim 9, wherein the instructions, when executed by the at least one processor individually or collectively, cause the electronic device to,determine a category of the one or more objects; andgenerate the plurality of test frames based on the determined category of the one or more objects.12.The electronic device of claim 9, wherein the instructions, when executed by the at least one processor individually or collectively, cause the electronic device to,determine a ROI of the combination of one or more objects;increase the ROIs of one or more objects in all directions or in one of all directions; andoverlap the ROI of the plurality of test frames with one or more objects.13.The electronic device of claim 9, wherein the instructions, when executed by the at least one processor individually or collectively, cause the electronic device to,identify one or more faces from the generated preview;determine the number and the coordinates of one or more faces; andgenerate the plurality of test frames, each test frame having one or more faces.14.The electronic device of claim 9, wherein the instructions, when executed by the at least one processor individually or collectively, cause the electronic device to,identify one or more flares from the generated preview; anddisregard at least one test frame from the plurality of test frames having the identified at least one flare.15.The electronic device of claim 9, wherein the instructions, when executed by the at least one processor individually or collectively, cause the electronic device to,adjust a width a ROI of the one or more objects in the scene for generating the plurality of test frames; anddetermine the aesthetic score based on the one or more objects and color properties of the plurality of test frames.
Citation Information
Patent Citations
Image processing method
KR101315464B1
Mobile terminal and method for controlling the same
KR1020170011321A
Composition for inducing proliferation and / or migration of mesenchymal stem cells
KR102669135B1
Intellegent frame skipping in video coding based on similarity metric in compressed domain
US20100027663A1