Method and system for improving quality of frames captured by multi-camera device

Abstract

A method and system for improving quality of a plurality of frames captured by a primary camera lens of a multi-camera device is disclosed. A plurality of processed primary lens frames are obtained by applying electronic image stabilization on the plurality of primary lens frames, and then a set of wide frame coordinates are determined, corresponding to at least one edge of a predefined wide frame profile associated with a wide-angle camera lens of the multi-camera device. Thereafter, edge sections from a plurality of wide lens frames, captured from the wide-angle camera lens, are extracted based on the determined set of coordinates. The processed primary lens frames are stitched with the extracted edge sections and a focal correction operation, and a tonal correction operation are applied on the stitched frames to obtain a plurality of stabilized frames.

Description

METHOD AND SYSTEM FOR IMPROVING QUALITY OF FRAMES CAPTURED BY MULTI-CAMERA DEVICE

[0001] The present invention generally relates to camera frame processing, and more particularly relates to a method and system for improving the quality of a plurality of frames captured by a primary camera lens of a multi-camera device.

[0002] Generally, during capturing an image or a video using a camera of a handheld user device, an unwanted motion may occur due to, for example, the camera shaking in a particular direction. To reduce the effects of such unwanted motions, techniques for digital image stabilization are usually applied. Presently known techniques for digital image stabilization are optical image stabilization (OIS), electronic image stabilization (EIS), and hybrid image stabilization which is a combination of EIS and OIS. OIS is a hardware-based implemented method where a sensor in the camera system moves against the motion of the smart device. OIS results in frames of the same dimensions as the original. However, OIS is considered a costly technique. Generally, EIS is preferred over OIS because of the low cost of implementation. However, EIS has several limitations, for instance, in EIS, a smaller cropped frame is moved inside the main frame of the primary lens, resulting in smaller frames than the original.

[0003] This summary is provided to introduce a selection of concepts, in a simplified format, that are further described in the detailed description of the invention. This summary is neither intended to identify key or essential inventive concepts of the invention nor is it intended for determining the scope of the invention.

[0004] According to an embodiment of the disclosure, a method for improving quality of a plurality of frames captured by a primary camera lens of a multi-camera device may include obtaining a plurality of processed primary lens frames by applying electronic image stabilization (EIS) on the plurality of primary lens frames. The method may include determining a set of wide frame coordinates, corresponding to at least one edge of a predefined wide frame profile associated with a wide-angle camera lens of the multi-camera device, based on a set of displaced coordinates of the plurality of processed primary lens frames. The method may include extracting one or more edge sections from a plurality of wide lens frames, captured from the wide-angle camera lens, based on the determined set of coordinates. The method may include stitching the processed primary lens frames with the one or more extracted edge sections for obtaining one or more stitched frames. The method may include applying a focal correction operation and a tonal correction operation on the one or more stitched frames corresponding to the plurality of processed primary lens frames.

[0005] According to an embodiment of the disclosure, a multi-camera device may include a memory. The multi-camera device may include at least one processor coupled to the memory. The at least one processor may be configured to obtain a plurality of processed primary lens frames by applying electronic image stabilization (EIS) on the plurality of primary lens frames. The at least one processor may be configured to determine a set of wide frame coordinates, corresponding to at least one edge of a predefined wide frame profile associated with a wide-angle camera lens of the multi-camera device, based on a set of displaced coordinates of the plurality of processed primary lens frames. The at least one processor may be configured to extract one or more edge sections from a plurality of wide lens frames, captured from the wide-angle camera lens, based on the determined set of coordinates. The at least one processor may be configured to stitch the processed primary lens frames with the one or more extracted edge sections for obtaining one or more stitched frames. The at least one processor may be configured to apply a focal correction operation and a tonal correction operation on the one or more stitched frames corresponding to the plurality of processed primary lens frames.

[0006] According to an embodiment of the present disclosure, a computer-readable storage medium which is configured to store instruction is provided. The instructions, when executed by at least one processor of a device, may cause the at least one processor to perform the method corresponding.

[0007] To further clarify the advantages and features of the present invention, a more particular description of the invention will be rendered by reference to specific embodiments thereof, which is illustrated in the appended drawings. It is appreciated that these drawings depict only typical embodiments of the invention and are therefore not to be considered limiting of its scope. The invention will be described and explained with additional specificity and detail with the accompanying drawings.

[0008] These and other features, aspects, and advantages of the present invention will become better understood when the following detailed description is read with reference to the accompanying drawings in which like characters represent like parts throughout the drawings, wherein:

[0009] Figure 1is a pictorial diagram illustrating limitations of implementing EIS, according to a related art;

[0010] Figure 2is a pictorial diagram illustrating an exemplary cropping of a camera frame after EIS, according to a related art;

[0011] Figure 3is a block diagram illustrating the frame stabilization system, according to an embodiment of the present disclosure;

[0012] Figure 4is a block diagram depicting modules of the frame stabilization system, according to an embodiment of the present disclosure;

[0013] Figure 5is a schematic diagram depicting an overall workflow for stabilizing the frames captured by a camera unit, according to an embodiment of the present disclosure;

[0014] Figure 6is a pictorial diagram illustrating an exemplary implementation of the padding module, according to an embodiment of the present disclosure;

[0015] Figure 7is a pictorial diagram illustrating the predefined main frame profile and predefined wide frame profile and implementation of the frame profiler module, according to an embodiment of the present disclosure;

[0016] Figure 8is a pictorial diagram illustrating the determination of SAFP in an implementation of the frame profiler module, according to an embodiment of the present disclosure;

[0017] Figure 9is a pictorial diagram illustrating the determination of ESP in an implementation of the frame profiler module, according to an embodiment of the present disclosure;

[0018] Figures 10a and 10bare schematic diagrams illustrating an exemplary implementation of the edge resampling module, according to an embodiment of the present disclosure;

[0019] Figure 11ais a pictorial diagram illustrating the determination of the stitching profile in an implementation of the stitching profiler module, according to an embodiment of the present disclosure;

[0020] Figure 11bis a schematic diagram illustrating an implementation of the stitching profiler module based on the determined stitching profile, according to an embodiment of the present disclosure;

[0021] Figure 12is a block diagram illustrating the method of focal correction in the implementation of the correction module, according to an embodiment of the present disclosure;

[0022] Figure 13is a block diagram illustrating the method of tonal correction in the implementation of the correction module, according to an embodiment of the present disclosure; and

[0023] Figures 14a and 14bare block diagrams illustrating a flow of the method for improving the quality of a plurality of frames, according to an embodiment of the present disclosure.

[0024] Further, skilled artisans will appreciate that elements in the drawings are illustrated for simplicity and may not have necessarily been drawn to scale. For example, the flow charts illustrate the method in terms of the most prominent steps involved to help to improve understanding of aspects of the present invention. Furthermore, in terms of the construction of the device, one or more components of the device may have been represented in the drawings by conventional symbols, and the drawings may show only those specific details that are pertinent to understanding the embodiments of the present invention so as not to obscure the drawings with details that will be readily apparent to those of ordinary skill in the art having the benefit of the description herein.

[0025] For the purpose of promoting an understanding of the principles of the invention, reference will now be made to the various embodiments and specific language will be used to describe the same. It will nevertheless be understood that no limitation of the scope of the invention is thereby intended, such alterations and further modifications in the illustrated system, and such further applications of the principles of the invention as illustrated therein being contemplated as would normally occur to one skilled in the art to which the invention relates.

[0026] It will be understood by those skilled in the art that the foregoing general description and the following detailed description are explanatory of the invention and are not intended to be restrictive thereof.

[0027] Reference throughout this specification to "an aspect", "another aspect" or similar language means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. Thus, appearances of the phrase "in an embodiment", "in another embodiment" and similar language throughout this specification may, but do not necessarily, all refer to the same embodiment.

[0028] The terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process or method that comprises a list of steps does not include only those steps but may include other steps not expressly listed or inherent to such process or method. Similarly, one or more devices or sub-systems or elements or structures or components proceeded by "comprises... a" does not, without more constraints, preclude the existence of other devices or other sub-systems or other elements or other structures or other components or additional devices or additional sub-systems or additional elements or additional structures or additional components.

[0029] Nowadays, most of the user devices are equipped with an advanced camera system. Such user devices may be smartphones, tablets, or the like, and the advanced camera system may feature multi-specification cameras equipped with different types of lenses. Generally, multi-specification cameras comprise at least one primary lens, and at least one wide angle lens. A wide angle lens has a large field of view (FOV), whereas a primary lens has a smaller FOV as compared to the wide angle lens. However, the quality of frames of wide angle lens is lower as compared to the quality of primary lens frames. The terms "frame" and "camera frame" are used interchangeably throughout the present disclosure, and may refer to a FOV of a camera unit of a user device before / during capturing of an image or video.

[0030] An object of the present disclosure is to address the problem of frame cropping by electronic image stabilization (EIS) due to which the resulting field of view (FOV) of a camera is reduced. Many use cases of using a camera system employing both the primary lens and the wide angle lens may involve using the combination of more than one camera lens. In view thereof, the present disclosure provides techniques for widening the resulting FOV by using the combination of a wide angle lens and a primary lens. The system and method of the present disclosure will now be described in detail.

[0031] Figure 1is a pictorial diagram illustrating the limitations of implementing EIS, according to an existing art. EIS reduces the resultant field of view (FOV) due to frame cropping. In current smartphone cameras, EIS works by cropping 20-25% of the original frame and stabilizes the FOV by moving the cropped frame based on inertial measurement unit (IMU) data. For example, frame 0 depicts an original FOV corresponding to the original frame of a smartphone camera. When an unwanted movement is detected, the EIS crops the original frame and moves or pans the cropped frame according to the IMU data to stabilize the FOV. In the example, frame 1 depicts the downward movement of the cropped frame when an unwanted movement towards the top is detected. Similarly, frame 2 depicts the upward movement of the cropped frame when the unwanted movement towards the bottom is detected, and finally, frame 3 depicts the movement of the cropped frame towards the left and bottom when an unwanted movement towards the top and right, respectively, is detected. Thus, since cropping in EIS results in a smaller frame, full FOV may not be captured. Hence, a key problem associated with the current implementation of EIS is cropped frames, where the final output, i.e., an EIS processed frame is always smaller than the original frame.

[0032] Figure 2is a pictorial diagram illustrating an exemplary cropping of a camera frame after EIS, according to an existing art. As shown in the figure, frame 201 depicts an original FOV corresponding to an original frame, whereas frame 203 depicts a reduced FOV corresponding to a cropped primary lens frame. For example, the original frame 201 may be cropped by 25% during EIS resulting in a smaller FOV corresponding to the cropped frame 203. Accordingly, if the original frame 201 had a dimension of AxB, the cropped frame 203 would have a dimension of 0.75Ax0.75B, resulting in reduced FOV and thus affecting user experience.

[0033] Accordingly, there is a need to provide a technique for overcoming the above described drawbacks of implementing EIS. Additionally, there is a need to provide a technique using which the final output camera frame remains the same as the original camera frame.

[0034] Figure 3is a block diagram illustrating the frame stabilization system, according to an embodiment of the present disclosure. The present disclosure provides a frame stabilization system 303 which may be employed in a user device 301. The user device 301 may correspond to a multi-camera device. The frame stabilization system 303 may comprise a processor 305, a memory 307, a camera unit 309, an inertial measurement unit (IMU) 319, and modules 321. The processor 305 may include one or more general processors, such as, but not limited to, digital signal processors, application-specific integrated circuits, field-programmable gate arrays, servers, networks, digital circuits, analog circuits, combinations thereof, or other now-known or later developed devices for analyzing and processing data. The processor 305 may implement a software program, such as code generated manually (i.e., programmed).

[0035] The memory 307 may be communicatively coupled to the processor 305. The memory 307 may be configured to store data and instructions executable by the processor 305 to perform the method(s) disclosed throughout the present disclosure. The memory 307 may include, but is not limited to, a non-transitory computer-readable storage media, such as various types of volatile and non-volatile storage media including, but not limited to, random access memory, read-only memory, programmable read-only memory, electrically programmable read-only memory, electrically erasable read-only memory, flash memory, magnetic tape or disk, optical media and the like. In one example, the memory 307 may include a cache or random-access memory for the processor 305. In alternative examples, the memory 307 may be separate from the processor 305, such as a cache memory of a processor, the system memory, or other memory. The memory 307 may be an external storage device or database for storing data. The memory 307 may be operable to store instructions executable by the processor 305. The functions, acts, or tasks illustrated in the figures or described herein may be performed by the programmed processor 305 for executing the instructions stored in the memory 307. The functions, acts, or tasks are independent of the particular type of instruction set, storage media, processor, or processing strategy and may be performed by software, hardware, integrated circuits, firmware, micro-code, and the like, operating alone or in combination. Likewise, processing strategies may include multiprocessing, multitasking, parallel processing, and the like.

[0036] The camera unit 309 may be equipped with multi-specification cameras comprising a primary camera 311 and a wide-angle camera 313. The primary camera 311 may comprise a primary lens 313 providing a plurality of primary lens frames having a predefined main frame profile. The predefined main frame profile may correspond to a predefined set of coordinates associated with the plurality of primary lens frames. The wide-angle camera 315 may comprise a wide-angle lens 317 providing a plurality of wide lens frames having a predefined wide frame profile. The predefined wide frame profile may correspond to a predefined set of coordinates associated with the plurality of wide lens frames. In an embodiment, the camera unit 309 may comprise a plurality of primary cameras and a plurality of wide-angle cameras, each having similar features as described for the primary camera 313 and the wide-angle camera 315, respectively. In an embodiment, the plurality of primary lens frames may be smaller than the plurality of wide lens frames. Therefore, FOV associated with the plurality of primary lens frames is smaller than the FOV associated with the plurality of wide lens frames. Since a FOV is directly associated with a frame of a camera lens, the term FOV may be used interchangeably with the term "frame" throughout the description. In an embodiment, the quality of the plurality of primary lens frames may be superior to the quality of the plurality of wide lens frames. In an embodiment, EIS is performed for stabilization of the plurality of primary lens frames.

[0037] The IMU 319 may comprise a combination of one or more accelerometers and one or more gyroscopes and may facilitate tracking of a movement of the user device 301. The movement may be tracked in respect of acceleration, velocity, orientation, and position of the user device 301. Such movement of the user device 301 may be translated to a movement of the camera unit 309 during capturing of an image or a video. When EIS is performed on the plurality of primary lens frames, movement information associated with the movement of the user device 301, and thereby the camera unit 309, as obtained from the IMU 319 may be utilized. In an embodiment, when EIS is applied to the plurality of primary lens frames, the resulting frames may be cropped by a predefined EIS crop factor. In an embodiment, the resulting cropped primary lens frames and the FOV corresponding to each of the resulting cropped primary lens frames may be reduced by dimensions equivalent to the EIS crop factor. The modules 321 of the frame stabilization system 303 may be configured to ensure that the dimensions of the resulting cropped primary frame lens remain equivalent to that of the plurality of primary lens frames before EIS. In an embodiment, the present disclosure may provide techniques for improving quality, in respect of dimensions and stabilization, of the plurality of frames captured by the primary camera lens of the multi-camera device.

[0038] The modules 321 may include a set of instructions that may be executed to cause the frame stabilization system 303 to perform any one or more of the methods disclosed herein. The modules 321 may be configured to perform the steps of the present disclosure using the data stored in the memory 307. In one embodiment, memory 307 may be configured to store the information as required by the modules 321 and the processor 305. The modules 321 of the frame stabilization system 303 will now be described in association with Figures 4 and 5.

[0039] Figure 4is a block diagram depicting modules 321 of the frame stabilization system 303, according to an embodiment of the present disclosure. The modules 321 of the frame stabilization system 303 may include a padding module 401, a frame profiler module 403, an edge resampling module 405, a stitching profiler module 407, and a correction module 409. The padding module 401 may be configured to insert padding in the plurality of primary lens frames and is described in greater detail in conjunction with figure 6. The frame profiler module 403 may be configured to determine a shake adjusted frame profile (SAFP) based on movement information obtained from the IMU 319. The shake adjusted frame profile may correspond to a set of coordinates of the plurality of primary lens that were displaced due to the movement of the user device 301. The frame profiler module 403 may also be configured to determine an edge section profile (ESP) corresponding to the coordinates of a polygon obtained by adding an overlapping region to a padded region within an area defined by the SAFP. The overlapping area may correspond to the processed primary lens frames with respect to the predefined main frame profile and has a width of a predefined buffer value. The frame profiler module 403 is described in greater detail in conjunction with figures 7, 8, and 9. The edge resampling module 405 may be configured to extract wide frame edge sections based on the determined ESP and perform resampling on the extracted wide frame edge sections. The edge resampling module 405 is described in greater detail in conjunction with figures 9, 10a, and 10b. The stitching profiler module 407 may be configured to stitch the resampled wide frame edge sections with a primary lens frame obtained after inserting padding and cropped after EIS. The stitching profiler module 407 may perform the stitching based on a stitching profile corresponding to a region in the primary lens frame where stitching may be performed. The stitching profiler module 407 is described in greater detail in conjunction with figures 11a and 11b. The correction module 409 may be configured to smoothen the stitched frames. The correction module 409 may comprise a focal correction module 411 and a tonal correction module 413, each of which is described in greater detail in conjunction with figures 13 and 14, respectively. According to an embodiment of the present disclosure, the above-mentioned modules, except the padding module 401, of the frame stabilization system 303 may be introduced as a part of an EIS widening pipeline, for widening the primary lens frames processed by EIS, in a related workflow for stabilizing the frames captured by the camera unit 309. The padding module 401 and the EIS widening pipeline may be incorporated in the related workflow to widen a resulting FOV of the primary lens frame obtained after inserting padding and cropped after EIS. An overall workflow, for stabilizing the frames captured by the camera unit 309, including the padding module 401 and the EIS widening pipeline, is described in conjunction with Figure 5.

[0040] Figure 5is a schematic diagram depicting an overall workflow for stabilizing the frames captured by a camera unit, according to an embodiment of the present disclosure. The workflow starts by obtaining a plurality of wide lens frames 501 from the wide-angle camera 315, and a plurality of primary lens frames 503 from the primary camera 311. Thereafter, the padding module 401 may be employed by inserting padding around the edges of the obtained plurality of primary lens frames 503. In an embodiment of the present disclosure, a padding of dimensions equivalent to a predefined EIS crop factor associated with an existing EIS technique, as implemented by an original equipment manufacturer (OEM), may be inserted. The padding module 401 may result in padded primary lens frames 503-1. Thereafter, the padded primary lens frames 503-1 may be stabilized using the EIS method 505. The EIS method 505 may obtain movement information from the IMU 319 and utilize the same for stabilization. During stabilization, the EIS method 505 may crop the padded primary lens frames 503-1 by the predefined EIS crop factor in directions as per the obtained movement information. As a result of stabilization by the EIS method 505, a plurality of processed primary lens frames may be obtained. The plurality of processed primary lens frames may correspond to stabilized primary lens frames with partial padding 503-2. The remaining modules of the frame stabilization system 303 may be applied as a part of an EIS widening pipeline 509 on the plurality of processed primary lens frames, such that resulting stabilized primary lens frames remain of the same dimensions as that of the primary lens frames before stabilization by the existing EIS method. In an embodiment, the movement information from the IMU 319 may be utilized by the EIS widening pipeline 509.

[0041] When the plurality of processed primary lens frames are processed through the EIS widening pipeline 309, the predefined main frame profile associated with the primary camera 311, and the predefined wide frame profile associated with the wide-angle camera 315 are obtained. The predefined main frame profile and the predefined wide frame profile may be fixed for a particular primary lens and wide lens arrangement respectively, and may be pre-stored in configuration settings associated with the camera unit 309 of the user device 301. The frame profiler module 403 may be applied to the plurality of processed primary lens frames to obtain an SAFP corresponding to a set of displaced coordinates of the plurality of processed primary lens frames with respect to the predefined wide frame profile. ESP may be determined corresponding to a set of wide frame coordinates associated with the predefined wide frame profile with respect to the determined SAFP. In an embodiment of the present disclosure, the ESP may represent the coordinates of a polygon obtained by adding an overlapping region to a padded region within an area defined by the SAFP. The overlapping area may correspond to the processed primary lens frames with respect to the predefined main frame profile and has a width of a predefined buffer value. Such an overlapping region may be required for the stitching of frames. The edge resampling module 405 may be employed to extract one or more edge sections from the plurality of wide lens frames based on the determined ESP. As discussed above, the quality of wide lens frames is lower than the quality of primary lens frames. Therefore, the extracted edge sections may be resampled by the edge resampling module 405 to obtain high quality edge sections of quality equivalent to that of the primary lens frames.

[0042] Thereafter, the high-quality edge sections may be stitched with the stabilized primary lens frames 503-2 with stitching profiler module 407 e.g., the plurality of processed primary lens frames), based on a stitching profile (SP) to obtain one or more stitched frames. The SP may correspond to a region in a processed primary lens frame where stitching may be performed. In an embodiment, the width or dimensions of the region may be equivalent to the predefined EIS crop factor. The one or more stitched frames may be processed through the focal correction module 411 for fixing any focal differences that may be present in the one or more stitched frames, and obtaining focal corrected stitched frames. The focal corrected stitched frames may be processed through the tonal correction module 413for fixing any tonal differences that may be present in the focal corrected stitched frames. The EIS widening pipeline 509 may result in one or more stabilized primary lens frames 511 of dimensions equivalent to the main frame profile. In an embodiment, quality and dimensions of the primary lens frames post EIS process may be maintained.Embodiments of the present disclosure provide techniques for compensating predefined EIS crop factor without losing quality of the primary lens frames. Each of the modules 321 of the frame stabilization system 303 will now be described in greater detail.

[0043] Figure 6is a pictorial diagram 600 illustrating an exemplary implementation of the padding module 401, according to an embodiment of the present disclosure. An objective of the padding module 401 may be to preserve the dimensions of primary lens frames and counter the effect of frame cropping by EIS. To achieve this objective, the present disclosure proposes to insert padding around the primary lens frames before the application of a predefined EIS technique. In an embodiment of the present disclosure, the dimensions of the padding may be based on a predefined crop factor associated with the predefined EIS technique. The padding module 401 may be configured to prepare a modified input for the existing EIS method. In an embodiment, when the existing EIS method is applied to the padded primary lens frames, the cropping effect of EIS is addressed. For example, as shown in Figure 6, block 601 may depict a primary lens frame having a dimension A x B. In an example, the predefined EIS method may have a predefined EIS crop factor, for example, 25%. Accordingly, a padding of dimension 0.25A X 0.25B may be added around the primary frame lens, resulting in the padded primary lens frames of dimensions 1.25A x 1.25B, as depicted in block 603. In an ideal scenario, when no movement is detected by the IMU and the predefined EIS method is applied, the padded primary lens frame 603 may be cropped by the predefined EIS crop factor resulting in a stabilized primary lens frame 605 of dimensions equivalent to the primary lens frame 601. However, when a movement of the user device is detected by the IMU, and the padded primary lens frames 603 may be cropped by the predefined EIS crop factor, one of the stabilized frames (607-1, 607-2, 607-3, 607-4) with partial padding may be obtained. The padded primary lens frame 603 may be cropped based on the movement information obtained from the IMU, and padding around the stabilized frames may be present in the directions corresponding to the movement information of the user device. The stabilized frames with partial padding may correspond to processed primary lens frames. In an embodiment, the operations of the remaining modules may be performed with respect to the processed primary lens frames. For example, SAFP associated with the processed primary lens frames may be determined by the frame profiler module 403, as described below in conjunction with Figure 8. The determination of SAFP may also be based on a predefined main frame profile and predefined wide frame profile, as described below in conjunction with Figure 7.

[0044] Figure 7is a pictorial diagram illustrating the predefined main frame profile and predefined wide frame profile and implementation of the frame profiler module 403, according to an embodiment of the present disclosure. As shown in Figure 7, both the primary lens and the wide frame lens may have overlapping FOVs. When the frame profiler module 403 is implemented, one or more boundary coordinates corresponding to overlapping regions between wide lens frame and primary lens frames may be determined. In an embodiment of the present disclosure, a predefined main frame profile corresponding to a set of coordinates of edges of the primary lens frame inside the wide frame profile may be determined based on the determined boundary coordinates of the overlapping region. A person skilled in the art may understand that since physical configurations (predefined main frame profile and predefined wide frame profile) of a camera unit remain fixed, the initial frame positioning of the primary lens frames may also remain fixed, i.e., the position of wide lens frames. For example, as shown in Figure 7, the frame bound in box 701 may represent a wide lens frame having a predefined wide frame profile along the x-axis and y-axis. Further, the frame bound in box 703 may represent a primary lens frame having a predefined main frame profile. In the example, a top, left, right, and bottom profile of the predefined main frame profile may be as follows:

[0045] Top Profile = ((x1,y1), (x2,y1));

[0046] Left Profile = ((x1,y1), (x1,y2));

[0047] Right Profile = ((x2,y1), (x2,y2)); and

[0048] Bottom Profile = ((x1,y2), (x2,y2)).

[0049] The predefined main frame profile may be used to determine an SAFP of the processed primary lens frame by the frame profiler module 403, as described below in conjunction with Figure 8.

[0050] Figure 8is a pictorial diagram 800 illustrating the determination of SAFP in an implementation of the frame profiler module 408, according to an embodiment of the present disclosure. In an embodiment of the present disclosure, during capturing of one or more frames by the camera unit, an unwanted movement of the user device may occur, for example, due to a handshake. As a result, both the primary lens frames and wide lens frames may move with reference to an imaginary horizontal and vertical baseline. In an embodiment, the stabilization reference (corresponding to the coordinates of the predefined main frame profile) may be kept fixed and parallel to the horizontal and vertical baseline. To neutralize the effect of unwanted movement, new coordinates corresponding to a set of displaced coordinates of the processed primary lens frame may be determined. In Figure 8, the "X" axis represents the vertical baseline, and the "Y" axis represents the horizontal baseline. The set of displaced coordinates of the processed primary lens frame may represent the SAFP. For example, as shown in Figure 8, block 801 may depict an initial (or ideal) state (Initial Frame Profile FP0) where the coordinates of the processed primary lens frame are equivalent to the coordinates of the main frame profile. In an embodiment of the present disclosure, when an unwanted movement is detected, the SAFP may be determined by adjusting (e.g., either by adding or subtracting a motion value to) the coordinates of the predefined main frame profile. The motion value may be associated with a magnitude of unwanted movement. The magnitude of the unwanted movement may be obtained from the movement information associated with the unwanted movement of the user device, as may be obtained from the IMU. In an embodiment, as shown in block 803, the SAFP may be determined when an unwanted movement in an upward direction is detected. Block 803 may depict Shake Adjusted Frame Profile FP1. For example, in an exemplary case-1 803, the unwanted movement may occur by A units in the upward direction with respect to the horizontal baseline. In such a scenario, the SAFP may be determined as follows:

[0051] Top Profile = [(x1,y1+A), (x2,y1+A)];

[0052] Left Profile = [(x1,y1+A), (x1,y2+A)];

[0053] Right Profile = [(x2,y1+A), (x2,y2+A)]; and

[0054] Bottom Profile = [(x1,y2+A), (x2,y2+A)] .

[0055] In an embodiment, as shown in block 805, the SAFP may be determined when an unwanted movement in the right, as well as upward direction, is detected. Block 805 may depict Shake Adjust Frame Profile FP2. For example, in an exemplary case-2 805, the unwanted movement may occur by B units in the right direction with respect to the vertical baseline, and C units in the upward direction with respect to the horizontal baseline. In such a scenario, the SAFP may be determined as follows:

[0056] Top Profile = [(x1-B,y1+C), (x2-B,y1+C)];

[0057] Left Profile = [(x1-B,y1+C), (x1-B,y2+C)];

[0058] Right Profile = [(x2-B,y1+C), (x2-B,y2+C)]; and

[0059] Bottom Profile = [(x1-B,y2+C), (x2-B,y2+C)].

[0060] The frame profiler module 403, finally, determine ESP corresponding to wide lens frames based on the determined SAFP, as described below in conjunction with Figure 9.

[0061] Figure 9is a pictorial diagram illustrating the determination of ESP in an implementation of the frame profiler module 403, according to an embodiment of the present disclosure. Since the present disclosure uses a stitching technique for widening the primary lens frames cropped by EIS, the predefined buffer value is added or subtracted to the coordinates of SAFP to allow some overlapping with wide lens frames. Therefore, the ESP represents a set of wide frame coordinates corresponding to the predefined wide frame profile. In Figure 9, the "X" axis represents the vertical baseline, and the "Y" axis represents the horizontal baseline. The coordinates for ESP may be associated with the coordinates of a polygon obtained by adding an overlapping region to a padded region within an area defined by the SAFP. The overlapping area may correspond to the processed primary lens frames with respect to the predefined main frame profile and has a width of a predefined buffer value. Therefore, the ESP may be defined by a set of coordinates of a polygon in a wide-angle frame. The polygon may be obtained by adding to a region with padding in the processed primary lens frame with the determined SAFP, an overlapping region of the wide lens frames of width equivalent to the predefined buffer value. For example, as shown in Figure 9, in the exemplary case-1 901 described in reference to Figure 9, an ESP may be defined by a polygon PQSR. Block 901 may depict Shake Adjusted Frame Profile FP1. For example, in an exemplary case-1 903, the unwanted movement may occur by A units in the upward direction with respect to the horizontal baseline. In this case, the polygon PQSR may be obtained by adding an overlapping region having a width equivalent to a predefined buffer value *?*A, to a bottom profile, with padding, of the processed primary lens frame as defined by the determined SAFP. Therefore, in the exemplary case-1, the polygon PQSR corresponding to the ESP, may be defined as follows:

[0062]

[0063] In an example, in the exemplary case-2 903 described in reference to Figure 9, an ESP may be defined by a polygon PQSUTR. Block 903 may depict Shake Adjusted Frame Profile FP2. For example, in an exemplary case-2 903, the unwanted movement may occur by B units in the right direction with respect to the vertical baseline, and C units in the upward direction with respect to the horizontal baseline. In this case, the polygon PQSUTR may be obtained by adding overlapping regions having width equivalent to a predefined buffer value , and to a left and a bottom profile, with padding, of the processed primary lens frame as defined by the determined SAFP. Therefore, in the exemplary case-2, the polygon PQSUTR corresponding to the ESP may be defined as follows:

[0064]

[0065] According to an embodiment of the present disclosure, the determined ESP provided by frame profiler module 403 may be used to extract sub-sections from a wide-angle frame by the edge resampling module 405, as described below in conjunction with Figures 10a and 10b.

[0066] Figures 10a and 10bare schematic diagrams illustrating an exemplary implementation of the edge resampling module 405, according to an embodiment of the present disclosure. The edge resampling module 405 may be configured to extract sub-sections from wide angle frames, as depicted in block 1001a and 1001b in figures 10a and 10b respectively, based on the ESP provided by the frame profiler module 403. In an embodiment, the extracted sub-sections may correspond to one or more edges of the wide-angle frames. As discussed above, the quality of wide lens frames is lower than the quality of the primary lens frames. Some of the issues in the quality of wide lens frames with respect to the primary lens frames may include, but are not limited to, clearly visible pixelation in details at higher zoom levels, extra noise in shadow / bright areas due to the low-quality sensor of a wide lens, tonal differences in white balance and color saturation, and focus differences due to different focal length. An objective of the edge resampling module 405 is to maintain the quality of the extracted sub-sections close to the quality of the primary lens frames by addressing the problem of pixelation and extra noise in the extracted sub-sections of the wide lens frames. The remaining issues of tonal difference and focal differences are addressed by the correction module 409 described in greater detail below. The edge resampling module 405 may perform resampling to match the quality of extracted sub-sections of the wide lens frames to that of the primary lens frames. In an embodiment, the resampling may be performed using one or more predefined deep learning techniques, as depicted in block 1003a and 1003b in figures 10a and 10b respectively. The resampling of extracted sub-sections may result in sub-sections of higher quality as depicted in block 1007. The quality of the resampled sub-sections may be validated using predefined quality comparisons metrics such as peak signal to noise ratio (PSNR) and structural similarity index measure (SSIM), as depicted in blocks 1005a and 1005b. The quality validated sub-sections, as depicted in blocks 1009a and 1009b, may be used for stitching with the processed primary lens frames based on a stitching profile, by the stitching profiler module 407, as described below in conjunction with Figures 11a and 11b.

[0067] Figure 11ais a pictorial diagram illustrating the determination of the stitching profile in an implementation of the stitching profiler module 407, according to an embodiment of the present disclosure.Figure 11bis a schematic diagram illustrating an implementation of the stitching profiler module 407 based on the determined stitching profile, according to an embodiment of the present disclosure. The stitching profile may correspond to the padded region of a processed primary lens frame where stitching of the wide lens frame sub-sections may be performed. As depicted in Figure 11a, the region of a processed primary lens frame, defined by a polygon LMNOPQ may correspond to the stitching profile. The stitching profile may be used by the stitching profiler module 407 for stitching the processed primary lens frames with the quality validated sub-sections of the wide lens frames. The stitching profiler module 407 may include three components comprising histogram matcher 1103, stitcher 1105, and cropper 1109. The histogram matcher 1103 may use high quality processed primary lens frames 1101-1 and quality validated sub-sections of the wide lens frames 1101-2 as input 1101 and correct any illumination differences between 1101-1 and 1101-2. The stitcher 1105 may perform stitching of the illumination corrected processed primary lens frames and quality validated sub-sections of the wide lens frames. The stitcher 1105 may employ any predefined seamless image stitching methods for the stitching. The stitcher may result in distortion at the edges of the stitched frames. Block 1107-1 depicts distortion when the stitching is performed at the left portion of the primary lens frames, block 1107-2 depicts distortion when the stitching is performed at the top and left portions of the primary lens frames, block 1107-3 depicts distortion when the stitching is performed at left and bottom portions of the primary lens frames, and block 1107-4 depicts distortion when the stitching is performed at the bottom portion of the primary lens frames. Finally, the cropper 1109 may be used to remove the distortion by cropping the stitched frames and get the rectangular or square cropped frames as per the dimensions of the main frame profile. An objective of cropping may be to preserve an aspect ratio of the primary lens frame before EIS. Thus, an overall output 1111 by the stitching profiler module 407 may be distortion free stitched frames 1113 and the stitching profile 1115 based on which the stitching has been performed. It may be observed that the stitching has been performed using the sub-sections of wide lens frames, for which issues related to pixelation and extra noise were fixed by the edge resampling module. However, issues related to tonal and focal differences were not addressed and may be prevalent in the stitched frame. In simpler terms, there may be tonal and focal differences between the primary lens frame region and the wide lens frame region of the stitched frames. Such issues may be addressed by the correction module 409 to fix tonal and focal differences. The correction module 409 is described below in conjunction with figures 12 and 13.

[0068] Figure 12is a block diagram illustrating method 1200 of focal correction in the implementation of the correction module 409, according to an embodiment of the present disclosure. As discussed above, the correction module 409 may include the focal correction module 411. The focal correction module 411 may address the issues related to focal differences between the primary lens frame region and the wide lens frame region of the stitched frames. The focal correction module 411 may fix the focal differences based on the stitching profile 1115 obtained from the stitching profiler module 407. At step 1201, a foreign region and a local region near the coordinates of the stitching profile may be identified. The foreign region may be corresponding to the wide lens frame region in the stitched frame. The foreign region may start from stitching profile coordinates till the end of the processed primary lens frame in the stitched frame.. The local region may be of a size equivalent to the dimension of the foreign region. At step 1203, a depth map corresponding to each of the local and foreign regions may be computed. Focal differences between foreign and local regions may be determined based on a difference in the computed depth maps. At step 1205, the focal differences may be equalized by applying a lens blur filter to the foreign region using depth masking techniques. Applying lens blurring based on depth information is called depth blurring. In depth blurring, a blur filter may be applied in different amounts to each pixel in a frame, according to a depth mask. Such technique can be used to simulate a narrow depth-of-field or tilt-shift effects. The depth mask defines which pixels in a frame will be blurred, as well as the relative amount of blurring that will occur. Bright pixels in the depth mask may correspond to the highest amount of blur. The focal correction module 411 may result in focus corrected frames which may be processed further for tonal correction, as described below in conjunction with figure 13.

[0069] Figure 13is a block diagram illustrating method 1300 of tonal correction in an implementation of the correction module 409, according to an embodiment of the present disclosure. As discussed above, the correction module 409 may include the tonal correction module 413. The tonal correction module 413 may address the issues related to tonal differences between the primary lens frame region and the wide lens frame region of the focus corrected frames. The tonal correction module 413 may fix the tonal differences based on the stitching profile 1115 obtained from the stitching profiler module 407. At step 1301, the foreign region and the local region near the coordinates of the stitching profile may be identified in the focus corrected frames. At step 1303, tonal differences between foreign and the local regions may be determined and corrected by applying predefined color transfer operations for tonal correction. During the tonal correction, color transfer operations may be performed to match the color of the foreign region to the local region, to obtain tonal corrected frames. The stitched frames obtained after tonal correction may be considered as finally stabilized primary lens frames. The stabilized primary lens frames obtained after processing through the EIS widening pipeline have the same dimensions as that of the primary lens frames before EIS.

[0070] At least by virtue of the aforesaid, the present subject matter at least provides the following advantages. However, the present disclosure is not limited in this regard, and there may be other advantages without departing from the scope of the present disclosure.

[0071] Firstly, the issue of frame cropping by EIS is resolved by stitching the cropped primary lens frames with sub-sections of the wide lens frames. Secondly, since the quality of the wide lens frames is lower than the quality of primary lens frames, the quality of the stitched frames including the sub-section regions of the lower quality wide lens frames is maintained by applying the method described in the present disclosure. Therefore, the present disclosure describes a method for improving the quality of a plurality of frames captured by a primary camera lens of a multi-camera device. In an embodiment, wider FOV post EIS with lesser computation may be obtained due to small section stitching. In an embodiment, the quality, focal and tonal differences in frame edge areas may be corrected. In an embodiment, stitching may be performed between primary frame and only the small sub-sections extracted from wide frame, so it may consume less computing resources than related arts. In an embodiment, FOV widening may be performed after the EIS process. FOV widening may be applied in post processing stage, for example, after shooting. In an embodiment, difference in focus which arises due to different lens blurs may be corrected. In an embodiment, edges may be taken care by DNN based edge section resampling method. Edges may not have a lower image quality than an image quality of main region.

[0072] The method is described below in conjunction with Figures 14a and 14b.

[0073] Figures 14a and 14bare block diagrams illustrating a flow of method 1400 for improving the quality of a plurality of frames, according to an embodiment of the present disclosure. At step 1401, the method 1400 may include obtaining a plurality of processed primary lens frames by applying EIS on a plurality of primary lens frames. In an embodiment, the processed primary lens frames may be obtained by padding the plurality of primary lens frames with a width equal to a predefined EIS crop factor, applying the EIS by cropping at least one edge of the padded plurality of primary lens frames by the predefined EIS crop factor, and obtaining, based on applying of the EIS, the cropped plurality of primary lens frames corresponding to the plurality of processed primary lens frames At step 1403, the method may include determining the set of displaced coordinates based on motion information obtained from one or more motion sensors, and with respect to a set of initial coordinates corresponding to a predefined main frame profile of the primary camera lens. In an embodiment, for determining the set of displaced coordinates, the method may include obtaining a value of a displacement in the predefined main frame profile of the primary camera lens from the obtained motion information and determining the set of displaced coordinates using the obtained displacement value and the set of initial coordinates. In an embodiment, determining the set of wide frame coordinates may include determining the coordinates of a polygon obtained by adding an overlapping region to a padded region within an area defined by the determined set of displaced coordinates, wherein the overlapping area corresponds to the processed primary lens frames with respect to the predefined main frame profile and has a width of a predefined buffer value.

[0074] At step 1405, the method may include determining a set of wide frame coordinates, corresponding to at least one edge of a predefined wide frame profile associated with a wide-angle camera lens of the multi-camera device, based on a set of displaced coordinates of the plurality of processed primary lens frames. At step 1407, the method may include extracting one or more edge sections from a plurality of wide lens frames, captured from the wide-angle camera lens, based on the determined set of coordinates. At step 1409, the method may include resampling the one or more extracted edge sections based on one or more predefined deep neural network techniques. At step 1411, the method may include obtaining one or more quality validated edge sections based on a comparison of a quality parameter of the one or more resampled edge sections with a corresponding quality parameter of the processed primary lens frames.

[0075] At step 1413, the method may include stitching processed primary lens frames with one or more extracted edge sections for obtaining one or more stitched frames. In an embodiment, the stitching may include determining a set of stitching coordinates associated with the plurality of processed primary lens frames; stitching, based on the determined set of stitching coordinates, the plurality of processed primary lens frames with the one or more quality validated edge sections using a predefined image stitching process to obtain one or more intermediate stitched frames with at least one distorted edge, and obtaining the one or more stitched frames by cropping the at least one distorted edge of the one or more intermediate stitched frames.

[0076] At step 1415, the method may include applying a focal correction operation and a tonal correction operation on the one or more stitched frames to obtain a plurality of stabilized frames corresponding to the plurality of processed primary lens frames. In an embodiment, for applying the focal correction operation, the method may include determining a foreign region corresponding to the set of stitching coordinates in the one or more stitched frames, and a local region, of equivalent width of the foreign region, corresponding to the plurality of processed primary lens frames in the one or more stitched frames. In an embodiment, the method may include obtaining, corresponding to each of the local region and the foreign region, a depth map. In an embodiment, the method may include applying a lens blurring process to the foreign region of the one or more stitched frames based on a difference in the depth map of the local region and the foreign region. In an embodiment, for applying the tonal correction operation, the method may include determining a foreign region corresponding to set of stitching coordinates in one or more focal corrected frames. In an embodiment, the method may include determining a local region, of equivalent width of the foreign region, corresponding to the processed primary lens frames in the one or more focal corrected frames. In an embodiment, the method may include performing a predefined color transfer operation on the foreign region, based on a difference in the color tone of the local region and the foreign region. The method may be for obtain one or more tonal corrected frames.

[0077] According to an embodiment of the disclosure, the obtaining the plurality of processed primary lens frames may include padding the plurality of primary lens frames with a width equal to a predefined EIS crop factor. According to an embodiment of the disclosure, the obtaining the plurality of processed primary lens frames may include applying the EIS by cropping at least one edge of the padded plurality of primary lens frames by the predefined EIS crop factor. According to an embodiment of the disclosure, the obtaining the plurality of processed primary lens frames may include obtaining, based on applying of the EIS, the cropped plurality of primary lens frames corresponding to the plurality of processed primary lens frames.

[0078] According to an embodiment of the disclosure, the method may include determining the set of displaced coordinates based on motion information obtained from one or more motion sensors, with respect to a set of initial coordinates corresponding to a predefined main frame profile of the primary camera lens.

[0079] According to an embodiment of the disclosure, the determining the set of displaced coordinates may include obtaining a value of a displacement in the predefined main frame profile of the primary camera lens from the obtained motion information. The determining the set of displaced coordinates may include determining the set of displaced coordinates using the obtained displacement value and the set of initial coordinates.

[0080] According to an embodiment of the disclosure, the determining the set of wide frame coordinates may include determining coordinates of a polygon obtained by adding an overlapping region to a padded region within an area defined by the determined set of displaced coordinates, wherein the overlapping area corresponds to the processed primary lens frames with respect to the predefined main frame profile and has a width of a predefined buffer value.

[0081] According to an embodiment of the disclosure, the method may include resampling the one or more extracted edge sections based on one or more predefined deep neural network techniques. The method may include obtaining one or more quality validated edge sections based on a comparison of a quality parameter of the one or more resampled edge sections with a corresponding quality parameter of the processed primary lens frames.

[0082] According to an embodiment of the disclosure, the stitching may include determining a set of stitching coordinates associated with the plurality of processed primary lens frames. The stitching may include stitching, based on the determined set of stitching coordinates, the plurality of processed primary lens frames with the one or more quality validated edge sections using a predefined image stitching process. The stitching may be for obtaining one or more intermediate stitched frames with at least one distorted edge. The stitching may be for obtaining the one or more stitched frames by cropping the at least one distorted edge of the one or more intermediate stitched frames.

[0083] According to an embodiment of the disclosure, the applying the focal correction operation may include determining a foreign region corresponding to the set of stitching coordinates in the one or more stitched frames, and a local region, of an equivalent width of the foreign region, corresponding to the plurality of processed primary lens frames in the one or more stitched frames. The applying the focal correction operation may include obtaining, corresponding to each of the local region and the foreign region, a depth map. The applying the focal correction operation may include applying a lens blurring process to the foreign region of the one or more stitched frames based on a difference in the depth map of the local region and the foreign region.

[0084] According to an embodiment of the disclosure, the applying the tonal correction operation may include determining a foreign region corresponding to set of stitching coordinates in one or more focal corrected frames, and a local region, of an equivalent width of the foreign region, corresponding to the processed primary lens frames in the one or more focal corrected frames. The applying the tonal correction operation may include performing a predefined color transfer operation on the foreign region, based on a difference in the color tone of the local region and the foreign region. The performing a predefined color transfer operation on the foreign region, based on a difference in the color tone of the local region and the foreign region may be to obtain one or more tonal corrected frames.

[0085] According to an embodiment of the disclosure, a system for improving quality of a plurality of frames captured by a primary camera lens of a multi-camera device may include a stabilization unit configured to obtain a plurality of processed primary lens frames by applying electronic image stabilization (EIS) on the plurality of primary lens frames. The system may include a determination unit configured to determine a set of wide frame coordinates, corresponding to at least one edge of a predefined wide frame profile associated with a wide-angle camera lens of the multi-camera device, based on a set of displaced coordinates of the plurality of processed primary lens frames. The system may include an extraction unit configured to extract one or more edge sections from a plurality of wide lens frames, captured from the wide-angle camera lens, based on the determined set of coordinates. The system may include a stitching unit configured to stitch the processed primary lens frames with the one or more extracted edge sections for obtaining one or more stitched frames. The system may include a correction unit configured to apply a focal correction operation and a tonal correction operation on the one or more stitched frames to obtain a plurality of stabilized frames corresponding to the plurality of processed primary lens frames.

[0086] According to an embodiment of the disclosure, the stabilization unit may be configured to pad the plurality of primary lens frames with a width equal to a predefined EIS crop factor. The stabilization unit may be configured to apply the EIS by cropping at least one edge of the padded plurality of primary lens frames by the predefined EIS crop factor. The stabilization unit may be configured to obtain, based on applying of the EIS, the cropped plurality of primary lens frames corresponding to the plurality of processed primary lens frames.

[0087] According to an embodiment of the disclosure, the determination unit may be configured to determine the set of displaced coordinates based on motion information obtained from one or more motion sensors, and with respect to a set of initial coordinates corresponding to a predefined main frame profile of the primary camera lens.

[0088] According to an embodiment of the disclosure, the determination unit may be configured to obtain a value of a displacement in the predefined main frame profile of the primary camera lens from the obtained motion information. According to an embodiment of the disclosure, the determination unit may be configured to determine the set of displaced coordinates using the obtained displacement value and the set of initial coordinates.

[0089] According to an embodiment of the disclosure, the determination of the set of wide frame coordinates may include determining coordinates of a polygon obtained by adding an overlapping region to a padded region within an area defined by the determined set of displaced coordinates, wherein the overlapping area corresponds to　the processed primary lens frames with respect to the predefined main frame profile and has a width of a predefined buffer value.

[0090] According to an embodiment of the disclosure, the system may resample the one or more extracted edge sections based on one or more predefined deep neural network techniques. The system may obtain one or more quality validated edge sections based on a comparison of a quality parameter of the one or more resampled edge sections with a corresponding quality parameter of the processed primary lens frames.

[0091] According to an embodiment of the disclosure, the stitching unit may be configured to determine a set of stitching coordinates associated with the plurality of processed primary lens frames. According to an embodiment of the disclosure, the stitching unit may be configured to stitch, based on the determined set of stitching coordinates, the plurality of processed primary lens frames with the one or more quality validated edge sections using a predefined image stitching process to obtain one or more intermediate stitched frames with at least one distorted edge. According to an embodiment of the disclosure, the stitching unit may be configured to obtain the one or more stitched frames by cropping the at least one distorted edge of the one or more intermediate stitched frames.

[0092] According to an embodiment of the disclosure, the correction unit may be configured to determine a foreign region corresponding to the set of stitching coordinates in the one or more stitched frames, and a local region, of an equivalent width of the foreign region, corresponding to the plurality of processed primary lens frames in the one or more stitched frames. According to an embodiment of the disclosure, the correction unit may be configured to obtain, corresponding to each of the local region and the foreign region, a depth map. According to an embodiment of the disclosure, the correction unit may be configured to apply a lens blurring process to the foreign region of the one or more stitched frames based on a difference in the depth map of the local region and the foreign region.

[0093] According to an embodiment of the disclosure, the correction unit may be configured to determine a foreign region corresponding to set of stitching coordinates in one or more focal corrected frames, and a local region, of an equivalent width of the foreign region, corresponding to the processed primary lens frames in the one or more focal corrected frames. According to an embodiment of the disclosure, the correction unit may be configured to perform a predefined color transfer operation on the foreign region, based on a difference in the color tone of the local region and the foreign region, to obtain one or more tonal corrected frames.

[0094] According to an embodiment of the disclosure, the at least one processor may be configured to execute the instructions to pad the plurality of primary lens frames with a width equal to a predefined EIS crop factor. The at least one processor may be configured to execute the instructions to apply the EIS by cropping at least one edge of the padded plurality of primary lens frames by the predefined EIS crop factor. The at least one processor may be configured to execute the instructions to obtain, based on applying of the EIS, the cropped plurality of primary lens frames corresponding to the plurality of processed primary lens frames.

[0095] According to an embodiment of the disclosure, the at least one processor may be configured to execute the instructions to determine the set of displaced coordinates based on motion information obtained from one or more motion sensors, and with respect to a set of initial coordinates corresponding to a predefined main frame profile of the primary camera lens.

[0096] According to an embodiment of the disclosure, the at least one processor may be configured to execute the instructions to obtain a value of a displacement in the predefined main frame profile of the primary camera lens from the obtained motion information.

[0097] According to an embodiment of the disclosure, the at least one processor may be configured to execute the instructions to determine the set of displaced coordinates using the obtained displacement value and the set of initial coordinates.

[0098] According to an embodiment of the disclosure, the at least one processor may be configured to execute the instructions to resample the one or more extracted edge sections based on one or more predefined deep neural network techniques.

[0099] According to an embodiment of the disclosure, the at least one processor may be configured to execute the instructions to obtain one or more quality validated edge sections based on a comparison of a quality parameter of the one or more resampled edge sections with a corresponding quality parameter of the processed primary lens frames.

[0100] According to an embodiment of the disclosure, the at least one processor may be configured to execute the instructions to determine a set of stitching coordinates associated with the plurality of processed primary lens frames.

[0101] According to an embodiment of the disclosure, the at least one processor may be configured to execute the instructions to stitch, based on the determined set of stitching coordinates, the plurality of processed primary lens frames with the one or more quality validated edge sections using a predefined image stitching process to obtain one or more intermediate stitched frames with at least one distorted edge.

[0102] According to an embodiment of the disclosure, the at least one processor may be configured to execute the instructions to obtain the one or more stitched frames by cropping the at least one distorted edge of the one or more intermediate stitched frames.

[0103] While specific language has been used to describe the present subject matter, any limitations arising on account thereto, are not intended. As would be apparent to a person in the art, various working modifications may be made to the method in order to implement the inventive concept as taught herein. The drawings and the foregoing description give examples of embodiments. Those skilled in the art will appreciate that one or more of the described elements may well be combined into a single functional element. Alternatively, certain elements may be split into multiple functional elements. Elements from one embodiment may be added to another embodiment.

Claims

1.A method for improving quality of a plurality of frames captured by a primary camera lens of a multi-camera device, the method comprising:obtaining a plurality of processed primary lens frames by applying electronic image stabilization (EIS) on the plurality of primary lens frames;determining a set of wide frame coordinates, corresponding to at least one edge of a predefined wide frame profile associated with a wide-angle camera lens of the multi-camera device, based on a set of displaced coordinates of the plurality of processed primary lens frames;extracting one or more edge sections from a plurality of wide lens frames, captured from the wide-angle camera lens, based on the determined set of coordinates;stitching the processed primary lens frames with the one or more extracted edge sections for obtaining one or more stitched frames;applying a focal correction operation and a tonal correction operation on the one or more stitched frames to obtain a plurality of stabilized frames corresponding to the plurality of processed primary lens frames.2.The method of claim 1, wherein the obtaining the plurality of processed primary lens frames comprises:padding the plurality of primary lens frames with a width equal to a predefined EIS crop factor;applying the EIS by cropping at least one edge of the padded plurality of primary lens frames by the predefined EIS crop factor; andobtaining, based on applying of the EIS, the cropped plurality of primary lens frames corresponding to the plurality of processed primary lens frames.3.The method any one of claims 1 to 2 further comprises:determining the set of displaced coordinates based on motion information obtained from one or more motion sensors, with respect to a set of initial coordinates corresponding to a predefined main frame profile of the primary camera lens.4.The method any one of claims 1 to 3, wherein the determining the set of displaced coordinates, the method comprises:obtaining a value of a displacement in the predefined main frame profile of the primary camera lens from the obtained motion information; anddetermining the set of displaced coordinates using the obtained displacement value and the set of initial coordinates.5.The method any one of claims 1 to 4 further comprises:resampling the one or more extracted edge sections based on one or more predefined deep neural network techniques; andobtaining one or more quality validated edge sections based on a comparison of a quality parameter of the one or more resampled edge sections with a corresponding quality parameter of the processed primary lens frames.6.The method any one of claims 1 to 5, wherein the stitching comprises:determining a set of stitching coordinates associated with the plurality of processed primary lens frames;stitching, based on the determined set of stitching coordinates, the plurality of processed primary lens frames with the one or more quality validated edge sections using a predefined image stitching process; wherein the stitching is for:obtaining one or more intermediate stitched frames with at least one distorted edge; andobtaining the one or more stitched frames by cropping the at least one distorted edge of the one or more intermediate stitched frames.7.The method any one of claims 1 to 6, wherein the applying the focal correction operation comprises:determining a foreign region corresponding to the set of stitching coordinates in the one or more stitched frames, and a local region, of an equivalent width of the foreign region, corresponding to the plurality of processed primary lens frames in the one or more stitched frames;obtaining, corresponding to each of the local region and the foreign region, a depth map; andapplying a lens blurring process to the foreign region of the one or more stitched frames based on a difference in the depth map of the local region and the foreign region.8.The method any one of claims 1 to 7, wherein the applying the tonal correction operation comprises:determining a foreign region corresponding to set of stitching coordinates in one or more focal corrected frames, and a local region, of an equivalent width of the foreign region, corresponding to the processed primary lens frames in the one or more focal corrected frames; andperforming a predefined color transfer operation on the foreign region, based on a difference in the color tone of the local region and the foreign region.9.A multi-camera device comprising:a memory;at least one processor coupled to the memory, wherein the at least one processor is configured to:obtain a plurality of processed primary lens frames by applying electronic image stabilization (EIS) on the plurality of primary lens frames;determine a set of wide frame coordinates, corresponding to at least one edge of a predefined wide frame profile associated with a wide-angle camera lens of the multi-camera device, based on a set of displaced coordinates of the plurality of processed primary lens frames;extract one or more edge sections from a plurality of wide lens frames, captured from the wide-angle camera lens, based on the determined set of coordinates;stitch the processed primary lens frames with the one or more extracted edge sections for obtaining one or more stitched frames; and apply a focal correction operation and a tonal correction operation on the one or more stitched frames to obtain a plurality of stabilized frames corresponding to the plurality of processed primary lens frames.10.The multi-camera device of claim 9, wherein the at least one processor is further configured to execute the instructions to: pad the plurality of primary lens frames with a width equal to a predefined EIS crop factor;apply the EIS by cropping at least one edge of the padded plurality of primary lens frames by the predefined EIS crop factor; andobtain, based on applying of the EIS, the cropped plurality of primary lens frames corresponding to the plurality of processed primary lens frames.11.The multi-camera device any one of claims 9 to 10, wherein the at least one processor is further configured to execute the instructions to:determine the set of displaced coordinates based on motion information obtained from one or more motion sensors, and with respect to a set of initial coordinates corresponding to a predefined main frame profile of the primary camera lens.12.The multi-camera device any one of claims 9 to 11, wherein the at least one processor is further configured to execute the instructions to:obtain a value of a displacement in the predefined main frame profile of the primary camera lens from the obtained motion information; anddetermine the set of displaced coordinates using the obtained displacement value and the set of initial coordinates.13.The multi-camera device any one of claims 9 to 12, wherein the at least one processor is further configured to execute the instructions to:resample the one or more extracted edge sections based on one or more predefined deep neural network techniques; andobtain one or more quality validated edge sections based on a comparison of a quality parameter of the one or more resampled edge sections with a corresponding quality parameter of the processed primary lens frames.14.The multi-camera device any one of claims 9 to 13, wherein the at least one processor is further configured to execute the instructions to:determine a set of stitching coordinates associated with the plurality of processed primary lens frames;stitch, based on the determined set of stitching coordinates, the plurality of processed primary lens frames with the one or more quality validated edge sections using a predefined image stitching process to obtain one or more intermediate stitched frames with at least one distorted edge; andobtain the one or more stitched frames by cropping the at least one distorted edge of the one or more intermediate stitched frames.15.A computer-readable storage medium storing instructions, wherein the instructions, when executed by at least one processor, cause the at least one processor to perform the method of any one of claims 1 to 8.

Images