Method of processing image and elecronic apparatus performing the same

Abstract

According to one embodiment disclosed in this document, an electronic device for processing images controls a camera module included in the electronic device based on camera setting information to acquire a plurality of short exposure images during a preset time interval, aligns at least some of the short exposure images based on shaking information of the electronic device while acquiring the plurality of short exposure images, and synthesizes the aligned short exposure images to acquire a long exposure image corresponding to a preset time interval.

Description

Technology Field

[0001] The embodiments disclosed in this document relate to a method for processing images and an electronic device for performing the same. Background Technology

[0002] With the development of information technology (IT), various types of electronic devices, such as smartphones and tablet personal computers (tablet PCs), are becoming widely distributed. Additionally, these electronic devices may include one or more camera modules. The camera module can be implemented as a digital camera that utilizes an image sensor—rather than a traditional film camera—to convert light into electrical image signals and store them as image data.

[0003] Meanwhile, when recording video using a camera module included in an electronic device, shaking of the device can be reflected in the recorded video, which may degrade video quality. Previously, a method of fixing the device with a tripod to prevent shaking was used; however, this can cause inconvenience to the user as it requires carrying the tripod along with the device for video recording. The problem to be solved

[0004] The present disclosure aims to provide a method for an electronic device to process an image and an electronic device for such a method. More specifically, an electronic device according to one embodiment of the present disclosure aims to provide an image processing technique for minimizing the reflection of shaking of the electronic device in the captured image when capturing an image. means of solving the problem

[0005] According to one embodiment, a method for an electronic device to process an image may include: a step of acquiring a plurality of short exposure images during a preset time interval by controlling a camera module included in the electronic device based on camera setting information; a step of aligning at least some of the short exposure images based on shaking information of the electronic device while acquiring the plurality of short exposure images; and a step of synthesizing the aligned short exposure images to acquire a long exposure image corresponding to a preset time interval.

[0006] According to one embodiment, an electronic device for processing images includes a memory for storing one or more instructions; a camera module; and at least one processor for executing one or more instructions stored in the memory. The at least one processor controls the camera module based on camera setting information to acquire a plurality of short exposure images during a preset time interval, aligns at least some of the short exposure images based on shaking information of the electronic device during the acquisition of the plurality of short exposure images, and synthesizes the aligned short exposure images to acquire a long exposure image corresponding to a preset time interval.

[0007] A computer program product comprising a recording medium storing a program that enables an electronic device according to one embodiment to perform a method of processing images may store a program that controls a camera module included in the electronic device based on camera setting information to perform an operation of acquiring a plurality of short exposure images during a preset time interval, an operation of aligning at least some of the short exposure images based on shaking information of the electronic device while acquiring the plurality of short exposure images, and an operation of synthesizing the aligned short exposure images to acquire a long exposure image corresponding to a preset time interval. Effects of the invention

[0008] According to the embodiments disclosed in this document, by synthesizing multiple short-exposure images to obtain a long-exposure image, the degradation of image quality caused by shaking of the electronic device during image capture can be minimized. In addition, various effects that can be identified directly or indirectly through this document may be provided. Brief explanation of the drawing

[0009] FIG. 1 is a conceptual diagram illustrating a method for an electronic device according to one embodiment to acquire a long-exposure image. FIG. 2 is a flowchart illustrating a method for an electronic device to process an image according to one embodiment. FIG. 3 is a diagram illustrating a method for an electronic device according to one embodiment to determine camera setting information using a first artificial intelligence model. FIG. 4 is a diagram illustrating a method for an electronic device according to one embodiment to acquire a long-exposure image using a second artificial intelligence model based on a plurality of short-exposure images. FIG. 5 is a flowchart illustrating a method for an electronic device according to one embodiment to acquire a long-exposure image based on some of a plurality of short-exposure images. FIG. 6 is a diagram illustrating a method for an electronic device according to one embodiment to select at least some of a plurality of single-exposure images using a user's heart rate information. FIG. 7 is a diagram illustrating a method for an electronic device according to one embodiment to align a plurality of single-exposure images using user's breathing information. FIG. 8 is a diagram illustrating a method for an electronic device according to one embodiment to acquire a long exposure image based on a plurality of short exposure images using the user's breathing information. FIG. 9 is a diagram illustrating a method for an electronic device according to one embodiment to acquire a long-exposure image based on a plurality of short-exposure images and shake information of the electronic device. FIG. 10 is a block diagram of an electronic device according to one embodiment. FIG. 11 is a block diagram of an electronic device in a network environment according to various embodiments. FIG. 12 is a diagram illustrating a method for an electronic device according to one embodiment to acquire a long-exposure image from a server. FIG. 13 is a flowchart illustrating a method for an electronic device and server to acquire a long-exposure image according to one embodiment. Specific details for implementing the invention

[0010] Prior to a specific description of the invention, the terms used in the present disclosure below may be defined or understood as follows.

[0011] In the present disclosure, a "short exposure image" may be an image captured with a shutter speed of less than a preset value (e.g., 1 / S), and a "long exposure image" may be an image captured with a shutter speed of greater than or equal to a preset value (e.g., 1 / L). Here, the preset values ​​may be determined by an electronic device or server based on a shooting environment or shooting mode.

[0012] When a long shutter speed is set (e.g., 1 / L or longer) to acquire a long-exposure image from an electronic device, the time the camera shutter remains open increases. In this case, if the position of the electronic device is not fixed, such as when the user holds the electronic device to photograph an object, the shaking of the electronic device may be reflected in the image, which may degrade the image quality.

[0013] In this disclosure, a method for acquiring a long-exposure image by synthesizing a plurality of short-exposure images is provided to improve the problem of shaking of an electronic device that occurs when acquiring a long-exposure image. Hereinafter, with reference to FIGS. 1 to 13, a method for acquiring a long-exposure image by synthesizing a plurality of short-exposure images according to this disclosure will be described in detail.

[0014] FIG. 1 is a conceptual diagram illustrating a method for an electronic device according to one embodiment to acquire a long-exposure image.

[0015] The process of acquiring a long-exposure image of an electronic device (100) may include a plurality of operations, and each operation may be represented as a block. According to various embodiments, the process of acquiring a long-exposure image of an electronic device is not limited to that shown in FIG. 1. For example, the process of acquiring a long-exposure image (100) may include additional operations not shown in FIG. 1, or any one of the operations shown in FIG. 1 may be omitted.

[0016] A camera module (110) may include one or more lenses and image sensors, and may receive external light generated from or reflected from an object under the control of an electronic device. An electronic device according to one embodiment may acquire an image by controlling the camera module (110) based on camera setting information. The camera setting information may include parameters that are subject to control when capturing an image using the camera module, for example, shutter speed, ISO information, number of shots, etc. However, this is merely an example, and the camera setting information is not limited to the example described above. Shutter speed may refer to the time when the camera shutter is open, and ISO information may be information indicating the sensitivity of the image sensor to light. For example, the electronic device may perform a single exposure image acquisition operation (120) during a preset time interval by controlling the camera module (110) based on a shutter speed of 1 / 2S and ISO information A.

[0017] When a plurality of single-exposure images are acquired as a result of performing a single-exposure image acquisition operation (120), the electronic device may perform a single-exposure image matching operation (130) on the acquired plurality of single-exposure images. The single-exposure image matching operation (130) may include an operation of selecting at least some of the plurality of single-exposure images based on shaking information of the electronic device while the plurality of single-exposure images are being acquired. The shaking information of the electronic device may include at least one of the user's heart rate information, the user's breathing information, or the three-dimensional acceleration information of the electronic device. According to another example, the single-exposure image matching operation (130) may include an operation of acquiring a single-exposure image to replace the unselected single-exposure image by performing interpolation on the selected single-exposure images. According to yet another example, the single-exposure image matching operation (130) may include an operation of correcting the plurality of single-exposure images based on shaking information of the electronic device or motion information between the plurality of single-exposure images. According to another example, the single-exposure image matching operation (130) may include an operation of selecting at least some of a plurality of single-exposure images and correcting at least some of the selected images. Various embodiments of the single-exposure image matching operation (130) will be described in detail later with reference to FIGS. 4 to 9.

[0018] The electronic device can acquire long-exposure images through a short-exposure image synthesis operation (140). The electronic device according to one embodiment can synthesize the matched short-exposure images by acquiring one of an average value, a maximum value, a minimum value, or a mode for corresponding pixels among the pixels constituting the matched short-exposure images, depending on the setting. For example, for a plurality of pixels consisting of N columns and M rows constituting each of the matched short-exposure images, one of an average value, a maximum value, a minimum value, or a mode of the pixel values ​​of the pixels at position (n, m) can be acquired. The pixel value can be understood as a color value corresponding to each pixel. The pixel value may include color values ​​for a plurality of colors determined according to the color filter array used.

[0019] The electronic device can acquire a long exposure image by calculating one of the average, maximum, minimum, or mode of pixel values ​​according to the setting for each pixel constituting the matched short exposure images. The electronic device can perform a storage operation (150) of the acquired long exposure image.

[0020] The electronic device according to the present disclosure can effectively solve the problem of image quality degradation caused by shaking of the electronic device by acquiring a long-exposure image based on a plurality of short-exposure images acquired during a preset time interval as described above.

[0021] FIG. 2 is a flowchart illustrating a method for an electronic device to process an image according to one embodiment.

[0022] In step S210, the electronic device can acquire multiple single-exposure images during a preset time interval by controlling a camera module included in the electronic device based on camera setting information.

[0023] An electronic device according to one embodiment can obtain camera setting information as output data by inputting information regarding a shooting mode and a shooting environment as input data to a first artificial intelligence model. In the present disclosure, input data may be understood as data input to at least one artificial intelligence model, and output data may be understood as data output from at least one artificial intelligence model corresponding to the input data. A shooting mode may be determined according to the characteristics of an object that the user intends to photograph or the type of image that the user intends to acquire, and for example, portrait mode, food mode, night mode, slow motion mode, panorama mode, pro mode, etc., may be included in the shooting mode. Pro mode refers to a mode in which the user can directly set shutter speed, ISO information, etc.

[0024] For example, if the shutter speed value output from the first artificial intelligence model is 1 / 2S and the number of shots is W, the electronic device can perform W shots at a shutter speed of 1 / 2S to obtain W single-exposure images.

[0025] According to another example, even if the user sets a long shutter speed as input data for the first artificial intelligence model, the shaking of the electronic device or the amount of ambient light is taken into account, and a short shutter speed value may be obtained as output data from the first artificial intelligence model.

[0026] According to another embodiment, the electronic device may acquire camera setting information based on a previously acquired database. The database may store camera setting information corresponding to a shooting mode or shooting environment. Accordingly, the electronic device may acquire camera setting information by comparing the current shooting mode and shooting environment with the information stored in the database. The electronic device may acquire a plurality of single-exposure images based on the acquired camera setting information.

[0027] In step S220, the electronic device can align at least some of the single-exposure images based on the shaking information of the electronic device while acquiring the multiple single-exposure images.

[0028] An electronic device according to one embodiment can acquire shaking information of the electronic device. For example, the electronic device may acquire one or more of the user's heart rate information, the user's respiration information, or the electronic device's three-dimensional acceleration information as shaking information while a plurality of short-exposure images are acquired. The shaking information may be used to select at least some of the plurality of short-exposure images used for long-exposure acquisition by the electronic device. For example, the electronic device may select a short-exposure image acquired in a section where the change in the user's heart rate or respiration is below a certain value. According to another example, the electronic device may select a short-exposure image acquired in a section where the electronic device's three-dimensional acceleration value is below a certain value. According to yet another example, the electronic device may select at least some of the plurality of short-exposure images using an artificial intelligence model based on the shaking information. A method for the electronic device to select at least some of the plurality of short-exposure images using shaking information will be described in detail later with reference to FIGS. 5 to 9.

[0029] The electronic device can correct selected single-exposure images based on shake information. Additionally, according to another example, the electronic device may perform interpolation on the selected single-exposure images to obtain single-exposure images to replace unselected single-exposure images.

[0030] However, this is merely an example, and the electronic device can obtain aligned single-exposure images by inputting a homography matrix between multiple single-exposure images into a second artificial intelligence model. A homography matrix is ​​information representing the transformation relationship between projected corresponding points when one plane is projected onto another plane, and in this disclosure, it can be used to represent the transformation relationship between a single-exposure image and another single-exposure image. This will be described later with reference to FIG. 4.

[0031] In step S230, the electronic device can synthesize the matched short-exposure images to obtain a long-exposure image corresponding to a preset time interval.

[0032] An electronic device according to one embodiment can acquire a long exposure image by calculating one of an average value, a maximum value, a minimum value, or a mode of pixel values ​​according to a setting for each of the pixels constituting the matched single exposure images. According to another embodiment, an electronic device can acquire a long exposure image by dividing each of the matched single exposure images into a plurality of regions and calculating one of the aforementioned average value, a maximum value, a minimum value, or a mode of pixel values ​​according to the characteristics of each region.

[0033] FIG. 3 is a diagram illustrating a method for an electronic device according to one embodiment to determine camera setting information using a first artificial intelligence model.

[0034] An electronic device (300) according to one embodiment can obtain camera setting information by inputting information regarding a shooting environment and a shooting mode into a first artificial intelligence model (350). The first artificial intelligence model (350) may include at least one layer, and each layer may include at least one weight. In one embodiment, the weight may be set to a preset initial value. According to various embodiments, the electronic device can train the first artificial intelligence model (350) by changing the value of the weight using input data and target data. For example, the electronic device may change the value of at least one weight included in the at least one layer so that the camera setting information obtained as a result of inputting information regarding a shooting environment and a shooting mode into the first artificial intelligence model (350) becomes similar to the target shutter speed or target ISO information.

[0035] Referring to FIG. 3, the electronic device can input information regarding the amount of light obtained through the illuminance sensor (310) as information regarding the shooting environment into the first artificial intelligence model (350). Additionally, the electronic device can input this into the first artificial intelligence model (350) upon receiving user input to set the shooting mode to Pro mode. In the case of Pro mode, since it is configured to allow the user to additionally input information such as exposure time, information regarding the exposure time set by the user can also be input into the first artificial intelligence model (350).

[0036] The electronic device (300) can obtain shutter speed and ISO information as output data of the first artificial intelligence model (350). However, this is merely an example, and the output data of the first artificial intelligence model (350) is not limited to the example described above. According to other examples, the number of shots can be obtained as output data of the first artificial intelligence model (350).

[0037] The electronic device (300) can acquire multiple single-exposure images during an exposure time set by the user based on the acquired shutter speed and ISO information.

[0038] FIG. 4 is a diagram illustrating a method for an electronic device according to one embodiment to acquire a long-exposure image using a second artificial intelligence model based on a plurality of short-exposure images.

[0039] Referring to FIG. 4, the electronic device can obtain homography matrices between a plurality of single-exposure images (410). The homography matrix can represent the transformation relationship between the plurality of single-exposure images. For example, the electronic device obtains a homography matrix between a first single-exposure image and a second single-exposure image Homography matrix between the second and third exposure images You can obtain.

[0040] The electronic device can obtain matched single-exposure images (430) by inputting the homography matrices of a plurality of single-exposure images (410) into a second artificial intelligence model (420). According to one example, the weight of each of at least one layer included in the second artificial intelligence model (420) may be set to a value obtained as a result of training such that the degree of match between the single-exposure images obtained as a result of inputting the homography matrices into the second artificial intelligence model (420) is greater than or equal to a threshold value. The degree of match may be calculated, for example, based on whether a fixed object (e.g., a building) included in the single-exposure images matches, but this is merely an example and the method of calculating the degree of match is not limited to the example described above.

[0041] The electronic device can obtain a long exposure image (440) by synthesizing the matched short exposure images (430) output from the second artificial intelligence model (420).

[0042] FIG. 5 is a flowchart illustrating a method for an electronic device according to one embodiment to acquire a long-exposure image based on some of a plurality of short-exposure images.

[0043] In step S510, the electronic device can acquire multiple single-exposure images during a preset time interval by controlling a camera module included in the electronic device based on camera setting information. For example, the electronic device can acquire multiple single-exposure images for N seconds with a shutter speed of 1 / 2S according to the camera setting information.

[0044] In step S520, the electronic device can identify shaking information of the electronic device while acquiring a plurality of single-exposure images.

[0045] An electronic device according to one embodiment may include one or more sensors. For example, the electronic device may include a biosensor and an accelerometer. An example of a biosensor may include an SpO2 (saturation of percutaneous oxygen) sensor capable of measuring heart rate, but this is merely an example and examples of biosensors are not limited to those described above. The electronic device may identify shaking information of the electronic device based on bio-information obtained through the biosensor. For example, based on heart rate information obtained through the SpO2 sensor, the electronic device may identify a section with a large change in heart rate as a section where the electronic device has shaken. According to another example, based on three-dimensional acceleration information obtained through the accelerometer, the electronic device may identify a section with a large acceleration value as a section where the electronic device has shaken.

[0046] Meanwhile, the electronic device may also acquire the user's biometric information or acceleration information from an external device. For example, if the electronic device is a smartphone, it may acquire acceleration information of the wearable device or the user's biometric information while acquiring multiple single-exposure images from the wearable device, which is an external device. Based on the information acquired from the wearable device, the electronic device may identify the section where the electronic device has shaken.

[0047] In step S530, the electronic device can select at least some of the multiple single-exposure images based on the shaking information of the electronic device.

[0048] An electronic device according to one embodiment may select single-exposure images obtained in a section where the electronic device is not shaken among a plurality of single-exposure images. For example, if the shaking information includes the user's heart rate information, the electronic device may select single-exposure images obtained in a section where the change in the user's heart rate is below a threshold. According to another example, if the shaking information includes the user's respiration information, the electronic device may select single-exposure images obtained in a section where the change in the user's respiration is below a threshold. According to yet another example, if the shaking information includes the acceleration information of the electronic device, the electronic device may select single-exposure images obtained in a section where the 3D acceleration value is below a threshold.

[0049] In step S540, the electronic device can acquire a long exposure image based on the selected short exposure images.

[0050] An electronic device according to one embodiment can obtain a long exposure image by synthesizing selected short exposure images. An electronic device according to another embodiment may obtain a long exposure image by correcting selected short exposure images and synthesizing the corrected short exposure images.

[0051] FIG. 6 is a diagram illustrating a method for an electronic device according to one embodiment to select at least some of a plurality of single-exposure images using a user's heart rate information.

[0052] Referring to FIG. 6, the electronic device can acquire a user's heart rate information while acquiring a plurality of single-exposure images (610a to 610l). For example, the electronic device can monitor the user's heart rate while the plurality of single-exposure images are acquired using an SpO2 sensor provided in the electronic device. According to another example, the electronic device may receive the user's heart rate information monitored from a wearable device worn by the user.

[0053] The electronic device can identify intervals (hereinafter, selected intervals) in which the change in heart rate is less than a threshold a, based on the user's heart rate information. The electronic device can select single-exposure images (610b, 610c, 610f, 610g, 610i, 610j) acquired in the selected intervals among a plurality of single-exposure images (610a to 610l).

[0054] The electronic device can acquire a long exposure image (620) based on selected short exposure images (610b, 610c, 610f, 610g, 610i, 610j).

[0055] An electronic device according to one embodiment can obtain a long exposure image by calculating one of the average, maximum, minimum, or mode of pixel values ​​according to a setting for each of the pixels constituting selected short exposure images (610b, 610c, 610f, 610g, 610i, 610j).

[0056] An electronic device according to another embodiment can perform correction on selected short-exposure images (610b, 610c, 610f, 610g, 610i, 610j) based on shaking information of the electronic device identified based on heart rate information. For example, the electronic device can identify shaking information of the electronic device by comparing the degree of shaking of the electronic device corresponding to changes in heart rate with a previously stored database and monitored heart rate information. At this time, the shaking information may be obtained in the form of vector values ​​representing speed and direction, but this is merely an example and the shaking information of the electronic device is not limited to the example described above. The electronic device can correct the selected short-exposure images (610b, 610c, 610f, 610g, 610i, 610j) based on the vector values ​​representing the shaking information. The electronic device can obtain a long-exposure image (620) by synthesizing the corrected short-exposure images.

[0057] In another embodiment, another electronic device may obtain a long exposure image by performing interpolation based on selected short exposure images (610b, 610c, 610f, 610g, 610i, 610j). This will be described in more detail later with reference to FIG. 7.

[0058] FIG. 7 is a diagram illustrating a method for an electronic device according to one embodiment to align a plurality of single-exposure images using user's breathing information.

[0059] Referring to FIG. 7, the electronic device can acquire breathing information of a user while acquiring a plurality of single-exposure images (710a to 710l). For example, the electronic device can monitor the user's breathing information while the plurality of single-exposure images (710a to 710l) are acquired by using a breathing detection sensor equipped in the user's electronic device or a breathing detection sensor equipped in a wearable device. According to another example, the electronic device can capture a face image of the user and acquire breathing information of the user from the captured face image. This will be explained in detail with reference to FIG. 8.

[0060] The electronic device can identify intervals (hereinafter, selected intervals) in which the change in heart rate is less than a threshold b based on the user's breathing information. The electronic device can select single exposure images (710a, 710b, 710c, 710d, 710g, 710i, 710j, 710k, 710l) acquired in the selected intervals among a plurality of single exposure images (710a to 710l).

[0061] The electronic device can perform interpolation based on selected single-exposure images (710a, 710b, 710c, 710d, 710g, 710i, 710j, 710k, 710l). For example, the electronic device can obtain motion information of an object included in the selected single-exposure images (710a, 710b, 710c, 710d, 710g, 710i, 710j, 710k, 710l), and based on this, perform interpolation on the selected single-exposure images (710a, 710b, 710c, 710d, 710g, 710i, 710j, 710k, 710l) to obtain new single-exposure images (720e, 720f, 720h). For example, the motion vector between the 7th single exposure image (710g) and the 9th single exposure image (710i) among the selected single exposure images (710a, 710b, 710c, 710d, 710g, 710i, 710j, 710k, 710l). Based on this, interpolation can be performed on the 7th single exposure image (710g) and the 9th single exposure image (710i) to obtain a new single exposure image (720h).

[0062] The electronic device can obtain a long exposure image (730) by synthesizing selected short exposure images (710a, 710b, 710c, 710d, 710g, 710i, 710j, 710k, 710l) and new short exposure images (720e, 720f, 720h) obtained as a result of interpolation.

[0063] FIG. 8 is a diagram illustrating a method for an electronic device according to one embodiment to acquire a long exposure image based on a plurality of short exposure images using the user's breathing information.

[0064] An electronic device according to one embodiment can acquire an image by controlling a camera module (810) based on camera setting information. The camera setting information may include parameters that are subject to control when capturing an image using the camera module, such as shutter speed, ISO information, and the number of shots. However, this is merely an example, and the camera setting information is not limited to the example described above. For example, the electronic device can perform a single exposure image acquisition operation (820) during a preset time interval by controlling the camera module (810) based on a shutter speed of 1 / 2S and ISO information A.

[0065] Meanwhile, the electronic device may perform a face image acquisition operation (830) while performing a single exposure image acquisition operation (820). For example, while the electronic device performs a single exposure image acquisition operation (820) through a rear camera, it may perform a face image acquisition operation (830) for a user using a front camera.

[0066] The electronic device can perform a breathing state identification operation (840) based on the user's face image. According to one embodiment, the electronic device can obtain the user's breathing information as output data by inputting the user's face image into a third artificial intelligence model. According to one example, the weight of each of at least one layer included in the third artificial intelligence model can be set to a value obtained as a result of training such that the difference between the breathing information obtained as a result of inputting the user's face image into the third artificial intelligence model and the actual user's breathing information is greater than or equal to a threshold value.

[0067] The electronic device can perform an operation (850) of selecting at least some of the single-exposure images among the plurality of single-exposure images obtained as a result of performing a single-exposure image acquisition operation (820) based on breathing information. For example, the electronic device can select single-exposure images obtained in a range where the change in the user's breathing is less than a threshold value b.

[0068] The electronic device may perform a single-exposure image correction operation (860) on selected single-exposure images. For example, the electronic device may correct the selected single-exposure images based on shake information of the electronic device or movement information between the selected single-exposure images. Shake information can be identified from breathing information as described above with reference to FIG. 7. For example, if the value of shake information at the time of shooting of the selected first single-exposure image is identified as vector v2, the electronic device may obtain a corrected first single-exposure image by moving the values ​​of the pixels included in the first single-exposure image according to a vector v2' in a direction that cancels out vector v2. However, this is merely an example, and the method of correcting the selected single-exposure images is not limited to the example described above.

[0069] The electronic device can perform a short-exposure image synthesis operation (870) on the corrected short-exposure images. By performing the short-exposure image synthesis operation (870), the electronic device can obtain a long-exposure image. For example, the electronic device can obtain a long-exposure image by calculating one of the average value, maximum value, minimum value, or mode of the pixel value according to the setting for each pixel constituting the corrected short-exposure images. The electronic device can perform a storage operation (880) of the obtained long-exposure image.

[0070] FIG. 9 is a diagram illustrating a method for an electronic device according to one embodiment to acquire a long-exposure image based on a plurality of short-exposure images and shake information of the electronic device.

[0071] Referring to FIG. 9, the electronic device can acquire a long-exposure image (950) by inputting a plurality of short-exposure images (910) and shaking information of the electronic device during the acquisition of the images (e.g., 920 and 930) into a fourth artificial intelligence model (940). The shaking information may be at least one of the user's heart rate information, the user's breathing information (920), or the acceleration information (930) of the electronic device during the acquisition of the plurality of short-exposure images (910), but this is merely an example and the shaking information is not limited to the example described above.

[0072] The fourth artificial intelligence model (940) may include at least one layer, and each layer may include at least one weight. According to various embodiments, the electronic device may determine the value of at least one weight included in at least one layer such that the difference in pixel values ​​between a long exposure image obtained as a result of inputting a plurality of short exposure images and shake information into the fourth artificial intelligence model (940) and a long exposure image generated as a result of synthesizing previously selected short exposure images is less than a threshold value.

[0073] Meanwhile, in the embodiment of FIG. 9, breathing information (920) and acceleration information (930) are described as being input to the fourth artificial intelligence model (940) as shaking information, but this is merely an example, breathing information (920) or acceleration information (930) may be input to the fourth artificial intelligence model (940), and according to other examples, other types of shaking information such as heart rate information may be input in addition to breathing information (920) and acceleration information (930).

[0074] FIG. 10 is a block diagram of an electronic device according to one embodiment.

[0075] Referring to FIG. 10, the electronic device (1000) may include a camera module (1010), memory (1020), a processor (1030), and a display module (1040). However, not all of the illustrated components are essential components. The electronic device (1000) may be implemented with more components than illustrated, or with fewer components.

[0076] The camera module (1010) may include one or more lenses and image sensors, and may receive external light generated from or reflected from an object under the control of an electronic device (1000). A camera module (1010) according to one embodiment may acquire a plurality of single-exposure images under the control of a processor (1030) based on camera setting information.

[0077] The memory (1020) may store a program that enables the electronic device (1000) to perform a method of acquiring a long-exposure image according to the present disclosure. Additionally, the memory (1020) may store a plurality of short-exposure images. The memory (1020) may store at least one of the aforementioned artificial intelligence models.

[0078] The processor (1030) can typically control the overall operation of the electronic device (1000). For example, the processor (1030) can acquire a long exposure image based on a plurality of short exposure images by executing programs stored in memory (1020).

[0079] A processor (1030) according to one embodiment can acquire a plurality of short exposure images during a preset time interval by controlling a camera module (1010) based on camera setting information. The processor (1030) can align at least some of the short exposure images based on shaking information of an electronic device while acquiring the plurality of short exposure images. Additionally, the processor (1030) can synthesize the aligned short exposure images to acquire a long exposure image corresponding to a preset time interval.

[0080] A processor (1030) according to one embodiment can obtain camera setting information by inputting information regarding a shooting mode and a shooting environment into a first artificial intelligence model. The first artificial intelligence model may correspond to the first artificial intelligence model described above with reference to FIG. 3.

[0081] A processor (1030) according to one embodiment can obtain matched single-exposure images by inputting a homography matrix between at least some of the single-exposure images among a plurality of single-exposure images into a second artificial intelligence model. The second artificial intelligence model may correspond to the second artificial intelligence model described above with reference to FIG. 4.

[0082] A processor (1030) according to one embodiment can acquire heart rate information of a monitored user while a plurality of single-exposure images are acquired. Based on the monitored heart rate information of the user, the processor (1030) can select at least some single-exposure images among the plurality of single-exposure images. A processor (1030) according to another embodiment can acquire a face image of a user while a plurality of single-exposure images are acquired. Based on the user's breathing information acquired by inputting the face image of the user into a third artificial intelligence model, the processor (1030) can select at least some single-exposure images among the plurality of single-exposure images. A processor (1030) according to yet another embodiment can acquire three-dimensional acceleration information of an electronic device using an acceleration sensor. Based on the three-dimensional acceleration information of the electronic device, the processor (1030) can select at least some single-exposure images among the plurality of single-exposure images. A processor (1030) according to another embodiment can align at least some of the single-exposure images by inputting shaking information of the electronic device during the acquisition of a plurality of single-exposure images into a fourth artificial intelligence model.

[0083] A processor (1030) according to one embodiment can obtain motion information of an object included in selected short exposure images by selecting some short exposure images among a plurality of short exposure images based on shaking information of an electronic device (1000). The processor (1030) can perform interpolation between the selected short exposure images based on the motion information of the object. The processor (1030) can obtain a long exposure image by synthesizing the short exposure image obtained as a result of performing interpolation and the selected short exposure images.

[0084] The processor (1030) can obtain pixel values ​​of a long exposure image based on one of the average, maximum, minimum, or mode of pixels included in each of the matched short exposure images.

[0085] The display module (1040) can display a preview image of a plurality of short-exposure images generated based on light detected through the camera module (1010). Additionally, the display module (1040) can display a long-exposure image acquired under the control of the processor (1030).

[0086] FIG. 11 is a block diagram of an electronic device in a network environment according to various embodiments.

[0087] Referring to FIG. 11, in a network environment (1100), an electronic device (1101) may communicate with an electronic device (1102) through a first network (1198) (e.g., a short-range wireless communication network) or with at least one of an electronic device (1104) or a server (1108) through a second network (1199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (1101) may communicate with the electronic device (1104) through a server (1108). According to one embodiment, the electronic device (1101) may include a processor (1120), memory (1130), input module (1150), sound output module (1155), display module (1160), audio module (1170), sensor module (1176), interface (1177), connection terminal (1178), haptic module (1179), camera module (1180), power management module (1188), battery (1189), communication module (1190), subscriber identification module (1196), or antenna module (1197). In some embodiments, at least one of these components (e.g., connection terminal (1178)) may be omitted from the electronic device (1101), or one or more other components may be added. In some embodiments, some of these components (e.g., sensor module (1176), camera module (1180), or antenna module (1197)) may be integrated into a single component (e.g., display module (1160)).

[0088] The processor (1120) can, for example, execute software (e.g., program (1140)) to control at least one other component (e.g., hardware or software component) of the electronic device (1101) connected to the processor (1120) and perform various data processing or operations. According to one embodiment, as at least part of the data processing or operations, the processor (1120) can store commands or data received from other components (e.g., sensor module (1176) or communication module (1190)) in volatile memory (1132), process the commands or data stored in volatile memory (1132), and store the resulting data in non-volatile memory (1134). According to one embodiment, the processor (1120) may include a main processor (1121) (e.g., a central processing unit or an application processor) or an auxiliary processor (1123) that can operate independently or together with it (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor). For example, if the electronic device (1101) includes a main processor (1121) and an auxiliary processor (1123), the auxiliary processor (1123) may be configured to use less power than the main processor (1121) or to be specialized for a specified function. The auxiliary processor (1123) may be implemented separately from the main processor (1121) or as part thereof.

[0089] The auxiliary processor (1123) may control at least some of the functions or states associated with at least one component of the electronic device (1101) (e.g., display module (1160), sensor module (1176), or communication module (1190)) on behalf of the main processor (1121) while the main processor (1121) is in an inactive (e.g., sleep) state, or together with the main processor (1121) while the main processor (1121) is in an active (e.g., application execution) state. According to one embodiment, the auxiliary processor (1123) (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component (e.g., camera module (1180) or communication module (1190)). According to one embodiment, the auxiliary processor (1123) (e.g., neural network processing unit) may include a hardware structure specialized for processing an artificial intelligence model. The artificial intelligence model may be generated through machine learning. Such learning may be performed, for example, on the electronic device (1101) itself where the artificial intelligence model is executed, or through a separate server (e.g., server (1108)). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model may include a plurality of artificial neural network layers.An artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to the hardware structure, the artificial intelligence model may include a software structure, either additionally or substantially.

[0090] The memory (1130) can store various data used by at least one component of the electronic device (1101) (e.g., processor (1120) or sensor module (1176)). The data may include, for example, software (e.g., program (1140)) and input or output data for related commands. The memory (1130) may include volatile memory (1132) or non-volatile memory (1134).

[0091] The program (1140) may be stored as software in memory (1130) and may include, for example, an operating system (1142), middleware (1144), or an application (1146).

[0092] The input module (1150) can receive commands or data to be used for a component of the electronic device (1101) (e.g., processor (1120)) from outside the electronic device (1101) (e.g., user). The input module (1150) may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0093] The sound output module (1155) can output a sound signal to the outside of the electronic device (1101). The sound output module (1155) may include, for example, a speaker or a receiver. The speaker may be used for general purposes, such as multimedia playback or recording playback. The receiver may be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from the speaker or as part thereof.

[0094] The display module (1160) can visually provide information to an external (e.g., user) of the electronic device (1101). The display module (1160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling said device. According to one embodiment, the display module (1160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of the force generated by said touch.

[0095] The audio module (1170) can convert sound into an electrical signal or, conversely, convert an electrical signal into sound. According to one embodiment, the audio module (1170) can acquire sound through the input module (1150) or output sound through the sound output module (1155) or an external electronic device (e.g., electronic device (1102)) (e.g., speaker or headphones) connected directly or wirelessly to the electronic device (1101).

[0096] The sensor module (1176) can detect the operating state of the electronic device (1101) (e.g., power or temperature) or the external environmental state (e.g., user state) and generate an electrical signal or data value corresponding to the detected state. According to one embodiment, the sensor module (1176) may include, for example, a gesture sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an accelerometer sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biosensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0097] The interface (1177) may support one or more specified protocols that can be used for the electronic device (1101) to be connected directly or wirelessly to an external electronic device (e.g., electronic device (1102)). According to one embodiment, the interface (1177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0098] The connection terminal (1178) may include a connector through which the electronic device (1101) can be physically connected to an external electronic device (e.g., electronic device (1102)). According to one embodiment, the connection terminal (1178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0099] The haptic module (1179) can convert an electrical signal into a mechanical stimulus (e.g., vibration or movement) or an electrical stimulus that the user can perceive through tactile or kinesthetic senses. According to one embodiment, the haptic module (1179) may include, for example, a motor, a piezoelectric element, or an electric stimulation device.

[0100] The camera module (1180) can capture still images and video. According to one embodiment, the camera module (1180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0101] The power management module (1188) can manage the power supplied to the electronic device (1101). According to one embodiment, the power management module (1188) can be implemented, for example, as at least part of a power management integrated circuit (PMIC).

[0102] The battery (1189) can supply power to at least one component of the electronic device (1101). According to one embodiment, the battery (1189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0103] The communication module (1190) can support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between an electronic device (1101) and an external electronic device (e.g., electronic device (1102), electronic device (1104), or server (1108)), and the performance of communication through the established communication channel. The communication module (1190) may include one or more communication processors that operate independently of the processor (1120) (e.g., application processor) and support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (1190) may include a wireless communication module (1192) (e.g., cellular communication module, short-range wireless communication module, or GNSS (global navigation satellite system) communication module) or a wired communication module (1194) (e.g., LAN (local area network) communication module, or power line communication module). The corresponding communication module among these communication modules can communicate with an external electronic device (1104) via a first network (1198) (e.g., a short-range communication network such as Bluetooth, WiFi (wireless fidelity) direct, or IrDA (infrared data association)) or a second network (1199) (e.g., a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (1192) can identify or authenticate the electronic device (1101) within a communication network such as the first network (1198) or the second network (1199) using subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module (1196).

[0104] The wireless communication module (1192) can support 5G networks and next-generation communication technologies following 4G networks, for example, new radio access technology. NR access technology can support high-speed transmission of high-capacity data (enhanced mobile broadband (eMBB)), minimization of terminal power and connection of multiple terminals (massive machine type communications (mMTC)), or high reliability and low latency (ultra-reliable and low-latency communications (URLLC)). The wireless communication module (1192) can support a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate, for example. The wireless communication module (1192) can support various technologies for securing performance in the high-frequency band, such as beamforming, massive MIMO (multiple-input and multiple-output), full-dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large-scale antenna. The wireless communication module (1192) can support various requirements specified in the electronic device (1101), external electronic device (e.g., electronic device (1104)), or network system (e.g., second network (1199)). According to one embodiment, the wireless communication module (1192) can support a Peak data rate (e.g., 20 Gbps or more) for realizing eMBB, loss coverage (e.g., 164 dB or less) for realizing mMTC, or U-plane latency (e.g., downlink (DL) and uplink (UL) each 0.5 ms or less, or round trip 1 ms or less) for realizing URLLC.

[0105] An antenna module (1197) can transmit a signal or power to or from an external source (e.g., an external electronic device). According to one embodiment, the antenna module (1197) may include an antenna comprising a radiator made of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). According to one embodiment, the antenna module (1197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as a first network (1198) or a second network (1199), may be selected from the plurality of antennas, for example, by a communication module (1190). A signal or power may be transmitted or received between the communication module (1190) and an external electronic device through the selected at least one antenna. According to some embodiments, in addition to the radiator, other components (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as part of the antenna module (1197). According to various embodiments, the antenna module (1197) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent to a first surface (e.g., bottom surface) of the printed circuit board and capable of supporting a specified high frequency band (e.g., mmWave band), and a plurality of antennas (e.g., array antennas) disposed on or adjacent to a second surface (e.g., top surface or side surface) of the printed circuit board and capable of transmitting or receiving a signal of the specified high frequency band.

[0106] At least some of the above components can be connected to each other via a communication method between peripheral devices (e.g., bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)) and exchange signals (e.g., commands or data) with each other.

[0107] According to one embodiment, commands or data may be transmitted or received between the electronic device (1101) and an external electronic device (1104) through a server (1108) connected to a second network (1199). Each of the external electronic devices (1102, or 1104) may be the same or a different type of device as the electronic device (1101). According to one embodiment, all or part of the operations performed on the electronic device (1101) may be performed on one or more of the external electronic devices (1102, 1104, or 1108). For example, if the electronic device (1101) needs to perform a function or service automatically or in response to a request from a user or another device, the electronic device (1101) may request one or more external electronic devices to perform at least part of the function or service instead of performing the function or service itself or additionally. One or more external electronic devices that receive the above request may execute at least part of the requested function or service, or additional function or service related to the request, and transmit the result of the execution to the electronic device (1101). The electronic device (1101) may provide the result as is or additionally processed as at least part of the response to the request. For this purpose, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used. The electronic device (1101) may provide ultra-low latency services using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device (1104) may include an Internet of Things (IoT) device. The server (1108) may be an intelligent server using machine learning and / or neural networks.According to one embodiment, an external electronic device (1104) or server (1108) may be included within the second network (1199). The electronic device (1101) may be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0108] FIG. 12 is a diagram illustrating a method for an electronic device according to one embodiment to acquire a long-exposure image from a server.

[0109] Referring to FIG. 12, the electronic device (1210) can acquire a long exposure image (40) from a plurality of short exposure images (20) by using at least one artificial intelligence model (1230) stored in a server (1220).

[0110] The electronic device (1210) can acquire a plurality of single-exposure images (20) by controlling a camera module based on camera setting information. In one embodiment, camera setting information may be acquired by the electronic device (1210) using the aforementioned first artificial intelligence model. In another embodiment, camera setting information may be received from the server (1220) in response to the electronic device (1210) transmitting information regarding the shooting environment and shooting mode to the server (1220). In this case, camera setting information may be determined by the server (1220) based on information regarding the shooting environment and shooting mode.

[0111] An electronic device (1210) can transmit a plurality of acquired single-exposure images (20) to a server (1220). The server (1220) can align the acquired plurality of single-exposure images (20). According to one embodiment, the server (1220) may include an operation of selecting at least some of the plurality of single-exposure images based on shaking information of the electronic device while the plurality of single-exposure images are being acquired. At this time, the shaking information may be received from the electronic device (1210) along with the plurality of single-exposure images (20). According to another embodiment, the server (1220) may perform interpolation on the selected single-exposure images to acquire a single-exposure image to replace the unselected single-exposure image. According to yet another example, the server (1220) may correct the plurality of single-exposure images based on shaking information of the electronic device or motion information between the plurality of single-exposure images. According to another example, the server (1220) may select at least some of a plurality of single-exposure images and perform an operation to correct at least some of the selected images. The operation of the server (1220) joining the plurality of single-exposure images may correspond to the operation of the electronic device described above joining the plurality of single-exposure images with reference to FIGS. 4 to 9.

[0112] The server (1220) can obtain a long exposure image (40) by synthesizing the aligned short exposure images. The server (1220) can transmit the obtained long exposure image (40) to an electronic device (1210).

[0113] Meanwhile, the server (1220) may be a computing device that provides services to the electronic device (1210), and may be, for example, a PC, laptop, mobile phone, micro server, GPS (global positioning system) device, home appliance, and other mobile or non-mobile computing device. However, it is not limited thereto, and the server (1220) may include all types of devices equipped with communication functions and data processing functions.

[0114] Additionally, the electronic device (1210) and the server (1220) may be connected via a network. In this case, the network may include a Local Area Network (LAN), a Wide Area Network (WAN), a Value Added Network (VAN), a mobile radio communication network, a satellite communication network, and combinations thereof, and is a data communication network in a comprehensive sense that enables the electronic device (1210) and the server (1220) to communicate smoothly with each other, and may include wired internet, wireless internet, and mobile wireless communication networks.

[0115] FIG. 13 is a flowchart illustrating a method for an electronic device and server to acquire a long-exposure image according to one embodiment.

[0116] In step S1310, the electronic device can acquire camera setting information. According to one embodiment, the electronic device can acquire camera setting information as output data by inputting information regarding the shooting mode and shooting environment as input data to the first artificial intelligence model. According to another embodiment, the electronic device can acquire camera setting information based on a previously acquired database. Camera setting information corresponding to the shooting mode or shooting environment may be stored in the database. Accordingly, the electronic device may acquire camera setting information by comparing the current shooting mode and shooting environment with the information stored in the database.

[0117] In step S1320, the electronic device may begin monitoring the user's breathing information or heart rate information. The electronic device may begin monitoring the user's breathing information or heart rate information to acquire shaking information of the electronic device before acquiring multiple single-exposure images based on camera setting information. However, this is merely an example, and the information that serves as the basis for identifying the shaking information of the electronic device is not limited to the user's breathing information or heart rate information. According to other examples, the electronic device may also monitor the three-dimensional acceleration information of the electronic device.

[0118] In step S1330, the electronic device can acquire multiple single-exposure images based on camera setting information.

[0119] In step S1340, the electronic device may transmit a plurality of short-exposure images and monitoring information to a server. The monitoring information may include at least one of the respiration information or heart rate information obtained in step S1320. According to one embodiment, the server may store at least one artificial intelligence model capable of outputting a long-exposure image based on the plurality of short-exposure images and the respiration information or heart rate information.

[0120] In step S1350, the server can align at least some of the single-exposure images among the plurality of single-exposure images. The operation of the server aligning at least some of the single-exposure images among the plurality of single-exposure images may correspond to the above description with reference to FIG. 12.

[0121] In step S1360, the server can obtain a long exposure image by synthesizing the aligned short exposure images. According to one embodiment, the server can obtain a long exposure image by calculating one of the average, maximum, minimum, or mode of pixel values ​​for each pixel constituting the aligned short exposure images, depending on the settings.

[0122] In step S1370, the server can transmit the long-exposure image to the electronic device.

[0123] In step S1380, the electronic device can store a long-exposure image.

[0124] The electronic device according to the various embodiments disclosed in this document may be of various forms. The electronic device may include, for example, a portable communication device (e.g., a smartphone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a consumer electronics device. The electronic device according to the embodiments of this document is not limited to the devices described above.

[0125] The various embodiments of this document and the terms used therein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of said embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of said items unless the relevant context clearly indicates otherwise. In this document, phrases such as "A or B," "at least one of A and B," "at least one of A or B," "A, B or C," "at least one of A, B and C," and "at least one of A, B, or C" may each include any one of the items listed together in the corresponding phrase, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used simply to distinguish said components from other said components and do not limit said components in any other aspect (e.g., importance or order). Where any (e.g., 1st) component is referred to as “coupled” or “connected” to another (e.g., 2nd) component, with or without the terms “functionally” or “communicationly,” it means that said any component may be connected to said other component directly (e.g., via a wire), wirelessly, or through a third component.

[0126] The term “module” as used in the various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. A module may be a component formed integrally, or a minimum unit of said component or a part thereof that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0127] Various embodiments of the present document may be implemented as software (e.g., program (1140)) comprising one or more instructions stored in a storage medium (e.g., internal memory (1136) or external memory (1138)) readable by a machine (e.g., electronic device (1101)). For example, a processor (e.g., processor (1120)) of the machine (e.g., electronic device (1101)) may call at least one of the one or more instructions stored from the storage medium and execute it. This enables the machine to be operated to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. The storage medium readable by the machine may be provided in the form of a non-transitory storage medium. Here, 'non-temporary' simply means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily.

[0128] According to one embodiment, the method according to the various embodiments disclosed herein may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or distributed online (e.g., download or upload) through an application store (e.g., Play Store™) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0129] According to various embodiments, each component (e.g., module or program) of the components described above may include a singular or multiple entities, and some of the multiple entities may be separated and placed in other components. According to various embodiments, one or more of the components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Generally or additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the multiple components in the same or similar manner as those performed by the corresponding component among the multiple components prior to integration. According to various embodiments, operations performed by the module, program, or other components may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

[0130] Functions related to artificial intelligence according to the present disclosure are operated through a processor and memory. The processor may be composed of one or more processors. In this case, the one or more processors may be general-purpose processors such as CPUs, APs, and DSPs (Digital Signal Processors), graphics-dedicated processors such as GPUs and VPUs (Vision Processing Units), or artificial intelligence-dedicated processors such as NPUs. The one or more processors control the processing of input data according to predefined operation rules or artificial intelligence models stored in memory. Alternatively, if the one or more processors are artificial intelligence-dedicated processors, the artificial intelligence-dedicated processors may be designed with a hardware structure specialized for processing a specific artificial intelligence model.

[0131] The predefined rules of operation or artificial intelligence models are characterized by being created through learning. Here, being created through learning means that a predefined rules of operation or artificial intelligence models configured to perform a desired characteristic (or objective) are created by a basic artificial intelligence model being trained using multiple learning data by a learning algorithm. Such learning may be performed on the device itself where the artificial intelligence according to the present disclosure is executed, or it may be performed through a separate server and / or system. Examples of learning algorithms include supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but are not limited to the examples described above.

[0132] An artificial intelligence model may be composed of multiple neural network layers. Each of the multiple neural network layers has multiple weight values ​​and performs neural network operations through operations between the results of previous layers and the multiple weights. The multiple weights possessed by the multiple neural network layers can be optimized based on the learning results of the artificial intelligence model. For example, the multiple weights may be updated so that the loss value or cost value obtained from the artificial intelligence model during the learning process is reduced or minimized. Artificial neural networks may include deep neural networks (DNNs), such as Convolutional Neural Networks (CNNs), Deep Neural Networks (DNNs), Recurrent Neural Networks (RNNs), Restricted Boltzmann Machines (RBMs), Deep Belief Networks (DBNs), Bidirectional Recurrent Deep Neural Networks (BRDNNs), or Deep Q-Networks, but are not limited to the examples mentioned above.

[0133] In a method for generating noise-reduced image data for an object using an electronic device including a camera module according to the present disclosure, an artificial intelligence model may be used to generate a reference color data set for generating noise-reduced image data from raw image data. A processor may perform a preprocessing step on the data to convert it into a form suitable for use as input to an artificial intelligence model. An artificial intelligence model may be created through learning. Here, being created through learning means that a basic artificial intelligence model is trained using a number of training data by a learning algorithm, thereby creating a predefined operation rule or artificial intelligence model configured to perform a desired characteristic (or purpose). An artificial intelligence model may be composed of a plurality of neural network layers. Each of the plurality of neural network layers has a plurality of weight values ​​and performs neural network operations through operations between the result of a previous layer and the plurality of weights.

[0134] Inference prediction is a technology that logically reasones and predicts by judging information, and includes knowledge-based reasoning, optimization prediction, preference-based planning, and recommendation.

Claims

Claim 1 A method for an electronic device to process images, comprising: a step of acquiring a plurality of short exposure images during a preset time interval by controlling a camera module included in the electronic device based on camera setting information; a step of inputting the plurality of short exposure images and shaking information of the electronic device, including user heart rate information, user respiration information, and three-dimensional acceleration information of the electronic device, during the acquisition of the plurality of short exposure images, into a first artificial intelligence model; and a step of acquiring a long exposure image corresponding to the preset time interval as the output of the first artificial intelligence model, wherein the first artificial intelligence model includes at least one layer, and the weight of the at least one layer is determined such that the difference in pixel values ​​between the long exposure image acquired as the output of the first artificial intelligence model and the long exposure image generated as a result of synthesizing selected short exposure images among the plurality of short exposure images is less than a threshold value. Claim 2 A method according to claim 1, further comprising the step of obtaining camera setting information by inputting information regarding a shooting mode and a shooting environment into a second artificial intelligence model, wherein the camera setting information includes information regarding at least one of a shutter speed, ISO information, or the number of shots. Claim 3 delete Claim 4 delete Claim 5 A method according to claim 1, further comprising the step of acquiring a user's face image while the plurality of single-exposure images are acquired, wherein the user's breathing information is acquired by inputting the user's face image into a third artificial intelligence model. Claim 6 A method according to claim 1, wherein three-dimensional acceleration information of the electronic device is obtained using an acceleration sensor included in the electronic device. Claim 7 delete Claim 8 delete Claim 9 delete Claim 10 delete Claim 11 An electronic device for processing images, comprising: a memory for storing one or more instructions; a camera module; and at least one processor for executing the one or more instructions stored in the memory, wherein the at least one processor controls the camera module based on camera setting information to acquire a plurality of short exposure images during a preset time interval, and inputs the plurality of short exposure images and shaking information of the electronic device, including user heart rate information, user respiration information, and three-dimensional acceleration information of the electronic device during the acquisition of the plurality of short exposure images, to a first artificial intelligence model, wherein a long exposure image corresponding to the preset time interval is acquired as the output of the first artificial intelligence model, and wherein the first artificial intelligence model includes at least one layer, and the weight of the at least one layer is determined such that the difference in pixel values ​​between the long exposure image acquired as the output of the first artificial intelligence model and the long exposure image generated as a result of synthesizing selected short exposure images among the plurality of short exposure images is less than a threshold value. Claim 12 An electronic device according to claim 11, wherein the at least one processor acquires camera setting information by inputting information regarding a shooting mode and a shooting environment into a second artificial intelligence model, and the camera setting information includes information regarding at least one of a shutter speed, ISO information, or shooting speed. Claim 13 delete Claim 14 delete Claim 15 An electronic device according to claim 11, wherein at least one processor acquires a user's face image while the plurality of single-exposure images are acquired, and the user's breathing information is acquired by inputting the user's face image into a third artificial intelligence model. Claim 16 In claim 11, the three-dimensional acceleration information of the electronic device, the electronic device obtained using an acceleration sensor included in the electronic device. Claim 17 delete Claim 18 delete Claim 19 delete Claim 20 delete Claim 21 A recording medium having one or more computer-readable media storing a program that enables an electronic device to perform a method of processing images, wherein the program comprises: an operation of acquiring a plurality of short exposure images during a preset time interval by controlling a camera module included in the electronic device based on camera setting information; an operation of inputting the plurality of short exposure images and shaking information of the electronic device, including user heart rate information, user respiration information, and three-dimensional acceleration information of the electronic device, during the acquisition of the plurality of short exposure images, into a first artificial intelligence model; and an operation of acquiring a long exposure image corresponding to the preset time interval as the output of the first artificial intelligence model, wherein the first artificial intelligence model includes at least one layer, and the weight of the at least one layer is determined such that the difference in pixel values ​​between the long exposure image acquired as the output of the first artificial intelligence model and the long exposure image generated as a result of synthesizing selected short exposure images among the plurality of short exposure images is less than a threshold value.

Images