Method for taking a picture using multiple cameras and apparatus therefor

Abstract

The processor included in the electronic device according to an embodiment can execute a first application associated with the first camera module and the second camera module, and output a first preview image to a first area of the flexible display based on first image data acquired through the first camera module in response to execution of the first application. While the flexible display outputs the first preview image, the processor can detect an event to expand the flexible display, and output a second preview image to a second area of the flexible display based on second image data acquired through the second camera module, the second area being expanded to be greater than the first area. Various other embodiments determined through the specification are possible.

Description

Technical Field

[0001] The embodiments disclosed herein relate to electronic devices and methods for taking photographs using multiple cameras. Background Technology

[0002] Mobile devices such as smartphones or tablets include cameras for capturing photos and videos. A mobile device may include multiple cameras on its front or rear surface to enhance camera performance. For example, multiple cameras may include a wide-angle camera, an ultra-wide-angle camera, a telephoto camera, and / or a macro camera. To switch between multiple cameras with the aforementioned functions, a user can change the camera by selecting an icon indicating the corresponding camera, thus selecting a camera with different features.

[0003] In addition to multiple cameras, mobile devices are equipped with various displays to improve the user experience. For example, mobile devices can be equipped with flexible displays designed to bend or slide. Summary of the Invention

[0004] Technical issues

[0005] The problem with mobile devices is that even when the sliding display expands the display and changes the resolution, the expanded area is not utilized. Therefore, if the appropriate resolution is not provided based on the activity state of the display before and after the expansion, the expanded area is a waste of space where the user interface is not displayed.

[0006] Furthermore, when the display is expanded during operation of an application used for taking photos or videos, the already operating camera module remains unchanged, regardless of whether the display is expanded. Therefore, despite the increased display area of ​​electronic devices, the ability to switch camera modules to reflect the user's intentions has become limited.

[0007] Various embodiments of this disclosure may provide an apparatus and method in which, in response to the expansion of a flexible display of a mobile device supporting multiple cameras, camera switching is supported, and the preview area is controlled according to the changed resolution.

[0008] Technical solution

[0009] An electronic device according to embodiments disclosed herein may include a flexible display, a first camera module, a second camera module, and at least one processor electrically connected to the flexible display, the first camera module, and the second camera module. The at least one processor may execute a first application associated with the first and second camera modules, and in response to the execution of the first application, output a first preview image based on first image data acquired by the first camera module in a first region of the flexible display. The at least one processor may simultaneously detect an event for expanding the flexible display while outputting the first preview image on the flexible display, and in response to the detection of the event, output a second preview image based on second image data acquired by the second camera module in a second region expanded compared to the first region of the flexible display.

[0010] Furthermore, the method for operating an electronic device according to the embodiments disclosed herein may include the following operations: executing a first application associated with a first camera module and a second camera module; in response to the execution of the first application, outputting a first preview image based on first image data acquired by the first camera module in a first region of a flexible display; while outputting the first preview image on the flexible display, detecting an event for expanding the flexible display; and in response to the detection of the event, outputting a second preview image based on second image data acquired by the second camera module in a second region expanded compared to the first region of the flexible display.

[0011] Furthermore, the electronic device according to embodiments disclosed herein may include a display, a plurality of camera modules including a first camera module, a second camera module, and a third camera module, and at least one processor electrically connected to the display and the plurality of camera modules. The at least one processor may execute a first application associated with the plurality of camera modules, and in response to the execution of the first application, output a first preview image based on first image data acquired by the first camera module in a first area of ​​the flexible display, detect an event for expanding the display while outputting the first preview image on the flexible display, activate a second camera module or a third camera module in response to the detection of the event, and output a second preview image based on second image data acquired by the second camera module or a third preview image based on third image data acquired by the third camera module in a second area expanded compared to the first area of ​​the display.

[0012] Beneficial effects

[0013] According to the various embodiments disclosed herein, the camera module can be switched depending on the extent of the display's expansion.

[0014] Furthermore, according to various embodiments, the resolution of the preview area can be changed depending on the extent of the display's expansion.

[0015] Furthermore, according to various embodiments, the usability of swiping for users can be increased.

[0016] Various other beneficial effects, whether explicitly or implicitly identified through this disclosure, may be provided. Attached Figure Description

[0017] Figure 1a The front surface of an electronic device according to an embodiment is shown.

[0018] Figure 1b The rear surface of an electronic device according to an embodiment is shown.

[0019] Figure 2 The hardware and software configurations of an electronic device according to an embodiment are shown.

[0020] Figure 3 The process of exchanging information about a camera and a display in an electronic device according to an embodiment is illustrated.

[0021] Figure 4 This is a flowchart illustrating the process of switching a camera module based on a display extension event in an electronic device according to an embodiment.

[0022] Figure 5a The process of switching the camera module when expanding the display in an electronic device, according to an embodiment, is illustrated.

[0023] Figure 5b The process of switching the camera module when expanding the display in an electronic device, according to an embodiment, is illustrated.

[0024] Figure 6 This is a flowchart illustrating a process of expanding a display in response to user drag-and-drop input in an electronic device, according to an embodiment.

[0025] Figure 7a This is a flowchart illustrating the process of switching a camera module to a second camera module in response to user drag input in an electronic device, according to an embodiment.

[0026] Figure 7b This is a flowchart illustrating the process of switching a camera module to a third camera module in response to user drag input in an electronic device, according to an embodiment.

[0027] Figure 8 This is a flowchart illustrating the process of expanding the display and switching the camera module based on a touch-based preview area in an electronic device according to an embodiment.

[0028] Figure 9 The process of expanding the display and switching the camera module based on a touch-based preview area in an electronic device, according to an embodiment, is illustrated.

[0029] Figure 10 This is a flowchart illustrating a process, according to an embodiment, of changing the resolution of a preview area based on the extent of expansion of a display in an electronic device.

[0030] Figure 11 The process of changing the resolution of the preview area according to the extent of the display in the electronic device, according to an embodiment, is illustrated.

[0031] Figure 12 The process of editing an image according to the extent of the display in the electronic device, according to an embodiment, is illustrated.

[0032] Figure 13 This is a block diagram illustrating an electronic device in a network environment according to various embodiments.

[0033] Figure 14 This is a block diagram illustrating a camera module according to various embodiments. Detailed Implementation

[0034] In the following description, various embodiments of the present disclosure will be illustrated with reference to the accompanying drawings. However, this is not intended to limit the disclosure with respect to any particular embodiment, but rather to be construed as including various modifications, equivalents, and / or substitutions of the disclosed embodiments.

[0035] Figure 1a The front surface of an electronic device according to an embodiment is shown. Figure 1b The rear surface of an electronic device according to an embodiment is shown.

[0036] refer to Figure 1a and Figure 1b Electronic devices (e.g.) Figure 13 The electronic device 1301) 100 may include a first housing 121 and a second housing 122. When the display (e.g., Figure 13 When the display module 1360) 110 is extended, at least a portion of the second housing 122 can be separated from the first housing 121. For example, the second housing 122 can protrude outside the electronic device 100 while being connected to the first housing via a first plate. The first housing 121 may be referred to herein as the main housing, and the second housing 122 may be referred to herein as the sliding housing. The description of the display 110 being extended may be understood herein to mean that the area of ​​the display 110 disposed on the front surface of the electronic device 100 is extended.

[0037] In an embodiment, display 110 may include a flexible display. Display 110 may employ a flexible display, a rolling structure, and / or a hinge structure, such that display 110 can fold or slide in a horizontal or vertical direction. Display 110 may be configured such that the front area of ​​display 110 can be expanded when viewed from the front via a reference scroll axis 170. The front area of ​​display 110 can be expanded or reduced via a scroll motor included within electronics 100, referencing scroll axis 170. For example, in response to detecting a command to expand display 110, a processor (e.g., Figure 2 The processor 240 can drive a rolling motor to expand or shrink the display 110. The display 110 can be expanded or shrunk relative to the rolling axis 170 by an external force. For example, the front area of ​​the display 110 can be expanded by a force applied by the user pulling the second housing 122. In response to the user pulling the surface of the display 110 or the second housing 122, the processor 240 can drive the rolling motor to assist the user in expanding the display 110.

[0038] In this embodiment, when the display 110 is not extended, the default area 161 of the display 110 may be disposed on the front surface of the electronic device 100. When the display 110 is not extended, the default area 161 may occupy a large portion of the front surface of the electronic device 100. When the display 110 is extended, the default area 161 of the display 110 and the extended area 162 extended and disposed on the front surface may be disposed on the front surface of the electronic device 100. The display 110 may occupy a portion of the side surface of the electronic device 100.

[0039] In an embodiment, the display 110 may be configured to surround at least a portion of the first housing 121 and at least a portion of the second housing 122. The display 110 and a bezel region surrounding at least a portion of the periphery of the display 110 may be formed on the front surface of the electronic device 100. Figure 1a In one example, the display 110 may include a flat region 111 and a curved region 112 extending from the flat region 111 toward the side surface of the electronic device 100. Although in Figure 1a The curved region 112 is shown only with respect to the surface of the electronic device 100 (e.g., the front surface), but it is understood that the curved region 112 may also be formed with respect to another surface of the electronic device 100 (e.g., the rear surface). For example, the curved region 112 may extend to the rear surface of the electronic device 100, and in this case, the electronic device 100 may include a display 110 on both its front and rear surfaces.

[0040] In one embodiment, the display 110 may extend to the rear surface of the electronic device 100 while occupying a large portion of the front surface of the electronic device 100. The area of ​​the display 110 extending to the rear surface is not exposed through the rear cover. For example, when the display 110 is not extended, the area of ​​the display 110 may be inserted into the rear cover 160.

[0041] In one embodiment, a first region 140 of the display 110 may include a fingerprint sensor 141 for identifying a user's fingerprint. The fingerprint sensor 141 may be disposed on the lower layer of the display 110, making it invisible to the user, or configured to be nearly invisible. Furthermore, in addition to the fingerprint sensor 141, additional sensors for user / biometric authentication may be disposed on a portion of the display 110. In another embodiment, the sensor for user / biometric authentication may be disposed in an area of ​​the bezel. For example, an IR sensor for iris authentication may be exposed either through an area of ​​the display 110 or through an area of ​​the bezel.

[0042] In one embodiment, the front-facing camera 131 may be disposed on the front surface of the electronic device 100. Although in Figure 1a In one embodiment, the front-facing camera 131 is shown to be exposed through an area of ​​the display 110, but in another embodiment, the front-facing camera 131 may be exposed through a border.

[0043] In an embodiment, electronic device 100 may include one or more front-facing cameras 131. For example, electronic device 100 may include multiple cameras, such as at least a first front-facing camera and a second front-facing camera. In an embodiment, the first front-facing camera and the second front-facing camera may be analogous cameras with the same specifications (e.g., pixels), but the first front-facing camera and the second front-facing camera may be implemented as cameras with different specifications. Electronic device 100 may support dual-camera related functions (e.g., 3D photography, autofocus, etc.) through two front-facing cameras.

[0044] In an embodiment, the rear camera 132 may be disposed on the rear surface of the electronic device 100. The rear surface 132 may be exposed through an area of ​​the rear cover 160. In an embodiment, the electronic device 100 may include multiple rear cameras arranged in the camera area 130. For example, the electronic device 100 may include two or more rear cameras. For example, the electronic device 100 may include a first rear camera, a second rear camera, and a third rear camera. The first rear camera, the second rear camera, and the third rear camera may have different specifications. For example, the first rear camera, the second rear camera, and / or the third rear camera may differ from each other in terms of FOV, pixels, aperture, whether optical zoom / digital zoom is supported, whether image stabilization is supported, and the type and arrangement of the lens groups included in each camera. For example, the first rear camera may be a general-purpose camera, the second rear camera may be a camera for wide-angle photography, and the third rear camera may be a camera for telephoto photography. The descriptions of the functions or features of the front cameras herein apply to the rear cameras, and vice versa.

[0045] In this embodiment, various hardware devices for assisting photography, such as a flash (e.g., ...), can be additionally provided in the camera area 130. Figure 14 The flash (1420) or sensor in the camera area 130 may be included. For example, a distance detection sensor (e.g., a time-of-flight (TOF) sensor) for detecting the distance between the object and the electronic device 100 may be further included in the camera area 130.

[0046] In an embodiment, at least one physical key may be located on the side of the electronic device 100. For example, a first function key 151 for turning the display 110 on / off or for turning the power of the electronic device 100 on / off may be located on the right periphery relative to the front surface of the electronic device 100. The first function key 151 may be located on the second housing 122. In an embodiment, a second function key 152 for controlling the volume of the electronic device 100 or for controlling the screen brightness, etc., may be located on the left periphery relative to the front surface of the electronic device 100. Furthermore, additional buttons or keys may be provided on the front or rear surface of the electronic device 100. For example, physical buttons or touch buttons mapped to specific functions may be located in the bottom area of ​​the front bezel.

[0047] Figure 1a and Figure 1b The electronic device 100 shown corresponds to an example and does not limit the type of device to which the technical ideas disclosed herein apply. The technical ideas disclosed herein are also applicable to tablet computers or laptops. For ease of description, reference will be made below. Figure 1a and Figure 1b The electronic device 100 shown is used to describe various embodiments.

[0048] Figure 2 The hardware and software configuration of an electronic device according to an embodiment is illustrated. Reference Figure 2 The components included in the electronic device 100 may have various electrical / operational connections.

[0049] Reference Figure 2 The electronic device 100 may include multiple camera modules, a processor (e.g., Figure 13 The processor 1320) 240, memory (e.g., Figure 13 The device includes a memory 1330) 250, an input / output device 260, a rolling motor 270, and a communication circuit 280.

[0050] In an embodiment, the plurality of camera modules may include at least a first camera module 210, a second camera module 220, and a third camera module 230. The first camera module 210 may include a lens assembly (e.g., Figure 14 Lens assembly 1410) 201, image sensor (e.g., Figure 14 The image sensor 1430) 203 and the image signal processor (e.g., Figure 14 The image signal processor 1460)205 in the image signal processor is referenced. Figure 2 Although the lens assembly 201, image sensor 203 and image signal processor 205 are shown as included in the first camera module 210, the description of the first camera module 210 may be applied in the same or similar way to the second camera module 220 and the third camera module 230.

[0051] In this embodiment, the image sensor 203 may be a complementary metal-oxide-semiconductor (CMOS) sensor or a charge-coupled device (CCD). The image sensor 203 may have multiple individual pixels integrated thereon, and each individual pixel may include a microlens, a color filter, and a photodiode. Each individual pixel can convert input light into an electrical signal.

[0052] In this embodiment, the image signal processor 205 can acquire the converted electrical signal. The image signal processor 205 can perform image processing on the electrically converted image data. The image signal processor 205 can perform image processing such as 3A processing, lens shading correction, edge enhancement, and bad pixel correction. 3A may include at least one of automatic white balance (AWB), automatic exposure (AE), and automatic focus (AF).

[0053] In this embodiment, the processor 240 can send and receive network information via communication circuit 280 based on wireless communication. The processor 240 can obtain user location information, information about the shooting location, etc., from external sources via communication circuit 280. The processor 240 can store multiple pieces of information obtained via communication circuit 280 in memory 250. The processor 240 can provide the multiple pieces of information obtained via communication circuit 280 to multiple camera modules.

[0054] In embodiments, memory 250 may store instructions and / or data that can be executed by processor 240. Memory 250 may store image data acquired by the camera module. Memory 250 may store still images and / or video acquired by the camera module. Memory 250 can conceptually be understood as containing components for temporary data storage, such as random access memory (RAM), and / or components for permanent data storage, such as solid-state drives (SSDs). In various embodiments, memory 250 may include various types and may adopt an appropriate type depending on the purpose of the device.

[0055] In this embodiment, memory 250 may store applications associated with the first camera module 210, the second camera module 220, and the third camera module 230. For example, memory 250 may store camera applications. These camera applications may support various photographic functions, such as photo capture, video capture, panorama capture, and slow-motion capture.

[0056] In this embodiment, the application associated with the camera module can correspond to various types of applications. For example, chat applications, web browser applications, email applications, or shopping applications can use the first camera module 210, the second camera module 220, and the third camera module 230 to support various functions such as video communication, photo / video attachments, streaming services, and product image or product-related virtual reality (VR) capture functions.

[0057] In an embodiment, the input / output device 260 may include at least a display 110, a speaker 261, and a microphone 262. The display 110 may be integrated with a touchpad. The display 110 may support touch functionality, detect user input such as touches made with a finger, and transmit it to the processor 240. The display 110 may display an execution screen about an application executed by the processor 240, or content such as images and / or videos stored in the memory 250 may be displayed on the display 110. The processor 240 may display on the display 110 in real time via a camera module (e.g., ...). Figure 2The first camera module 210 acquires image data. The processor 240 can acquire the user's voice input via the microphone 262. The processor 240 can perform a voice recognition function corresponding to the acquired user voice. The speaker 261 can output a voice signal.

[0058] In one embodiment, the rolling motor 270 may be electrically connected to the rolling shaft 170 and the processor 240. The processor 240 may rotate the rolling shaft by applying voltage to the rolling motor 270. The rolling motor 270 may determine the degree of rotation of the rolling shaft based on the sliding state information of the display 110 obtained from the processor 240.

[0059] Figure 3 The process of exchanging information about a camera and a display in an electronic device according to an embodiment is illustrated.

[0060] In this embodiment, the operating system of the electronic device 100 can operate a camera application. The camera application can request information from the operating system. This information may include information about the rotation angle of the scroll axis, information about the sliding state of the display, and / or information about the camera module. The operating system can obtain information about the rotation angle of the scroll axis and / or information about the sliding state of the display 110 from the scroll motor. The operating system can obtain information about the currently operating camera module from at least one camera module.

[0061] In this embodiment, the camera application may acquire information about the rotation angle of the scroll axis, information about the sliding state of the display, and / or information about the camera module. Based on the acquired information, the camera application may provide the operating system with information about the camera module to be operated.

[0062] In this embodiment, the operating system can determine the sliding state of the display 110 and the resolution of the display 110. The operating system can expand or shrink the display by operating a scroll motor. The operating system can perform camera module switching based on the resolution of the display 110 or the sliding state of the display 110. For example, when the display 110 is not sliding and therefore not in an expanded state, the operating system can maintain the operation of the preview area of ​​the display 110 and the camera module. When the display 110 has slid and is therefore in an expanded state, the operating system can perform at least one of changing the preview area of ​​the display 110, changing the resolution, or switching the camera module.

[0063] Figure 4 This is a flowchart illustrating the process of switching camera modules based on display extension events in an electronic device according to an embodiment. The process of switching camera modules based on display extension events can be applied not only to capturing photographs but also to capturing video.

[0064] In operation 410 according to an embodiment, processor 240 may execute a first application associated with the first camera module 210 and the second camera module 220. The first application may be a camera application and / or a camera driver-related application. Processor 240 may execute the first application in response to input regarding its execution. Processor 240 may activate the first camera module 210 and / or the second camera module 220 in response to the execution of the first application. Processor 240 may acquire image data through the activated camera module.

[0065] In operation 420 according to an embodiment, processor 240 may output a first preview image based on first image data acquired by first camera module 210 in a first area of ​​display 110. The first area may be at least a portion of the area where a preview image can be output before display 110 is expanded. When display 110 is not expanded, the first area may be an area configured to face the front surface of electronic device 100. Figure 4 The first camera module 210 mentioned above can be understood as a camera module that includes a wide-angle camera.

[0066] In operation 430 according to an embodiment, the processor 240 can detect events for extending the display 110. The processor 240 can detect events for extending the display 110 while simultaneously outputting a first preview image on the display 110.

[0067] In an embodiment, events that extend the display 110 may be generated by external force and / or control of the processor 240. For example, the front area of ​​the display 110 may be extended by a force applied by a user to pull a housing (e.g., the second housing 122 in FIG. 1).

[0068] In an embodiment, processor 240 may expand the front area of ​​display 110 in response to user input for expanding display 110. User input may include touch input and / or voice input via display 110. For example, upon receiving voice input such as “Bixby, expand display”, processor 240 may expand display 110 by driving scroll motor 270. User input may also include user input via pressing a physical button. For example, electronic device 100 may include a separate button for expanding / reducing display 110 on the housing surface, such that if a user presses the button, the display 110 is expanded or reduced by driving scroll motor 270.

[0069] According to an embodiment, in operation 440, the processor 240 can output a second preview image based on second image data acquired by the second camera module 220 in a second area of ​​the display. In response to detecting an event for expanding the display 110, the processor 240 can output the second preview image in a second area expanded compared to the first area of ​​the display 110. The second area can be at least a portion of the area where a preview image can be output after the display 110 is expanded. When the display 110 is expanded, the second area can be an area arranged to face the front surface of the electronic device 100. Figure 4 The second camera module 220 mentioned above can be a camera module that includes at least one of a macro lens, a telephoto lens, or an ultra-wide-angle lens.

[0070] In an embodiment, upon detecting an event for extending the display 110, the processor 240 may analyze the user's intent and / or preview image to determine which camera module to switch to. The processor 240 may detect objects included in the acquired image data, thereby acquiring data about those objects. The processor 240 may determine the type of object using object detection functionality. The processor 240 may determine the distance between the electronics 100 and the object to be photographed using a distance detection sensor (e.g., a time-of-flight (TOF) sensor).

[0071] In an embodiment, processor 240 may determine whether to perform macro photography based on data about the object and / or focal length. When it is confirmed that the user is performing macro photography, processor 240 may switch the currently operating camera module from the first camera module 210 to a second camera module 220 that includes a macro lens. For example, when the object is within a first distance (e.g., 30 cm) from the electronic device 100, processor 240 may switch the currently operating camera module to the second camera module 220 that includes a macro lens in response to an event from the extended display 110.

[0072] In an embodiment, processor 240 may determine whether a landscape is being photographed based on image data, data about objects, and / or focal length. When it is determined that the user is photographing a landscape, processor 240 may switch the operating camera module from first camera module 210 to second camera module 220, which includes an ultra-wide-angle lens. For example, when analysis of image data confirms that natural objects (e.g., natural objects such as sky, ocean, trees, mountains, and fields) occupy a first proportion or more, and that the natural objects are out of focus, processor 240 may switch the operating camera module to second camera module 220, which includes an ultra-wide-angle lens, in response to an event on extended display 110.

[0073] In an embodiment, processor 240 may determine whether to perform long-distance photography of the object based on data about the object and / or focal length. When it is determined that the user is performing long-distance photography, processor 240 may switch the operating camera module from first camera module 210 to second camera module 220, which includes a telephoto lens. For example, when the main object to be photographed is displayed in the center of the preview image, and when the object is located beyond a second distance (e.g., 1 m) from electronic device 100 and beyond focal length, processor 240 may switch the operating camera module to second camera module 220, which includes a telephoto lens, in response to an event for extending display 110.

[0074] In this embodiment, when switching camera modules, the processor 240 can perform the switching seamlessly. To achieve seamless camera module switching, the processor 240 can gradually change the preview image over a predetermined number of frames. For example, when switching the currently operating camera module from the first camera module 210 to the second camera module 220, the processor 240 can gradually display the changes between the preview image acquired by the first camera module 210 and the preview image acquired by the second camera module 220 over N frames.

[0075] Figure 5a The process of switching the camera module when expanding the display in an electronic device, according to an embodiment, is illustrated. Figure 5b The process of switching the camera module when expanding the display in an electronic device, according to an embodiment, is illustrated. Figure 5a and Figure 5b It can be shown that corresponds to Figure 4 Examples of implementations.

[0076] Reference Figure 5a and Figure 5b The processor 240 can display a first icon 501, a second icon 502, a third icon 503, and a first button 504 related to the shooting mode in the area of ​​the display 110. The first icon 501 can be a button for telephoto photography. The second icon 502 can be a button for normal photography. The third icon 503 can be a button for ultra-wide-angle photography. The first button 504 can be a button for starting photography. The position of the first button 504 can be changed depending on whether the display is expanded.

[0077] Reference Figure 5a Based on a camera module including a wide-angle camera, the display 110 can display a first preview image 510 in a first area of ​​the display 110. The first preview image 510 may include a first object 511 having a magnification corresponding to the wide-angle lens.

[0078] Reference Figure 5aWhen the display 110 has been expanded, a second preview image 530 can be displayed on a second area of ​​the display 110 based on a camera module including an ultra-wide-angle camera. The second preview image 530 may include a second object 531 having a magnification corresponding to the ultra-wide-angle lens.

[0079] In this embodiment, the processor 240 can acquire image data via a camera module including an ultra-wide-angle lens. The field of view of the image data acquired via the camera module including the ultra-wide-angle lens can be wider than that of the image data acquired via the camera module including the wide-angle lens. The processor 240 can output the image data acquired based on the ultra-wide-angle lens as a preview image via the extended display 110.

[0080] In one embodiment, the processor 240 can detect that the extended display 110 is shrinking. The processor 240 can shrink the display 110 when the display 110 shrinks in response to a change in camera mode, or when an external force is detected to shrink the display 110. This can also be applied in the case where the display 110 has been extended in the following embodiments.

[0081] In one embodiment, the processor 240 can modify the camera module when the extended display 110 is scaled down. For example, when the display 110 is scaled down while outputting a preview based on image data acquired by a camera module including an ultra-wide-angle lens, the processor 240 can output a preview based on image data acquired by a camera module including a wide-angle lens. In the following embodiments, this can be applied similarly to cases where the display 110 is expanded and then scaled down.

[0082] In this embodiment, the processor 240 can maintain the camera module even if the extended display 110 is scaled down. The processor 240 can determine whether to keep the camera module unchanged by analyzing elements such as detected objects and the photographic environment. The processor 240 can keep the camera module unchanged even though the display 110 is scaled down, provided that elements such as the photographed object and the photographic environment have not changed. This also applies to… Figure 5b And the accompanying diagram.

[0083] Reference Figure 5b When the display 110 is extended, a third preview image 540 can be displayed on a second area of ​​the display 110 based on a camera module including a macro lens. The third preview image 540 may include a third object 541 having a magnification corresponding to that of the macro lens.

[0084] In this embodiment, the processor 240 can acquire image data via a camera module including a macro lens. The field of view of the image data acquired via the camera module including a macro lens can be narrower than that of the image data acquired via a camera module including a wide-angle lens. The processor 240 can output the image data acquired via the macro lens as a preview image via the extended display 110.

[0085] In an embodiment, the processor 240 can modify the camera module when the extended display 110 is scaled down. For example, when scaling down the display 110 while outputting a preview based on image data acquired by a camera module including a macro lens, the processor 240 can output a preview based on image data acquired by a camera module including a wide-angle lens. Figure 6 This is a flowchart illustrating a process of expanding a display in response to user drag-and-drop input in an electronic device, according to an embodiment. Figure 6 This can be understood as being in Figure 4 The operation will be executed after step 420.

[0086] In operation 610 according to an embodiment, when the photography mode icon is touched, the processor 240 can determine whether the user's drag direction is the display expansion direction. The photography mode icon can be understood as an icon associated with a photography mode. (See below for further details.) Figure 7a and Figure 7b Detailed description of the icons associated with the photography mode.

[0087] In this embodiment, the processor 240 can determine whether to expand the display 110 based on the direction of the user's drag input. For example, when the direction of the user's drag input is the same as the expansion direction of the display 110, the processor 240 can expand the display. When the direction of the user's drag input is opposite to the expansion direction of the display 110, the processor 240 may not expand the display.

[0088] In operation 620 according to an embodiment, processor 240 can expand the display. When the camera mode icon is touched, and the user's drag direction is the same as the display expansion direction, processor 240 can expand the display 110.

[0089] In operation 630 according to an embodiment, processor 240 can switch camera modules. Processor 240 can switch the currently operating camera module to the camera module corresponding to the drag input, regardless of whether the display 110 is extended.

[0090] In this embodiment, the processor 240 can switch the currently operating camera module to a second camera module 220 or a third camera module 230 based on the type of drag-and-drop input. The processor 240 can determine the type of drag-and-drop input based on its starting point. For example, when the starting point of the drag-and-drop input is included in an icon associated with ultra-wide-angle photography, the processor 240 can switch the currently operating camera module to a second camera module 220 that includes an ultra-wide-angle lens. As another example, when the starting point of the drag-and-drop input is included in an icon associated with telephoto and / or macro photography, the processor 240 can switch the currently operating camera module to a third camera module 230 that includes a telephoto lens and / or a macro lens.

[0091] In this embodiment, the processor 240 can acquire image data via a switched camera module. The processor 240 can then output a preview image based on the acquired image data through a region of the display 110.

[0092] Figure 7a This is a flowchart illustrating the process of switching the currently operating camera module to a second camera module 220 in response to user drag input in an electronic device, according to an embodiment; Figure 7b This is a flowchart illustrating the process of switching the currently operating camera module to a third camera module in response to user drag input in an electronic device, according to an embodiment. Figure 7a and Figure 7b It can be shown that corresponds to Figure 6 Examples of implementations.

[0093] refer to Figure 7a and Figure 7b The processor 240 can display a first icon 701, a second icon 702, a third icon 703, and a first button 704 related to the shooting mode in the area of ​​the display 110. The first icon 701 can be a button for telephoto photography. The second icon 702 can be a button for normal photography. The third icon 703 can be a button for ultra-wide-angle photography. The first button 704 can be a button for starting photography. The position of the first button 704 can be changed depending on whether the display is extended. For example, when the display 110 is not extended, the first button 704 can be located at the bottom of the display 110 when viewed from the front. When the display 110 is extended, the first button 704 can be located near the side surface of the display 110 when viewed from the front.

[0094] In this embodiment, processor 240 can acquire user drag-and-drop input. Processor 240 can determine whether to expand the display and whether to switch the camera module in response to the user's drag-and-drop input. After acquiring the user's drag-and-drop input, processor 240 can detect the starting point of the drag-and-drop input. Processor 240 can determine whether to switch the camera module based on the starting point position of the drag point. For example, when the starting point of the drag-and-drop input is included in the first icon 701, processor 240 can switch the currently operating camera module to a camera module including a telephoto lens.

[0095] In this embodiment, the processor 240 may determine whether to expand the display 110 based on the direction of the user's drag input. For example, when the direction of the user's drag input is the same as the expansion direction of the display 110, the processor 240 may expand the display. As another example, when the direction of the user's drag input is opposite to the expansion direction of the display 110, the processor 240 may not expand the display 110.

[0096] In an embodiment, when drag input is received in the same direction as the expansion direction of the display 110 or in a direction within 90° of the expansion direction of the display 110, the processor 240 can expand the display 110 in response to drag input from left to right. For example, when the expansion direction of the display 110 is from left to right, the processor 240 can expand the display 110 in response to drag input.

[0097] In an embodiment, when drag input is received in a direction opposite to the expansion direction of display 110 or in a direction within 90° of the expansion direction of display 110, processor 240 may not expand display 110. For example, when the expansion direction of display 110 is from left to right, processor 240 may not expand display 110 in response to drag input from right to left by the user.

[0098] Figure 8 This is a flowchart illustrating the process of expanding the display and switching the camera module based on a touch-based preview area in an electronic device according to an embodiment.

[0099] In operation 810 according to an embodiment, processor 240 may determine whether an event for extended display 110 is performed through a first preview area 901 or through a second preview area 902.

[0100] In this embodiment, the processor 240 can divide the area of ​​the output preview image of the display 110 into at least two regions. For example, the processor 240 can divide it into a first preview area 901 and a second preview area 902. The processor 240 can configure a blank area between the first preview area 901 and the second preview area 902. The processor 240 can ignore touches associated with the blank area.

[0101] In one embodiment, when an event for expanding the display 110 is executed while the user is holding a portion of the first preview area 901, the processor 240 can switch the currently operating camera module to a camera module including a macro lens. When an event for expanding the display 110 is executed while the user is holding a portion of the second preview area 902, the processor 240 can switch the currently operating camera module to a camera module including an ultra-wide-angle lens.

[0102] In an embodiment, the processor 240 may change configuration information regarding the camera module corresponding to the first preview area 901 and the camera module corresponding to the second preview area 902.

[0103] In operation 820 according to an embodiment, processor 240 may determine whether an event for expanding the display was performed via the first preview area 901 of display 110. Processor 240 may determine whether a user's touch occurred in the first preview area 901 or the second preview area 902 via a touch panel included in display 110.

[0104] In operation 830 according to an embodiment, processor 240 may switch the running camera module to the second camera module 220. Figure 8 The second camera module 220 mentioned above can be understood as a camera module that includes a macro lens and / or a telephoto lens.

[0105] In operation 840 according to an embodiment, processor 240 may switch the running camera module to a third camera module 230. Figure 8 The third camera module 230 mentioned can be understood as a camera module that includes an ultra-wide-angle lens.

[0106] Figure 9 The process of expanding the display and switching the camera module based on a touch-based preview area in an electronic device, according to an embodiment, is illustrated. Figure 9 It can be shown that corresponds to Figure 8 Examples of implementations.

[0107] In an embodiment, when the display 110 is expanded and the first preview area 901 is touched, the processor 240 can switch the operating camera module from the first camera module 210 to the second camera module 220. For example, when a user expands the display 110 by holding the first preview area 901, the processor 240 can switch the operating camera module from the first camera module 210 to the second camera module 220. The second camera module 220 may include a macro lens. The processor 240 can switch the operating camera module from the second camera module 220 to the first camera module 210 in response to a reduction in the size of the display 110.

[0108] In an embodiment, when the display 110 is expanded and the second preview area 902 is touched, the processor 240 can switch the operating camera module from the first camera module 210 to the third camera module 230. For example, when a user expands the display 110 by holding the second preview area 902, the processor 240 can switch the operating camera module from the first camera module 210 to the third camera module 230. The third camera module 230 may include an ultra-wide-angle lens. The processor 240 can switch the operating camera module from the third camera module 230 to the first camera module 210 in response to a reduction in the size of the display 110.

[0109] In this embodiment, the processor 240 can output specific indications via the display 110 for identifying camera modules corresponding to the first preview area 901 and the second preview area 902. For example, the processor 240 can display the boundary between the first preview area 901 and the second preview area 902, which are separate from each other. The processor 240 can assign specific indications to the first preview area 901 and the second preview area 902 to ensure visibility. For example, the processor 240 can display the characters "telephoto" or "macro," or related icons, in a portion of the first preview area 901. The processor 240 can display the characters "ultra-wide," or related icons, in a portion of the second preview area 902.

[0110] Figure 10 This is a flowchart illustrating a process, according to an embodiment, of changing the resolution of a preview area based on the extent of expansion of a display in an electronic device. Figure 10 This can be understood as being in Figure 4 The operation will be executed after step 430.

[0111] In operation 1010 according to an embodiment, processor 240 may configure the resolution and scale of the preview image to correspond to the extended length of display 110.

[0112] In this embodiment, the processor 240 can determine the aspect ratio of the preview area in the output preview image to correspond to the configured resolution. For example, when the configured resolution is 1920×1080, the processor 240 can determine that the aspect ratio of the preview area is 16:9. The processor 240 can output the preview image through the display 110 based on the determined aspect ratio of the preview area.

[0113] In this embodiment, the processor 240 can change the resolution according to the extent of expansion of the display 110. The processor 240 can determine the ratio of the preview area of ​​the output preview image based on the changed resolution. For example, when the resolution of the preview area is N×N as a result of the display 110 being fully expanded, the ratio of the preview area can be determined to be 1:1. The processor 240 can output a preview image through the display 110 based on the determined ratio of the preview area.

[0114] Figure 11 The process of changing the resolution of the preview area according to the extent of the display in the electronic device, according to an embodiment, is illustrated. Figure 11 It can be shown that corresponds to Figure 10 Examples of implementations.

[0115] Figure 11 The display 110 is shown in three states: 1101 when it is not extended, 1102 when it is extended to a predetermined length, and 1103 when it is fully extended.

[0116] In one embodiment, the processor 240 may change the resolution of the preview area in response to user input for expanding or shrinking the display 110 by adjusting the sliding housing 1110.

[0117] In an embodiment, when the display 110 is not extended (state 1101), the processor 240 can configure a resolution corresponding to the state where the display 110 is not extended. For example, when the horizontal / vertical ratio of the display 110 is 9:16 in the state where the display 110 is not extended, the processor 240 can control the ratio of the preview image to 9:16 to correspond to the ratio of the display 110.

[0118] In an embodiment, when the display 110 is extended by a predetermined length (state 1102), the processor 240 can configure a resolution corresponding to the state in which the display 110 is extended by the predetermined length. For example, when the display 110 is extended by a first length and the horizontal / vertical ratio of the display 110 is 3:4, the processor 240 can control the ratio of the preview image to 3:4 to correspond to the ratio of the display 110.

[0119] In an embodiment, when the display 110 is in a fully extended state 1103, the processor 240 can configure a resolution corresponding to the fully extended state of the display 110. For example, when the horizontal / vertical ratio of the display 110 is 1:1 in the fully extended state, the processor 240 can control the ratio of the preview image to 1:1 to correspond to the ratio of the display 110.

[0120] Figure 12 The process of editing an image according to the extent of the display in the electronic device, according to an embodiment, is illustrated.

[0121] In one embodiment, processor 240 can edit the image according to a horizontal scale in response to user input for expanding or shrinking display 110 by adjusting sliding housing 1210. For example, when user input is received for expanding display 110, processor 240 may increase the horizontal scale of objects included in the preview image. As another example, when user input is received for shrinking display 110, processor 240 may decrease the horizontal scale of objects included in the preview image.

[0122] In an embodiment, the horizontal scale of a portion can be reduced when a user grasps and expands a specific part of an object included in a preview image as input for expanding the display 110. For example, when the processor 240 detects that the display 110 is being expanded by grasping an area of ​​the display 110 containing the shoulder of a person in the preview image, the processor 240 can expand the person's shoulder in response to the expansion of the display 110. The process of shrinking the display 110 can be applied in the same way.

[0123] In an embodiment, processor 240 may apply aesthetic effects to an image in response to user input for expanding or shrinking display 110 by adjusting sliding housing 1210. For example, when expanding display 110 is detected by the area where the waist of a person holding display 110 is located, processor 240 may apply aesthetic effects to make the waist area appear slimmer as display 110 expands. As another example, when expanding display 110 by the area where the face of a person holding display 110 is located is detected, processor 240 may apply aesthetic effects to make the face appear freckle-free as display 110 expands.

[0124] In this embodiment, the processor 240 can adjust image-related lighting effects in response to user input. For example, upon receiving user input to expand the display 110, the processor 240 can recognize that the user wants to view the image in detail, thereby increasing the image brightness. As another example, upon receiving user input to shrink the display 110, the processor 240 can decrease the image brightness. Depending on whether the display 110 is expanded, not only the aforementioned lighting effects can be applied, but various image effects can also be applied or not applied.

[0125] Figure 13 This is a block diagram illustrating an electronic device 1301 in a network environment 1300 according to various embodiments. (Refer to...) Figure 13 In network environment 1300, electronic device 1301 can communicate with electronic device 1302 via a first network 1398 (e.g., a short-range wireless communication network), or with at least one of electronic device 1304 or server 1308 via a second network 1399 (e.g., a long-range wireless communication network). According to an embodiment, electronic device 1301 can communicate with electronic device 1304 via server 1308. According to an embodiment, electronic device 1301 may include a processor 1320, a memory 1330, an input module 1350, a sound output module 1355, a display module 1360, an audio module 1370, a sensor module 1376, an interface 1377, a connection terminal 1378, a haptic module 1379, a camera module 1380, a power management module 1388, a battery 1389, a communication module 1390, a Subscriber Identity Module (SIM) 1396, or an antenna module 1397. In some embodiments, at least one of the aforementioned components (e.g., connection terminal 1378) may be omitted from electronic device 1301, or one or more other components may be added to electronic device 1301. In some embodiments, some of the aforementioned components (e.g., sensor module 1376, camera module 1380, or antenna module 1397) may be implemented as a single integrated component (e.g., display module 1360).

[0126] Processor 1320 may run software (e.g., program 1340) to control at least one other component (e.g., hardware or software component) of electronic device 1301 connected to processor 1320, and may perform various data processing or calculations. According to one embodiment, as at least part of the data processing or calculation, processor 1320 may store commands or data received from another component (e.g., sensor module 1376 or communication module 1390) in volatile memory 1332, process the commands or data stored in volatile memory 1332, and store the result data in non-volatile memory 1334. According to embodiments, processor 1320 may include a main processor 1321 (e.g., central processing unit (CPU) or application processor (AP)) or an auxiliary processor 1323 (e.g., graphics processing unit (GPU), neural processing unit (NPU), image signal processor (ISP), sensor central processor, or communication processor (CP)) that is operationally independent of or combined with the main processor 1321. For example, when electronic device 1301 includes a main processor 1321 and an auxiliary processor 1323, the auxiliary processor 1323 may be adapted to consume less power than the main processor 1321, or may be adapted to be dedicated to a specific function. The auxiliary processor 1323 may be implemented separately from the main processor 1321, or may be implemented as part of the main processor 1321.

[0127] When the main processor 1321 is inactive (e.g., in sleep mode), the auxiliary processor 1323 (rather than the main processor 1321) can control at least some of the functions or states associated with at least one component of the electronic device 1301 (e.g., display module 1360, sensor module 1376, or communication module 1390). Alternatively, when the main processor 1321 is active (e.g., running an application), the auxiliary processor 1323 can work with the main processor 1321 to control at least some of the functions or states associated with at least one component of the electronic device 1301 (e.g., display module 1360, sensor module 1376, or communication module 1390). According to embodiments, the auxiliary processor 1323 (e.g., an image signal processor or a communication processor) may be implemented as part of another component (e.g., camera module 1380 or communication module 1390) functionally associated with the auxiliary processor 1323. According to embodiments, the auxiliary processor 1323 (e.g., a neural processing unit) may include hardware architectures dedicated to artificial intelligence model processing. Artificial intelligence models can be generated through machine learning. For example, such learning can be performed via electronic device 1301 where the artificial intelligence is executed, or via a separate server (e.g., server 1308). Learning algorithms may include, but are not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning. The artificial intelligence model may include multiple layers of artificial neural networks. The artificial neural networks may be, but are not limited to, deep neural networks (DNNs), convolutional neural networks (CNNs), recurrent neural networks (RNNs), restricted Boltzmann machines (RBMs), deep belief networks (DBNs), bidirectional recurrent deep neural networks (BRDNNs), or deep Q-networks, or combinations of two or more thereof. Additionally or optionally, the artificial intelligence model may include software structures in addition to hardware structures.

[0128] The memory 1330 may store various data used by at least one component of the electronic device 1301 (e.g., processor 1320 or sensor module 1376). The various data may include, for example, software (e.g., program 1340) and input or output data for commands associated with it. The memory 1330 may include volatile memory 1332 or non-volatile memory 1334.

[0129] The program 1340 may be stored as software in the memory 1330, and the program 1340 may include, for example, an operating system (OS) 1342, middleware 1344, or application 1346.

[0130] Input module 1350 can receive commands or data from outside electronic device 1301 (e.g., a user) that will be used by other components of electronic device 1301 (e.g., processor 1320). Input module 1350 may include, for example, a microphone, mouse, keyboard, keys (e.g., buttons), or digital pen (e.g., stylus).

[0131] The audio output module 1355 can output audio signals to the outside of the electronic device 1301. The audio output module 1355 may include, for example, a speaker or a receiver. The speaker can be used for general purposes such as playing multimedia or playing records. The receiver can be used to receive incoming calls. According to embodiments, the receiver can be implemented separately from the speaker or as part of the speaker.

[0132] Display module 1360 can visually provide information to the outside of electronic device 1301 (e.g., to a user). Display device 1360 may include, for example, a display, a holographic device, or a projector, and control circuitry for controlling a respective one of the display, holographic device, and projector. According to an embodiment, display module 1360 may include a touch sensor adapted to detect touch or a pressure sensor adapted to measure the intensity of the force caused by touch.

[0133] The audio module 1370 can convert sound into electrical signals and vice versa. According to an embodiment, the audio module 1370 can obtain sound via the input module 1350, or output sound via the sound output module 1355 or headphones of an external electronic device (e.g., electronic device 1302) that is directly (e.g., wired) or wirelessly connected to the electronic device 1301.

[0134] Sensor module 1376 can detect the operating state of electronic device 1301 (e.g., power or temperature) or the environmental state outside electronic device 1301 (e.g., user state), and then generate an electrical signal or data value corresponding to the detected state. According to embodiments, sensor module 1376 may include, for example, a gesture sensor, gyroscope sensor, atmospheric pressure sensor, magnetic sensor, accelerometer, grip sensor, proximity sensor, color sensor, infrared (IR) sensor, biometric sensor, temperature sensor, humidity sensor, or illuminance sensor.

[0135] Interface 1377 may support one or more specific protocols used to enable electronic device 1301 to connect directly (e.g., wired) or wirelessly to external electronic device (e.g., electronic device 1302). According to embodiments, interface 1377 may include, for example, a High Definition Multimedia Interface (HDMI), a Universal Serial Bus (USB) interface, a Secure Digital Card (SD) interface, or an audio interface.

[0136] Connection 1378 may include a connector, via which electronic device 1301 may be physically connected to an external electronic device (e.g., electronic device 1302). According to embodiments, connection 1378 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0137] The haptic module 1379 can convert electrical signals into mechanical stimulation (e.g., vibration or motion) or electrical stimulation that can be recognized by a user through his touch or kinesthesia. According to an embodiment, the haptic module 1379 may include, for example, a motor, a piezoelectric element, or an electrical stimulator.

[0138] Camera module 1380 can capture still or moving images. According to an embodiment, camera module 1380 may include one or more lenses, an image sensor, an image signal processor, or a flash.

[0139] The power management module 1388 manages the power supply to the electronic device 1301. According to an embodiment, the power management module 1388 may be implemented as at least part of, for example, a power management integrated circuit (PMIC).

[0140] Battery 1389 can power at least one component of electronic device 1301. According to an embodiment, battery 1389 may include, for example, a non-rechargeable primary battery, a rechargeable rechargeable battery, or a fuel cell.

[0141] Communication module 1390 can support the establishment of a direct (e.g., wired) or wireless communication channel between electronic device 1301 and external electronic devices (e.g., electronic device 1302, electronic device 1304, or server 1308), and perform communication via the established communication channel. Communication module 1390 may include one or more communication processors capable of operating independently of processor 1320 (e.g., application processor (AP)) and support direct (e.g., wired) or wireless communication. According to embodiments, communication module 1390 may include wireless communication module 1392 (e.g., cellular communication module, short-range wireless communication module, or Global Navigation Satellite System (GNSS) communication module) or wired communication module 1394 (e.g., local area network (LAN) communication module or power line communication (PLC) module). One of these communication modules can communicate with an external electronic device via a first network 1398 (e.g., a short-range communication network such as Bluetooth, Wi-Fi Direct, or Infrared Data Association (IrDA)) or a second network 1399 (e.g., a long-range communication network such as a traditional cellular network, 5G network, next-generation communication network, the Internet, or a computer network (e.g., a LAN or a wide area network (WAN))). These various types of communication modules can be implemented as a single component (e.g., a single chip) or as multiple components (e.g., multiple chips) that are separate from each other. The wireless communication module 1392 can identify and verify the electronic device 1301 in the communication network (such as the first network 1398 or the second network 1399) using user information (e.g., the International Mobile Subscriber Identity (IMSI)) stored in the user identification module 1396.

[0142] Wireless communication module 1392 can support 5G networks following 4G networks and next-generation communication technologies (such as new radio (NR) access technologies). NR access technologies can support enhanced mobile broadband (eMBB), massive machine-type communications (mMTC), or ultra-reliable low-latency communications (URLLC). Wireless communication module 1392 can support high-frequency bands (e.g., millimeter-wave bands) to achieve, for example, high data transmission rates. Wireless communication module 1392 can support various technologies used to ensure performance in high-frequency bands, such as, for example, beamforming, massive MIMO, full-dimensional MIMO (FD-MIMO), array antennas, analog beamforming, or massive antennas. Wireless communication module 1392 can support various requirements specified in electronic device 1301, external electronic device (e.g., electronic device 1304), or network system (e.g., second network 1399). According to an embodiment, the wireless communication module 1392 may support peak data rates (e.g., 20 Gbps or greater) for implementing eMBB, lost coverage (e.g., 164 dB or less) for implementing mMTC, or U-plane latency (e.g., 0.5 ms or less for each of the downlink (DL) and uplink (UL), or 1 ms or less round trip) for implementing URLLC.

[0143] Antenna module 1397 can transmit or receive signals or power to or from the exterior of electronic device 1301 (e.g., external electronic device). According to an embodiment, antenna module 1397 may include an antenna comprising a radiating element formed of a conductive material or conductive pattern formed in or on a substrate (e.g., a printed circuit board (PCB)). According to an embodiment, antenna module 1397 may include multiple antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication scheme used in a communication network (such as a first network 1398 or a second network 1399) can be selected from the multiple antennas by, for example, communication module 1390 (e.g., wireless communication module 1392). Signals or power can then be transmitted or received between communication module 1390 and the external electronic device via the selected at least one antenna. According to an embodiment, additional components besides the radiating element (e.g., a radio frequency integrated circuit (RFIC)) may be additionally incorporated into antenna module 1397.

[0144] According to various embodiments, antenna module 1397 can form a millimeter-wave antenna module. According to embodiments, the millimeter-wave antenna module may include a printed circuit board, a radio frequency integrated circuit (RFIC), and multiple antennas (e.g., an array antenna), wherein the RFIC is disposed on or adjacent to a first surface (e.g., a bottom surface) of the printed circuit board and is capable of supporting a specified high-frequency band (e.g., a millimeter-wave band), and the multiple antennas are disposed on or adjacent to a second surface (e.g., a top or side surface) of the printed circuit board and are capable of transmitting or receiving signals in the specified high-frequency band.

[0145] At least some of the aforementioned components can be interconnected and communicate signals (e.g., commands or data) between them via an inter-peripheral communication scheme (e.g., bus, general purpose input / output (GPIO), serial peripheral interface (SPI), or mobile industrial processor interface (MIPI)).

[0146] According to an embodiment, commands or data can be sent or received between electronic device 1301 and external electronic device 1304 via server 1308 connected to a second network 1399. Each of electronic device 1302 or electronic device 1304 can be a device of the same type as electronic device 1301, or a device of a different type. According to an embodiment, all or some operations that would run on electronic device 1301 can be run on one or more of external electronic devices 1302, external electronic devices 1304, or server 1308. For example, if electronic device 1301 is required to automatically perform a function or service, or is required to perform a function or service in response to a request from a user or another device, electronic device 1301 may request the one or more external electronic devices to perform at least a portion of the function or service instead of running the function or service, or electronic device 1301 may request the one or more external electronic devices to perform at least a portion of the function or service in addition to running the function or service. Upon receiving the request, the one or more external electronic devices may perform at least a portion of the requested function or service, or perform additional functions or services related to the request, and transmit the result of the execution to electronic device 1301. Electronic device 1301 may provide the result as at least a partial response to the request, with or without further processing. For this purpose, technologies such as cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing may be used. Electronic device 1301 may use, for example, distributed computing or mobile edge computing to provide ultra-low latency services. In another embodiment, external electronic device 1304 may include an Internet of Things (IoT) device. Server 1308 may be an intelligent server using machine learning and / or neural networks. According to an embodiment, external electronic device 1304 or server 1308 may be included in a second network 1399. Electronic device 1301 may be applied to intelligent services based on 5G communication technology or IoT-related technologies (e.g., smart homes, smart cities, smart cars, or healthcare).

[0147] The electronic device according to various embodiments can be one of a variety of types of electronic devices. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. According to embodiments of this disclosure, the electronic device is not limited to those described above.

[0148] It should be understood that the various embodiments of this disclosure and the terminology used therein are not intended to limit the technical features set forth herein to the specific embodiments, but rather to include various changes, equivalents, or substitutions to the respective embodiments. In the description of the drawings, similar reference numerals may be used to refer to similar or related elements. It will be understood that nouns in the singular form corresponding to terms may include one or more things unless the relevant context clearly indicates otherwise. As used herein, each of the phrases such as “A or B,” “at least one of A and B,” “at least one of A or B,” “A, B, or C,” “at least one of A, B, and C,” and “at least one of A, B, or C” may include any one or all possible combinations of the items enumerated together with the corresponding phrase among the plurality of phrases. As used herein, terms such as “first” and “second” or “first” and “second” may be used to simply distinguish the respective component from another component and do not limit the component in other respects (e.g., importance or order). It will be understood that, whether the terms “operably” or “communically” are used or not, if an element (e.g., a first element) is referred to as “combined with another element (e.g., a second element),” “combined to another element (e.g., a second element),” “connected to another element (e.g., a second element),” or “attached to another element (e.g., a second element)”, it means that the first element can be directly (e.g., wiredly) connected to the second element, wirelessly connected to the second element, or connected to the second element via a third element.

[0149] As used in connection with various embodiments of this disclosure, the term "module" may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with other terms (e.g., "logic," "logic block," "part," or "circuit"). A module may be a single integrated component adapted to perform one or more functions, or the smallest unit or part of such a single integrated component. For example, according to embodiments, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0150] The various embodiments set forth herein can be implemented as software (e.g., program 1340) containing one or more instructions readable by a machine (e.g., electronic device 1301) stored in a storage medium (e.g., internal memory 1336 or external memory 1338). For example, under the control of a processor, the processor (e.g., processor 1320) of the machine (e.g., electronic device 1301) can invoke and execute at least one of the one or more instructions stored in the storage medium, with or without the use of one or more other components. This enables the machine to operate to perform at least one function according to the invoked at least one instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. Machine-readable storage media may be provided in the form of non-transitory storage media. The term "non-transitory" means only that the storage medium is a tangible device and does not include signals (e.g., electromagnetic waves), but this term does not distinguish between data being stored semi-permanently in the storage medium and data being temporarily stored in the storage medium.

[0151] According to embodiments, methods according to various embodiments of this disclosure may be included and provided in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., a compact disk read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) via an app store (e.g., the Play Store™), or may be distributed directly between two user devices (e.g., smartphones) (e.g., downloaded or uploaded). If distributed online, at least a portion of the computer program product may be temporarily generated, or at least a portion of the computer program product may be stored at least temporarily in a machine-readable storage medium (such as the memory of a manufacturer's server, an app store's server, or a forwarding server).

[0152] According to various embodiments, each of the above-described components (e.g., a module or program) may include a single entity or multiple entities, and some of the multiple entities may be separately disposed in different components. According to various embodiments, one or more of the above-described components may be omitted, or one or more other components may be added. Optionally or additionally, multiple components (e.g., modules or programs) may be integrated into a single component. In this case, according to various embodiments, the integrated component may still perform the one or more functions of each of the multiple components in the same or similar manner as the corresponding component of the multiple components performed one or more functions before integration. According to various embodiments, the operations performed by a module, program, or other component may be performed sequentially, in parallel, repeatedly, or heuristically, or one or more of the operations may be run in a different order or omitted, or one or more other operations may be added.

[0153] Figure 14 This is a block diagram 1400 illustrating a camera module 1380 according to various embodiments. (Refer to...) Figure 14 Camera module 1380 may include lens assembly 1410, flash 1420, image sensor 1430, image stabilizer 1440, memory 1450 (e.g., buffer memory), or image signal processor 1460. Lens assembly 1410 may capture light emitted or reflected from an object whose image is to be captured. Lens assembly 1410 may include one or more lenses. According to embodiments, camera module 1380 may include multiple lens assemblies 1410. In this case, camera module 1380 may form, for example, a dual-camera, a 360-degree camera, or a spherical camera. Some of the multiple lens assemblies 1410 may have the same lens properties (e.g., angle of view, focal length, autofocus, f-number, or optical zoom), or at least one lens assembly may have one or more lens properties that are different from the lens properties of the other lens assemblies. Lens assembly 1410 may include, for example, a wide-angle lens or a telephoto lens.

[0154] Flash 1420 is capable of emitting light, wherein the emitted light is used to enhance light reflected from an object. According to an embodiment, flash 1420 may include one or more light-emitting diodes (LEDs) (e.g., red-green-blue (RGB) LEDs, white LEDs, infrared (IR) LEDs, or ultraviolet (UV) LEDs) or xenon lamps. Image sensor 1430 acquires an image corresponding to an object by converting light emitted or reflected from an object and transmitted through lens assembly 1410 into an electrical signal. According to an embodiment, image sensor 1430 may include one image sensor selected from a plurality of image sensors having different properties (e.g., an RGB sensor, a black-and-white (BW) sensor, an IR sensor, or a UV sensor), a plurality of image sensors having the same properties, or a plurality of image sensors having different properties. Each image sensor included in image sensor 1430 may be implemented using, for example, a charge-coupled device (CCD) sensor or a complementary metal-oxide-semiconductor (CMOS) sensor.

[0155] Image stabilizer 1440 can move image sensor 1430 or at least one lens included in lens assembly 1410 in a specific direction, or control the operability properties of image sensor 1430 (e.g., adjust readout timing) in response to movement of camera module 1380 or electronics 1301 including camera module 1380. This allows compensation for at least a portion of the negative effects (e.g., image blur) caused by movement of the image being captured. According to embodiments, image stabilizer 1440 can use a gyroscope sensor (not shown) or accelerometer sensor (not shown) disposed within or outside camera module 1380 to sense such movement of camera module 1380 or electronics 1301. According to embodiments, image stabilizer 1440 can be implemented as, for example, an optical image stabilizer.

[0156] Memory 1450 may at least temporarily store at least a portion of the images acquired via image sensor 1430 for subsequent image processing tasks. For example, if multiple images are captured rapidly or if image capture is delayed due to shutter lag, the acquired raw images (e.g., Bayer pattern images, high-resolution images) may be stored in memory 1450, and their corresponding copy images (e.g., low-resolution images) may be previewed via display module 1360. Then, if specified conditions are met (e.g., by user input or system commands), at least a portion of the raw images stored in memory 1450 may be acquired and processed by, for example, image signal processor 1460. According to embodiments, memory 1450 may be configured as at least a portion of memory 1330, or memory 1450 may be configured as a separate memory operating independently of memory 1330.

[0157] Image signal processor 1460 can perform one or more image processing operations on an image acquired via image sensor 1430 or an image stored in memory 1450. The one or more image processing operations may include, for example, depth map generation, 3D modeling, panorama generation, feature point extraction, image compositing, or image compensation (e.g., noise reduction, resolution adjustment, brightness adjustment, blurring, sharpening, or softening). Alternatively or additionally, image signal processor 1460 can perform control (e.g., exposure time control or readout timing control) on at least one component included in camera module 1380 (e.g., image sensor 1430). The image processed by image signal processor 1460 can be stored back in memory 1450 for further processing, or the image can be provided to external components outside camera module 1380 (e.g., memory 1330, display module 1360, electronic device 1302, electronic device 1304, or server 1308). According to embodiments, the image signal processor 1460 can be configured as at least a part of the processor 1320, or the image signal processor 1460 can be configured as a separate processor that operates independently of the processor 1320. If the image signal processor 1460 is configured as a separate processor from the processor 1320, the processor 1320 can display at least one image processed by the image signal processor 1460 as is via the display module 1360, or the at least one image can be displayed after further processing.

[0158] According to an embodiment, the electronic device 1301 may include a plurality of camera modules 1380 with different attributes or functions. In this case, at least one of the plurality of camera modules 1380 may form, for example, a wide-angle camera, and at least another of the plurality of camera modules 1380 may form a telephoto camera. Similarly, at least one of the plurality of camera modules 1380 may form, for example, a front-facing camera, and at least another of the plurality of camera modules 1380 may form a rear-facing camera.

[0159] In various embodiments, the electronic device 100 may include a flexible display 110, a first camera module 210, a second camera module 220, and at least one processor. This at least one processor (e.g., Figure 2 The processor 240 in the first camera module 210 and the second camera module 220 can execute a first application associated with the first camera module 210 and the second camera module 220. In response to the execution of the first application, at least one processor (e.g., Figure 2The processor 240 in the flexible display 110 can output a first preview image based on first image data acquired by the first camera module 210 in a first area of ​​the flexible display 110. Simultaneously with outputting the first preview image on the flexible display 110, the at least one processor (e.g., Figure 2 The processor 240 in the process can detect events for extending the flexible display 110. In response to the detection of the event, at least one processor (e.g., Figure 2 The processor 240 can output a second preview image based on second image data acquired by the second camera module 220 in a second region that is extended compared to the first region of the flexible display 110.

[0160] In an embodiment, in addition to the first camera module 210 and the second camera module 220, the electronic device 100 may also include a third camera module 230. The at least one processor (e.g., Figure 2 The processor 240 in the image can detect at least one object or background using the first image data. The at least one processor (e.g., Figure 2 The processor 240 in the middle can analyze at least one of the detected objects or background. The at least one processor (e.g., Figure 2 The processor 240 can output a second preview image based on the second image data acquired by the second camera module 220, or output a third preview image based on the third image data acquired by the third camera module 230.

[0161] In an embodiment, at least one processor (e.g., Figure 2 The processor 240 in the first camera module 210 may include a wide-angle lens, the second camera module 220 may include at least one of a macro lens or a telephoto lens, and the third camera module 230 may include an ultra-wide-angle lens.

[0162] In an embodiment, an event may be generated by at least one of an external force pulling at least one housing of the electronic device, a user drag input, or a user touch input.

[0163] In an embodiment, at least one processor (e.g., Figure 2 The processor 240 in the module can acquire the user's drag input and determine the starting point of the drag input. When the starting point of the drag input is included in an icon associated with the ultra-wide-angle camera module, at least one processor (e.g., Figure 2 The processor 240 in the image can output a preview image based on image data acquired by the ultra-wide-angle camera module on a flexible display. When the starting point of the drag input is included in an icon associated with a macro or telephoto camera module, at least one processor (e.g., Figure 2The processor 240 in the middle can output a preview image on a flexible display based on image data acquired through a macro or telephoto camera module.

[0164] In an embodiment, at least one processor (e.g., Figure 2 The processor 240 can acquire the user's drag input, and can extend the flexible display 110 when the direction of the drag input is the same as the extension direction of the flexible display 110.

[0165] In an embodiment, when the operating camera module switches from the first camera module 210 to the second camera module 220, at least one processor (e.g., Figure 2 The processor 240 in the middle can output an intermediate image between the first preview image and the second preview image in N frames.

[0166] In an embodiment, when the flexible display 110 is expanded while a user touch is detected in the first preview area of ​​the preview area of ​​the output preview image, at least one processor (e.g., Figure 2 The processor 240 in the flexible display can output a preview image based on image data acquired by the second camera module 220. When the flexible display 110 is extended while a user touch is detected in a second preview area other than the first preview area, at least one processor (e.g., Figure 2 The processor 240 in the middle can output a preview image on the flexible display based on image data acquired through the third camera module 230.

[0167] In an embodiment, at least one processor (e.g., Figure 2 The processor 240 in the image can adjust the scale of the preview area of ​​the output preview image based on the extent of expansion of the flexible display 110.

[0168] In an embodiment, at least one processor (e.g., Figure 2 The processor 240 in the flexible display 110 can perform image editing on objects included in the preview image based on the extent of the flexible display 110.

[0169] In various embodiments, a method for operating electronic device 100 may include the following operations: executing a first application associated with a first camera module 210 and a second camera module 220; in response to the execution of the first application, outputting a first preview image based on first image data acquired by the first camera module 210 in a first region of flexible display 110; while outputting the first preview image on flexible display 110, detecting an event for expanding flexible display 110; and in response to the detection of the event, outputting a second preview image based on second image data acquired by the second camera module 220 in a second region expanded compared to the first region of flexible display 110.

[0170] In an embodiment, a method for operating an electronic device 100 may include the following operations: detecting at least one of an object or a background using first image data; analyzing the detected object or background; based on the analysis, outputting a second preview image based on second image data acquired by a second camera module 220; or outputting a third preview image based on third image data acquired by a third camera module 230.

[0171] In an embodiment, a method for operating the electronic device 100 may include the following operations: acquiring a user's drag input, determining the starting point of the drag input, outputting a preview image on the flexible display 110 based on image data acquired by the ultra-wide-angle camera module when the starting point of the drag input is included in an icon associated with the ultra-wide-angle camera module, and outputting a preview image on the flexible display 110 based on image data acquired by the macro or telephoto camera module when the starting point of the drag input is included in an icon associated with the macro or telephoto camera module.

[0172] In an embodiment, the method for operating the electronic device 100 may include acquiring drag input from a user, and expanding the flexible display 110 when the direction of the drag input is the same as the expansion direction of the flexible display 110.

[0173] In an embodiment, a method for operating an electronic device 100 may include the following operations: when the flexible display 110 is expanded while a user touch is detected in a first preview area of ​​the preview area of ​​the output preview image, a preview image based on image data acquired by a second camera module 220 is output on the flexible display; and when the flexible display 110 is expanded while a user touch is detected in a second preview area other than the first preview area of ​​the preview area, a preview image based on image data acquired by a third camera module 230 is output on the flexible display 110.

[0174] In various embodiments, the electronic device 100 may include a display 110, a plurality of camera modules including a first camera module 210, a second camera module 220, and a third camera module 230, and at least one processor electrically connected to the display 110 and the plurality of camera modules. This at least one processor (e.g., Figure 2 The processor 240 in the middle can execute a first application associated with multiple camera modules. In response to the execution of the first application, at least one processor (e.g., Figure 2 The processor 240 in the flexible display can output a first preview image based on first image data acquired by the first camera module 210 in a first area of ​​the flexible display. Simultaneously with outputting the first preview image on the flexible display 110, at least one processor (e.g., Figure 2 The processor 240 in the display can detect events used to extend the display 110. At least one processor (e.g., Figure 2 The processor 240 in the middle can activate the second camera module 220 or the third camera module 230 in response to detecting an event. At least one processor (e.g., Figure 2 The processor 240 can output a second preview image based on second image data acquired by the second camera module 220, or a third preview image based on third image data acquired by the third camera module 230, in a second region that is extended compared to the first region of the display 110.

[0175] In an embodiment, at least one processor (e.g., Figure 2 The processor 240 in the middle can analyze the distance between the electronic device 100 and the object, the focus state and the first image data, and can determine whether to output a second preview image or a third preview image in the second area based on the analysis.

[0176] In an embodiment, at least one processor (e.g., Figure 2 The processor 240 in the module can acquire the user's drag input and determine the starting point of the drag input. When the starting point of the drag input is included in an icon associated with the ultra-wide-angle camera module, at least one processor (e.g., Figure 2 The processor 240 in the flexible display can output a preview image based on image data acquired by the ultra-wide-angle camera module on the flexible display, and at least one processor can output a preview image based on image data acquired by the macro or telephoto camera module on the flexible display when the starting point of the drag input is included in an icon associated with the macro or telephoto camera module.

[0177] In an embodiment, at least one processor (e.g., Figure 2The processor 240 can acquire the user's drag input and expand the display 110 when the direction of the drag input is the same as the expansion direction of the display 110.

[0178] In an embodiment, when the display 110 is expanded while a user touch is detected in the first preview area of ​​the preview area of ​​the output preview image, at least one processor (e.g., Figure 2 The processor 240 in the flexible display 110 can output a preview image based on image data acquired by the second camera module 220. When the display 110 is extended while a user touch is detected in a second preview area outside the first preview area, at least one processor (e.g., Figure 2 The processor 240 in the middle can output a preview image on the display based on the image data acquired through the third camera module 230.

Claims

1. An electronic device comprising: Flexible displays that can slide and expand; The first camera module has a first-person perspective; The second camera module has a second perspective that is different from the first perspective; as well as At least one processor is electrically connected to the flexible display, the first camera module, and the second camera module. Wherein, the at least one processor is configured to: Execute the first application associated with the first camera module and the second camera module; In response to the execution of the first application, a first preview image based on first image data acquired through the first camera module is output in a first area of ​​the flexible display; While outputting the first preview image on the flexible display, user input for extending the flexible display is detected; and Based on the position of the user input on the flexible display used to expand the flexible display, a second preview image based on second image data acquired by the second camera module is output in a second region that is expanded compared to the first region of the flexible display.

2. The electronic device according to claim 1 further includes a third camera module. in, The at least one processor is configured to: Detect at least one object or background using the first image data; Analyze at least one of the detected objects or background; as well as Based on the analysis, a second preview image based on the second image data acquired by the second camera module is output, or a third preview image based on the third image data acquired by the third camera module is output.

3. The electronic device according to claim 2, wherein, The first camera module includes a wide-angle lens, the second camera module includes at least one of a macro lens or a telephoto lens, and the third camera module includes an ultra-wide-angle lens.

4. The electronic device according to claim 1, wherein, The user input is generated by at least one of the following: an external force pulling on at least one housing of the electronic device, a user drag input, or a user touch input.

5. The electronic device according to claim 1, wherein, The at least one processor is configured to: Get the user's drag-and-drop input; Determine the starting point for drag-and-drop input; When the starting point of the drag-and-drop input is included in an icon associated with the ultra-wide-angle camera module, a preview image based on image data acquired through the ultra-wide-angle camera module is output on the flexible display. as well as When the starting point of the drag-and-drop input is included in an icon associated with the macro or telephoto camera module, a preview image based on image data acquired through the macro or telephoto camera module is output on a flexible display.

6. The electronic device according to claim 1, wherein, The at least one processor is configured to: Get the user's drag-and-drop input; and Expand the display when the direction of drag input is the same as the expansion direction of the flexible display.

7. The electronic device according to claim 1, wherein, The at least one processor is configured to output an intermediate image between the first preview image and the second preview image over N frames when the operating camera module is switched from the first camera module to the second camera module.

8. The electronic device according to claim 1, wherein, The at least one processor is configured to: When the flexible display is expanded in the first preview area of ​​the preview area of ​​the output preview image, a preview image based on image data acquired by the second camera module is output on the flexible display. as well as When the flexible display is expanded to a second preview area outside the first preview area, a preview image based on image data acquired by the third camera module is output on the flexible display.

9. The electronic device according to claim 1, wherein, The at least one processor is configured to adjust the scale of the preview area of ​​the output preview image based on the extent of the flexible display's expansion.

10. The electronic device according to claim 1, wherein, The at least one processor is configured to perform image editing on objects included in the preview image, based on the extent of expansion of the flexible display.

11. A method for operating an electronic device, the method comprising: Execute a first application associated with a first camera module and a second camera module, wherein the first camera module has a first viewpoint and the second camera module has a second viewpoint different from the first viewpoint of the first camera module; In response to the execution of the first application, a first preview image based on first image data acquired through the first camera module is output in a first area of ​​the flexible display; While outputting the first preview image on the flexible display, user input via sliding to extend the flexible display is detected. as well as Based on the position of the user input on the flexible display used to expand the flexible display, a second preview image based on second image data acquired by the second camera module is output in a second region that is expanded compared to the first region of the flexible display.

12. The method of claim 11, comprising: Detect at least one object or background using the first image data; Analyze at least one of the detected objects or background; as well as Based on the analysis, a second preview image based on the second image data acquired by the second camera module is output, or a third preview image based on the third image data acquired by the third camera module is output. The first camera module includes a wide-angle lens, the second camera module includes at least one of a macro lens or a telephoto lens, and the third camera module includes an ultra-wide-angle lens.

13. The method of claim 11, comprising: Get the user's drag-and-drop input; Determine the starting point for drag-and-drop input; When the starting point of the drag-and-drop input is included in an icon associated with the ultra-wide-angle camera module, a preview image based on image data acquired through the ultra-wide-angle camera module is output on the flexible display. as well as When the starting point of the drag-and-drop input is included in an icon associated with the macro or telephoto camera module, a preview image based on image data acquired through the macro or telephoto camera module is output on a flexible display.

14. The method of claim 11, comprising: Get the user's drag-and-drop input; as well as Expand the display when the direction of drag input is the same as the expansion direction of the flexible display.

15. The method of claim 11, comprising: When the flexible display is expanded in the first preview area of ​​the preview area of ​​the output preview image, a preview image based on image data acquired by the second camera module is output on the flexible display. as well as When the flexible display is expanded to a second preview area outside the first preview area, a preview image based on image data acquired by the third camera module is output on the flexible display.

Images