Electronic device and control method therefor

Abstract

An electronic device is disclosed. One or more processors, when instructions are individually or collectively executed, identify a first line connecting a first point corresponding to a first object included in a first captured image acquired through a camera and a second point corresponding to a second object included in the first captured image, identify a second line connecting the first point and the second point in a second captured image acquired through the camera, and control a display to output the second captured image on the basis of rotation information of the display acquired on the basis of the first line and the second line.

Description

Electronic device and method of controlling the same

[0001] The present disclosure relates to an electronic device and a method for controlling the same.

[0002] Thanks to advancements in electronic technology, technology is being developed and disseminated that utilizes various sensors equipped within electronic devices to determine whether a device is rotating, processes captured images based on the rotation status, and displays the results.

[0003] However, in cases where there is no sensor within the electronic device to detect whether the device is rotating, limitations have been pointed out regarding the method of processing images captured while the device is rotating.

[0004] An electronic device according to one or more embodiments of the present disclosure comprises a display, a camera, a memory for storing instructions, and one or more processors including processing circuitry.

[0005] According to one or more embodiments, when the instructions are executed individually or collectively, the one or more processors identify a first line connecting a first point corresponding to a first object included in a first captured image acquired through the camera and a second point corresponding to a second object included in the first captured image, identify a second line connecting the first point and the second point in a second captured image acquired through the camera, and control the display to output the second captured image based on rotation information of the display acquired based on the first line and the second line.

[0006] According to one or more embodiments, when the instructions are executed individually or collectively, the one or more processors identify a plurality of objects in a plurality of first captured images acquired through the camera, identify a plurality of points corresponding to each of the plurality of objects, identify a plurality of static objects among the plurality of objects based on location information of the plurality of points corresponding to each of the plurality of objects in the plurality of first captured images, and identify the identified plurality of static objects as the first object and the second object.

[0007] According to one or more embodiments, the one or more processors acquire rotation information of the display based on the angle formed by the first line and the second line when the instructions are executed individually or collectively, and the rotation information of the display includes at least one of the rotation direction, rotation angle, and rotation acceleration of the electronic device.

[0008] According to one or more embodiments, when the instructions are executed individually or collectively, the one or more processors identify a plurality of candidate points in the first captured image such that the distance from each of the first object and the second object is greater than or equal to a threshold distance, identify at least one target point among the plurality of candidate points based on the sum of the distance between each candidate point among the plurality of candidate points and the first point and the distance between each candidate point and the second point, and identify whether the display is rotated based on the target point identified in the first captured image and the target point identified in the second captured image.

[0009] According to one or more embodiments, when the instructions are executed individually or collectively, the one or more processors identify a first color of the target point identified in the first captured image and a second color of the target point identified in the second captured image, and if the difference between the first color and the second color is less than a preset value, identify that at least one of the first object and the second object has rotated, and if the difference between the first color and the second color is greater than or equal to the preset value, identify that the display has rotated.

[0010] According to one or more embodiments, when the instructions are executed individually or collectively, the one or more processors identify the first point and the second point within an area excluding a pre-set margin area based on the boundary of the first captured image.

[0011] According to one or more embodiments, when the instructions are executed individually or collectively, the one or more processors identify a first image corresponding to a central area of ​​the first captured image, identify a second image corresponding to the identified first image in the second captured image, and control the display to output a UI (User Interface) corresponding to the position adjustment of the camera based on the rotation information and the position of the second image in the second captured image.

[0012] According to one or more embodiments, the first point and the second point are identified based on at least one of the type of the first object and the second object or the distance between points.

[0013] According to one or more embodiments, the one or more processors control the display to output a preset screen corresponding to the rotation of the display when the rotational acceleration of the display exceeds a preset value when the instructions are executed individually or collectively.

[0014] A method for controlling an electronic device according to one or more embodiments of the present disclosure comprises: identifying a first line connecting a first point corresponding to a first object included in a first captured image obtained through a camera and a second point corresponding to a second object included in the first captured image; identifying a second line connecting the first point and the second point in a second captured image obtained through the camera; and outputting the second captured image based on rotation information of a display obtained based on the first line and the second line.

[0015] A non-transient computer-readable storage medium storing computer instructions that cause the electronic device to perform an operation when executed by a processor of an electronic device according to one or more embodiments of the present disclosure, wherein the operation comprises: identifying a first object and a second object in a first captured image acquired through a camera; identifying a first point corresponding to the first object and a second point corresponding to the second object when the first point and the second point are identified, identifying a first line connecting the first point and the second point; identifying a second line connecting the first point and the second point in a second captured image acquired through the camera; identifying rotation information of a display based on the first line and the second line; and displaying the second captured image based on the rotation information.

[0016] FIG. 1 is a drawing for explaining the operation of an electronic device according to one or more embodiments.

[0017] FIG. 2a is a block diagram illustrating the configuration of an electronic device according to one or more embodiments.

[0018] FIG. 2b is a block diagram illustrating the detailed configuration of an electronic device according to one or more embodiments.

[0019] FIG. 3 is a diagram illustrating a point identification process of an electronic device according to one or more embodiments.

[0020] FIG. 4 is a diagram illustrating a static object identification process of an electronic device according to one or more embodiments.

[0021] FIGS. 5 and 6 are drawings for illustrating a point identification process for identifying a first line and a second line of an electronic device according to one or more embodiments.

[0022] FIG. 7 is a drawing for explaining the process of identifying a first line and a second line of an electronic device according to one or more embodiments.

[0023] FIG. 8 is a drawing for explaining a method for identifying rotation information of a display according to one or more embodiments.

[0024] FIGS. 9 and FIGS. 10 are drawings for explaining the process of identifying whether a display is rotated according to one or more embodiments.

[0025] FIG. 11 is a diagram illustrating a process for identifying whether a display is rotated based on the color value of a target point according to one or more embodiments.

[0026] FIG. 12 is a drawing for explaining a UI display process corresponding to camera position adjustment of an electronic device according to one or more embodiments.

[0027] FIG. 13 is a drawing for explaining the black screen display process of an electronic device according to one or more embodiments.

[0028] FIG. 14 is a flowchart illustrating the overall operation of an electronic device according to one or more embodiments.

[0029] FIG. 15 is a flowchart illustrating the operation of an electronic device according to one or more embodiments.

[0030] The terms used in the various embodiments of this Disclosure have been selected to be as widely used and general as possible, taking into account their functions within this Disclosure; however, these terms may vary depending on the intent of those skilled in the art, case law, the emergence of new technologies, etc. Additionally, in specific cases, terms have been selected at the applicant's discretion, and in such cases, their meanings will be described in detail in the relevant description section of this Disclosure. Therefore, terms used in this Disclosure should be defined not merely by their names, but based on their meanings and the overall content of this Disclosure.

[0031] In the present disclosure, expressions such as “have,” “may have,” “include,” or “may include” indicate the presence of such features (e.g., numerical values, functions, actions, or components such as parts) and do not exclude the presence of additional features.

[0032] The expression "at least one of A or / and B" should be understood as representing either "A" or "B" or "A and B".

[0033] Expressions such as "first," "second," "first," or "second" used in this disclosure may modify various components regardless of order and / or importance, and are used only to distinguish one component from another and do not limit said components.

[0034] Where it is stated that a component (e.g., Component 1) is "(operatively or communicatively) coupled with / to" or "connected to" another component (e.g., Component 2), it should be understood that the component may be directly connected to the other component or connected through the other component (e.g., Component 3).

[0035] The singular expression includes the plural expression unless the context clearly indicates otherwise. In this disclosure, terms such as “comprising” or “consisting of” are intended to specify the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0036] In the present disclosure, a "module" or "part" performs at least one function or operation and may be implemented in hardware or software, or a combination of hardware and software. Additionally, a plurality of "modules" or a plurality of "parts" may be integrated into at least one module and implemented by at least one processor (not shown), except for a "module" or "part" that needs to be implemented in specific hardware.

[0037] In the present disclosure, the term "user" may refer to a person using an electronic device or a device used by such person.

[0038] An embodiment of the present disclosure will be described in more detail below with reference to the attached drawings.

[0039] FIG. 1 is a drawing for explaining the operation of an electronic device according to one or more embodiments.

[0040] According to one embodiment, the electronic device (100) may be implemented as various types of display devices such as a TV, monitor, kiosk, tablet PC, digital photo frame, mobile phone, LFD (large format display), Digital Signage, DID (Digital Information Display), video wall, projector display, etc. However, depending on the case, it may be implemented as a video processing device (e.g., set-top box, one connected box) that is connected to the display device to provide video.

[0041] According to one embodiment, the electronic device (100) may not include a display. The electronic device (100) may be connected to an external display device and may transmit images or content stored in the electronic device (100) to the external display device.

[0042] The electronic device (100) can transmit an image or content to an external display device along with a control signal for controlling the display of the image or content on the external display device. Here, the external display device may be connected to the electronic device (100) through a communication interface or an input / output interface. For example, the electronic device (100) may not include a display, such as a Set Top Box (STB).

[0043] The electronic device (100) may include only a small display capable of displaying simple information such as text information. The electronic device (100) may transmit images or content to an external display device via a wired or wireless connection through a communication interface, or transmit them to an external display device via an input / output interface.

[0044] According to one example, a display provided in an electronic device (100) may be implemented as a device capable of physical rotation but without a sensor to detect whether rotation has occurred. Accordingly, the electronic device (100) may identify whether rotation has occurred based on an image taken of the front of the electronic device (100), and if rotation has occurred, rotate and display the displayed image.

[0045] According to one example, the electronic device (100) may acquire a captured image including a plurality of objects through a camera provided within the electronic device (100). According to one example, the electronic device (100) may also receive a captured image from an external camera connected via a wired / wireless communication cable. In this case, the external camera may be implemented to rotate in the same direction and angle as the electronic device (100).

[0046] Referring to FIG. 1, a display provided in an electronic device (100) may be physically rotated at various angles in a clockwise or counterclockwise direction. The electronic device (100) may acquire a first captured image (10) including a plurality of objects (e.g., a person, a sofa, a chair) located in front of the electronic device (100) through a camera (120).

[0047] When the display provided in the electronic device (100) is rotated 90 degrees clockwise, the camera (120) located at the top of the electronic device (100) can be rotated in the same direction and angle as the display. In this case, the electronic device (100) can obtain a captured image (11) rotated 90 degrees counterclockwise relative to the first captured image through the camera (120).

[0048] According to one embodiment, the electronic device (100) acquires an image of the front of the electronic device (100) and can identify whether the electronic device (100) is rotated based on a plurality of objects identified in the acquired image. For example, the electronic device (100) can identify whether the electronic device (100) is rotated based on a plurality of images (10, 11) of the front of the electronic device (100).

[0049] Hereinafter, various embodiments will be described with reference to the drawings in which an electronic device (100) identifies whether it is rotating based on an image taken of the front.

[0050] FIG. 2a is a block diagram illustrating the configuration of an electronic device according to one or more embodiments.

[0051] According to FIG. 2a, the electronic device (100) includes a display (110), a camera (120), a memory (130), and one or more processors (140). However, it is not limited thereto, and the electronic device (100) may be implemented with some components excluded or with other components included.

[0052] The display (110) is configured to display a captured image obtained through the camera (120). The display (110) may be implemented as a display module including a self-emissive element or as a display module including a non-emissive element and a backlight. Additionally, the display (110) may be implemented as an LFD display according to the above description. For example, it may be implemented as various types of displays such as an LCD (Liquid Crystal Display), an OLED (Organic Light Emitting Diodes) display, an LED (Light Emitting Diodes), a micro LED, a Mini LED, a PDP (Plasma Display Panel), a QD (Quantum dot) display, or a QLED (Quantum dot light-emitting diodes). The display (110) may also include a driving circuit, a backlight unit, etc., which can be implemented in forms such as an a-si TFT, an LTPS (low temperature poly silicon) TFT, or an OTFT (organic TFT).

[0053] The camera (120) may be a device for identifying the position of a user's eyes located in front of the electronic device (100). The camera (120) may be implemented as various types of cameras such as a depth camera, a stereo camera, an AI camera, an infrared camera, a motion camera, etc.

[0054] According to one example, the camera (120) may be positioned to capture the front of the display (110). For example, the camera (120) may be positioned in the central area of ​​the top bezel of the display (110). According to one example, the camera (120) may be positioned in a direction and angle to capture the front of the display (110).

[0055] The memory (130) can store at least one instruction, data, program, etc. required for the operation of the electronic device (100). For example, the memory (130) can store a captured image taken from the camera (120). For example, the memory (120) can store the location of an object identified by one or more processors (140).

[0056] The memory (130) may be implemented in the form of a memory embedded in the electronic device (100) or in the form of a memory detachable from the electronic device (100), depending on the purpose of data storage. For example, data for operating the electronic device (100) may be stored in a memory embedded in the electronic device (100), and data for the expansion function of the electronic device (100) may be stored in a memory detachable from the electronic device (100).

[0057] In the case of memory embedded in the electronic device (100), it may be implemented as at least one of volatile memory (e.g., DRAM (dynamic RAM), SRAM (static RAM), or SDRAM (synchronous dynamic RAM), non-volatile memory (e.g., OTPROM (one time programmable ROM), PROM (programmable ROM), EPROM (erasable and programmable ROM), EEPROM (electrically erasable and programmable ROM), mask ROM, flash ROM, flash memory (e.g., NAND flash or NOR flash), hard drive, or solid state drive (SSD).

[0058] The memory (130) may be implemented as a single memory that stores data generated in various operations according to the present disclosure, but is not limited thereto, and the memory (130) may be implemented to include a plurality of memories that each store different types of data or each store data generated in different stages.

[0059] One or more processors (140) control the overall operation of the electronic device (100). Specifically, one or more processors (140) may be connected to each component of the electronic device (100) to control the overall operation of the electronic device (100). For example, one or more processors (140) may be electrically connected to the display (110), camera (120), and memory (130) to control the overall operation of the electronic device (100). One or more processors (140) may include processing circuits and may be composed of one or more processors.

[0060] One or more processors (140) can perform the operation of an electronic device (100) according to various embodiments by executing one or more instructions stored in memory (130).

[0061] One or more processors (140) may include one or more of a CPU (Central Processing Unit), GPU (Graphics Processing Unit), APU (Accelerated Processing Unit), MIC (Many Integrated Core), DSP (Digital Signal Processor), NPU (Neural Processing Unit), hardware accelerator, or machine learning accelerator. One or more processors (140) may execute one or more programs or instructions stored in memory. For example, one or more processors may perform a method according to one or more embodiments of the present disclosure by executing one or more instructions stored in memory.

[0062] When a method according to one or more embodiments of the present disclosure includes a plurality of operations, the plurality of operations may be performed by a single processor or by a plurality of processors. For example, when a first operation, a second operation, and a third operation are performed by a method according to one or more embodiments, the first operation, the second operation, and the third operation may all be performed by a first processor, or the first operation and the second operation may be performed by a first processor (e.g., a general-purpose processor) and the third operation may be performed by a second processor (e.g., an artificial intelligence dedicated processor).

[0063] One or more processors (140) may be implemented as a single-core processor including one core, or as one or more multicore processors including multiple cores (e.g., homogeneous multicore or heterogeneous multicore). When one or more processors (140) are implemented as multicore processors, each of the multiple cores included in the multicore processor may include internal processor memory such as cache memory or on-chip memory, and a common cache shared by multiple cores may be included in the multicore processor. Additionally, each of the multiple cores included in the multicore processor (or some of the multiple cores) may independently read and execute program instructions for implementing a method according to one or more embodiments of the present disclosure, or all (or some) of the multiple cores may be linked together to read and execute program instructions for implementing a method according to one or more embodiments of the present disclosure.

[0064] When a method according to one or more embodiments of the present disclosure includes a plurality of operations, the plurality of operations may be performed by one of the plurality of cores included in a multi-core processor, or may be performed by a plurality of cores. For example, when a first operation, a second operation, and a third operation are performed by a method according to one or more embodiments, the first operation, the second operation, and the third operation may all be performed by a first core included in a multi-core processor, or the first operation and the second operation may be performed by a first core included in a multi-core processor and the third operation may be performed by a second core included in a multi-core processor.

[0065] In the embodiments of the present disclosure, a processor may refer to a system-on-chip (SoC) in which one or more processors and other electronic components are integrated, a single-core processor, a multi-core processor, or a core included in a single-core processor or a multi-core processor, wherein the core may be implemented as a CPU, GPU, APU, MIC, DSP, NPU, hardware accelerator, or machine learning accelerator, but the embodiments of the present disclosure are not limited thereto. For convenience of explanation, one or more processors (140) will be referred to as processors (140) below.

[0066] According to one embodiment, the processor (140) can identify a first line connecting a first point corresponding to a first object included in a first captured image obtained through a camera (120) and a second point corresponding to a second object included in the first captured image.

[0067] The first point and the second point may be any one of a plurality of points located on or inside the boundary of each object to display the shape of the first object and the second object.

[0068] According to one embodiment, the processor (140) can identify a second line connecting a first point and a second point in a second captured image obtained through the camera (120).

[0069] The first line and the second line may be straight lines connecting two points.

[0070] According to one embodiment, the processor (140) can identify rotation information of the display (110) based on the first line and the second line. The rotation information may be information including at least one of the rotation direction, rotation angle, and rotation acceleration of the electronic device (100).

[0071] According to one embodiment, the processor (140) can display a second captured image through the display (110) based on rotation information. For example, the processor (140) can identify rotation information that is rotated 90 degrees clockwise and display an image of the second captured image rotated 90 degrees clockwise through the display (110).

[0072] FIG. 2b is a block diagram illustrating the detailed configuration of an electronic device according to one or more embodiments.

[0073] According to FIG. 2b, the electronic device (100) includes a display (110), a camera (120), a memory (130), one or more processors (140), a communication interface (150), an input / output interface (160), and a microphone (170). A detailed description of the configurations shown in FIG. 2b that overlap with the configuration shown in FIG. 2a will be omitted.

[0074] The communication interface (150) includes a circuit and can perform communication with an external device (mobile device or server). For example, the processor (140) can receive various data or information from an external device connected through the communication interface (150) and can also transmit various data or information to the external device.

[0075] The communication interface (150) may include at least one of a Wi-Fi module, a Bluetooth module, a wireless communication module, an NFC module, and a UWB module (Ultra Wide Band). In this case, the wireless communication module can perform communication according to various communication standards such as IEEE, Zigbee, 3G (3rd Generation), 3GPP (3rd Generation Partnership Project), LTE (Long Term Evolution), 5G (5th Generation), etc.

[0076] The input / output interface (160) may be any one of the following interfaces: HDMI (High Definition Multimedia Interface), MHL (Mobile High-Definition Link), USB (Universal Serial Bus), DP (Display Port), Thunderbolt, VGA (Video Graphics Array) port, RGB port, D-SUB (D-subminiature), and DVI (Digital Visual Interface). The input / output interface (160) may input and output at least one of audio and video signals. Depending on the implementation example, the input / output interface (160) may include separate ports for inputting and outputting only audio signals and for inputting and outputting only video signals, or it may be implemented as a single port for inputting and outputting both audio and video signals.

[0077] The electronic device (100) can transmit at least one of audio and video signals to an external device (e.g., an external display device or an external speaker) through an input / output interface (160). Specifically, an output port included in the input / output interface (160) may be connected to an external device, and the electronic device (100) can transmit at least one of audio and video signals to the external device through the output port.

[0078] Here, the input / output interface (160) can be connected to the communication interface (150). The input / output interface (160) can transmit information received from an external device to the communication interface (150) or transmit information received through the communication interface (150) to the external device.

[0079] The microphone (170) is configured to receive user voice or other sounds and convert them into audio data. The processor (140) can control the display (110) based on the user voice signal received through the microphone (170). For example, when a user voice signal to display content A is received, the processor (140) can control the display (110) to display content A.

[0080] According to one embodiment, the processor (140) can control an external display device connected to the electronic device (100) based on a user voice signal received through the microphone (170). Specifically, the processor (140) can generate a control signal to control the external display device so that an operation corresponding to the user voice signal is performed on the external display device, and transmit the generated control signal to the external display device. Here, the processor (140) can store a remote control application for controlling the external display device. The processor (140) can transmit the generated control signal to the external display device using at least one communication method among Bluetooth, Wi-Fi, or infrared. For example, when a user voice signal for displaying content A is received, the processor (140) can transmit a control signal to the external display device to control the display of content A on the external display device. Here, the electronic device (100) may refer to various terminal devices capable of installing a remote control application, such as a smartphone or an AI speaker.

[0081] According to one embodiment, the processor (140) may use a remote control device to control an external display device connected to the electronic device (100) based on a user voice signal received through a microphone (170). Specifically, the processor (140) may transmit a control signal to the remote control device to control the external display device so that an operation corresponding to the user voice signal is performed on the external display device. The remote control device may transmit the control signal received from the electronic device (100) to the external display device. For example, when a user voice signal for displaying content A is received, the processor (140) transmits a control signal to the remote control device to control the display of content A on the external display device, and the remote control device transmits the received control signal to the external display device.

[0082] FIG. 3 is a diagram illustrating a point identification process of an electronic device according to one or more embodiments.

[0083] According to one embodiment, the electronic device (100) can identify a plurality of objects included in the captured image by inputting the captured image into an artificial intelligence model. The artificial intelligence model can be trained to identify objects included in the captured image based on various factors such as the type, size, location, shape, and color of the objects.

[0084] Here, the term "training an artificial intelligence model" means that a basic artificial intelligence model (e.g., an artificial intelligence model containing arbitrary random parameters) is trained by a learning algorithm using multiple training data, thereby creating predefined behavioral rules or an artificial intelligence model configured to perform a desired characteristic (or objective). This learning may be performed via a separate server and / or system, but is not limited thereto, and may also be performed in a cooking device. Examples of learning algorithms include supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but are not limited to the examples mentioned above.

[0085] Here, the artificial intelligence model may be implemented as, for example, CNN (Convolutional Neural Network), RNN (Recurrent Neural Network), RBM (Restricted Boltzmann Machine), DBN (Deep Belief Network), BRDNN (Bidirectional Recurrent Deep Neural Network), or Deep Q-Networks, but is not limited thereto.

[0086] According to one embodiment, the electronic device (100) can identify a plurality of points corresponding to an object based on an object identified from a captured image (10). The electronic device (100) can identify a plurality of objects (310 to 340) from a captured image (10) obtained through a camera (120) as described above, and can identify a plurality of points corresponding to each of the plurality of objects (310 to 340).

[0087] According to one example, points corresponding to an object may be randomly located on the boundary line or inside the object. For example, in the case of the picture frame object (310) illustrated in FIG. 3, a plurality of points (310-1 to 310-4) corresponding to the object (310) may be randomly located on the boundary line or inside the object (310).

[0088] For example, points corresponding to an object may be positioned based on the type of object or the distance between points. For example, in the case of a person object (340), the position of the person's face, arms, and legs may be important compared to other objects (310 to 330), so multiple points (340-1 to 340-5) may be positioned with emphasis on the face, arms, and legs. For example, in the case of a sofa object (320), multiple points (320-1 to 320-9) may be positioned based on the distance between points set within the object (320).

[0089] According to one embodiment, the electronic device (100) can identify the (x, y) coordinate values ​​of a plurality of points corresponding to each object (310 to 340). For example, the electronic device (100) can set the bottom left end of the captured image (10) as the origin (0, 0) and identify the coordinate values ​​of each point corresponding to the object (e.g., (3,3)).

[0090] In the above-described embodiment, the electronic device (100) is shown as identifying multiple points corresponding to each object, but is not limited thereto and may identify only one point for one object. Also, in FIG. 3, the sofa object (320) is shown as having 9 points, but may have n points (n is an integer greater than or equal to 1).

[0091] FIG. 4 is a diagram illustrating a static object identification process of an electronic device according to one or more embodiments.

[0092] According to one embodiment, the electronic device (100) can identify a static object among a plurality of objects (310 to 340) based on a plurality of first captured images. The static object may be a fixed object that does not move.

[0093] According to one embodiment, the electronic device (100) can identify a plurality of objects (310 to 340) based on at least two first captured images obtained through a camera (120) and identify a plurality of points corresponding to each of the plurality of objects (310 to 340). The electronic device (100) can identify a plurality of static objects among the plurality of objects (310 to 340) based on location information (e.g., coordinate values, distance between points) of the plurality of points corresponding to each of the plurality of objects (310 to 340).

[0094] Referring to FIG. 4, the electronic device (100) can acquire two first captured images (10, 20) through a camera (120). The electronic device (100) can identify static / dynamic objects based on location information of a plurality of points identified from the first first captured image (10) and the second first captured image (20).

[0095] For example, the electronic device (100) can identify location information of a plurality of points (330-1 to 330-6) corresponding to a chair object (330) in the first first captured image (10). For example, the electronic device (100) can identify the coordinate value (1, 1) of the first point (330-1) among the plurality of points (330-1 to 330-6) corresponding to the chair object (330). The electronic device (100) can identify the coordinate value (e.g., (8, 1)) of the first point (330-1) among the plurality of points (330-1 to 330-6) corresponding to the chair object in the second first captured image (20).

[0096] The electronic device (100) can identify changed position information (e.g., coordinate values) of a chair object (330) by comparing the coordinate values ​​of a first point identified in a first captured image (10) with the coordinate values ​​of a first point identified in a second first captured image (20). Based on the changed position information of the chair object (330), the electronic device (100) can identify the chair object (330) as a dynamic object.

[0097] In the same manner as described above, the electronic device (100) can identify the picture frame object (310) and the sofa object (320) as static objects based on location information of a plurality of points corresponding to each of the picture frame object (310) and the sofa object (320).

[0098] According to one embodiment, the electronic device (100) can identify a first line and a second line based on points corresponding to static objects (310, 320) among a plurality of objects (310 to 340).

[0099] The electronic device (100) can identify rotation information of the display (110) based on the first line and the second line. However, if the first line and the second line are identified based on multiple points corresponding to the dynamic object (330, 340), the accurate rotation information of the display may not be identified due to the movement of the object itself, even though the display (110) has not rotated.

[0100] Accordingly, the electronic device (100) can identify a static object among a plurality of objects and identify a first line and a second line based on a point corresponding to the static object.

[0101] FIGS. 5 and 6 are drawings for illustrating a point identification process for identifying a first line and a second line of an electronic device according to one or more embodiments.

[0102] According to one embodiment, the electronic device (100) can acquire a plurality of captured images through a camera (120) while the display (110) is rotating. For example, a user can rotate the display (110) itself from a horizontal display (110) to a vertical display (110). For example, the horizontal display (110) can be rotated to a vertical display (110) through a rotating device physically connected to the display (110).

[0103] For the sake of convenience, the following explanation assumes that the camera is located at the top center of the display and rotates clockwise.

[0104] Referring to FIG. 5, the electronic device (100) can acquire a plurality of captured images (10, 510 to 530) taken at various angles through a camera (120) while the display (110) is rotating. The camera (120) can rotate as a set with the display (110) in the same direction and angle as the direction and angle in which the display (110) is rotating.

[0105] For example, when the display (110) is rotated 30 degrees clockwise, the electronic device (100) can obtain a captured image (510) rotated 30 degrees counterclockwise relative to the first captured image (10) through the camera (120). Since the horizontal and vertical lengths of the display (110) itself are fixed, the captured image (510) rotated 30 degrees obtained by the electronic device (100) may be an image in which the images of the lower left and upper right corners are excluded compared with the first captured image. For example, the captured image (510) rotated 30 degrees may be an image in which the first, second, and fourth points (310-1, 310-2, 310-4) of the chair object (330) and the picture frame object (310) are excluded.

[0106] For example, when the display (110) is rotated 60 degrees clockwise, the electronic device (100) can acquire a captured image (520) rotated 60 degrees counterclockwise relative to the first captured image (10) through the camera (120). The captured image (520) rotated 60 degrees acquired by the electronic device (100) may be an image in which the lower left and upper right images are further excluded than the captured image (510) rotated 30 degrees. For example, the captured image (520) rotated 60 degrees may be an image in which even some points (320-1, 320-2) of the sofa object (320) are excluded.

[0107] For example, when the display (110) is rotated 90 degrees clockwise, the electronic device (100) can obtain a captured image (530) rotated 90 degrees counterclockwise relative to the first captured image (10) through the camera (120). The electronic device (100) can obtain a captured image (530) with a horizontal length corresponding to the vertical length of the first captured image.

[0108] As illustrated in FIG. 5, while the display (110) is rotating, the captured image obtained through the camera (120) may be an image in which a specific object (e.g., a chair object (330)) and a specific point corresponding to the object (e.g., a second and fourth point (310-2, 310-4) corresponding to a picture frame object) are excluded compared with the first captured image (10).

[0109] According to one embodiment, the electronic device (100) can identify points (e.g., 310-3, 320-3) excluding points (310-1, 310-2, 320-1, 320-2) that are likely to be excluded from the image captured while the display (110) is rotating, by comparing with the first captured image (10).

[0110] According to one embodiment, the electronic device (100) can identify a first point and a second point within an area excluding a pre-set margin area based on the boundary of the first captured image (10). The margin area may include an image area that may be excluded as the horizontal and vertical lengths of the display are fixed in the image captured while the display (110) is rotating. The margin area is not limited thereto and may be referred to in various ways, such as an exclusion area, a crop area, or a lost area, but in this disclosure, it will be collectively referred to as a margin area.

[0111] According to one example, the electronic device (100) can set a margin area based on the position of the camera (120) and the size of the display (110). For example, if the position of the camera (120) is located to the left of the display, the margin area may be set to be skewed to the left.

[0112] Referring to FIG. 6, the electronic device (100) can identify a pre-set margin area (610) based on the horizontal / vertical boundaries of the first captured image (10). The electronic device (100) can identify a first point and a second point for identifying a first line and a second line within an area (620) excluding the pre-set margin area.

[0113] FIG. 7 is a drawing for explaining the process of identifying a first line and a second line of an electronic device according to one or more embodiments.

[0114] According to one embodiment, the electronic device (100) can identify a first point among a plurality of points corresponding to a first object in a first captured image, and identify a second point among a plurality of points corresponding to a second object, thereby identifying a first line connecting the first point and the second point.

[0115] According to one example, the electronic device (100) can identify a first point and a second point based on static objects among a plurality of objects included in a first captured image (10). For example, the electronic device (100) can identify a first point and a second point based on a picture frame object (hereinafter the first object) (310) and a sofa object (hereinafter the second object) (320) among the static objects shown in FIG. 4.

[0116] According to one example, the electronic device (100) can identify a point (310-3) within an area (620) excluding the margin area (610) shown in FIG. 6 among a plurality of points (310-1 to 310-4) corresponding to the first object (310) as the first point (310-3).

[0117] According to one example, the electronic device (100) can identify a point (320-3) within the area (620) excluding the margin area (610) shown in FIG. 6 among a plurality of points (320-1 to 320-9) corresponding to the second object (320) as the second point (320-3).

[0118] Referring to FIG. 7, the electronic device (100) can identify a first line (710) connecting a first point (310-3) and a second point (320-3) in a first captured image (10).

[0119] According to one embodiment, the electronic device (100) can identify a second line connecting the first point and the second point in the second captured image (530). For convenience of explanation, the captured image (530) rotated 90 degrees as shown in FIG. 5 will be assumed to be the second captured image for the following description.

[0120] According to one embodiment, the electronic device (100) can identify the first point (310-3) based on the number of points, the distance between points, the coordinate values ​​of the points, and the location of each object among a plurality of points (310-1 to 310-4) corresponding to the first object (310) in the second captured image (530). For example, the electronic device (100) can identify the coordinate values ​​of the first point (310-3), the distance between other points (310-1, 310-2, 310-4), the number of points (4), and the location of each object (310 to 340) in the first captured image. Subsequently, based on the information described above, the electronic device (100) can identify the first point (310-3) located at the lower left of the frame object (310) located above the person object (340) in the second captured image. The method of identifying the first point in the second captured image is not limited to this, and an object may be identified through an AI model, and a point corresponding to the same location as the first point (310-3) in the first captured image (10) may be identified as the first point (310-3).

[0121] According to one example, the electronic device (100) can identify a second point (320-3) in a second captured image (530) in the same way as described above.

[0122] Referring to FIG. 7, the electronic device (100) can identify a second line (720) connecting the first point (310-3) and the second point (320-3) in the second captured image (530).

[0123] FIG. 8 is a drawing for explaining a method for identifying rotation information of a display according to one or more embodiments.

[0124] According to one embodiment, the electronic device (100) can identify rotation information of the display (110) based on the angle formed by the first line (710) and the second line (720).

[0125] According to one example, the electronic device (100) can identify the rotation angle of the display (110) based on the angle formed by the first line (710) and the second line (720).

[0126] For example, the electronic device (100) can identify a first angle formed by a horizontal line (740) of the display (110) and a first line (710), and a second angle formed by a horizontal line (740) and a second line (720). The electronic device (100) can identify an angle formed by the first line (710) and the second line (720) based on the first angle and the second angle.

[0127] For example, the electronic device (100) can identify the slope of each of the first line (710) and the second line (720) based on the coordinate values ​​of the first point (310-3) and the second point (320-3). The electronic device (100) can identify the angle formed by the first line (710) and the second line (720) based on the slope of the first line (710) and the slope of the second line (720).

[0128] The method for identifying the angle formed by the first line (710) and the second line (720) is not limited to the method described above and can be identified through various mathematical formulas.

[0129] According to one example, the electronic device (100) can identify the rotation direction of the display (110) based on the position of a first point on a first line and the position of a first point on a second line.

[0130] For example, the electronic device (100) can identify that it has moved in a counterclockwise direction by comparing the position of the first point (310-3) of the second captured image (530) with the position of the first point (310-3) of the first captured image (10). As illustrated in FIG. 5, when the display (110) is rotated clockwise, the image obtained through the camera (120) can be an image rotated counterclockwise with respect to the first captured image (10). Accordingly, the electronic device (100) can identify that the display (110) has rotated clockwise based on the fact that the first point (310-3) has moved counterclockwise (730).

[0131] According to one example, the electronic device (100) can identify a plurality of rotation angles from a plurality of captured images and identify a change in rotation based thereon. The electronic device (100) can identify rotational acceleration (or rotational angular velocity) based on the time taken during rotation and the change in rotation.

[0132] FIGS. 9 and FIGS. 10 are drawings for explaining the process of identifying whether a display is rotated according to one or more embodiments.

[0133] According to one embodiment, the electronic device (100) can identify whether the display (110) is rotated based on points where the distance from each of the plurality of objects in the first captured image (10) is greater than or equal to a threshold distance.

[0134] According to one example, the electronic device (100) can acquire a captured image (hereinafter referred to as the third captured image) including an object that has rotated clockwise or counterclockwise from a fixed position.

[0135] In FIG. 9, for the convenience of explanation, it is assumed that the object included in the first captured image (10) and the second captured image (530) shown in FIG. 7 is different from the object included in FIG. 7.

[0136] Referring to FIG. 9, the second captured image (530) may be an image of the first object (920) and the second object (930) when the display (110) is rotated 90 degrees clockwise. The third captured image (910) may be an image in which the first object (910) is rotated 90 degrees counterclockwise without rotation of the display (110), and the second object (930) is positioned above the first object (910). In this case, the electronic device (100) may identify that the display (110) has rotated based on the third captured image (910) obtained through the camera (120).

[0137] According to one embodiment, the electronic device (100) can identify a plurality of candidate points in the first captured image (10) such that the distance from each of the first object (920) and the second object (930) is greater than or equal to a threshold distance.

[0138] According to one embodiment, the electronic device (100) can identify at least one target point among a plurality of candidate points based on the sum of the distance between each candidate point and a first point and the distance between each candidate point and a second point.

[0139] Referring to FIG. 10, the electronic device (100) can identify a virtual first region (1010) including a virtual line located at a critical distance from the border line of a first object (920). The electronic device (100) can identify a virtual second region (1020) including a virtual line located at a critical distance from the border line of a second object (930).

[0140] According to one example, the electronic device (100) can identify a plurality of candidate points (1030-1 to 1030-n) in an area excluding the first area (1010) and the second area (1020) of the first captured image (10).

[0141] Referring to FIG. 10, the electronic device (100) can identify the candidate point (1030-2) as the target point among a plurality of candidate points (1030-1 to 1030-n) for which the sum of the distances between each candidate point (1030-1 to 1030-n) and the first point (920-1) corresponding to the first object (920) and the second point (930-1) corresponding to the second object (930) is the largest. Although FIG. 10 shows one target point (1030-2), it is not limited thereto and may be a plurality of target points.

[0142] According to one embodiment, the electronic device (100) can identify whether the display (110) is rotated based on the target point (1030-2) identified in the first captured image (10) and the target point (1030-2) identified in the second captured image (530). The electronic device (100) can identify the target point (1030-2) in the second captured image in the same way as the method for identifying the first point in the second captured image that corresponds to the same position as the first point in the first captured image.

[0143] FIG. 11 is a diagram illustrating a process for identifying whether a display is rotated based on the color value of a target point according to one or more embodiments.

[0144] According to one embodiment, the electronic device (100) identifies a first color of a target point (1030-2) identified in a first captured image (10) and a second color of a target point (1030-2) identified in a second captured image (530), and if the difference between the first color and the second color is greater than or equal to a preset value, it can identify that the display (110) has rotated.

[0145] The first color and the second color may be colors based on R, G, and B pixel values ​​corresponding to the location of the target point (1030-2). For example, if the R, G, and B pixel values ​​of the target point (1030-2) identified in the first captured image (10) are R:80, G:0, and B:0, the first color may be red.

[0146] Referring to FIG. 11, when the background color at the top center of the first captured image (10) is red (hatched area) and the background color on the right is blue (dotted area), the electronic device (100) can identify a first color of red from the target point (1030-2) identified in the first captured image (10).

[0147] On the other hand, the second captured image (530) taken with the display (110) rotated 90 degrees clockwise may be an image in which the right background of the first captured image (10) is located at the top center. That is, the second captured image (530) may be an image in which the background color at the top center is blue and the background color on the left is red. In this case, the electronic device (100) can identify a second color of blue from the target point (1030-2) identified in the second captured image (530).

[0148] The electronic device (100) can identify that the difference between the R, G, B pixel values ​​of the first color (red) and the R, G, B pixel values ​​of the second color (blue) is greater than or equal to a preset value, thereby identifying that the display (110) has rotated.

[0149] According to one embodiment, the electronic device (100) identifies a first color of a target point (1030-2) identified in a first captured image (10) and a third color of a target point (1030-2) identified in a third captured image (910), and if the difference between the first color and the third color is less than a preset value, it can identify that at least one of the first object (920) and the second object (930) has rotated.

[0150] Referring to FIG. 11, the electronic device (100) can identify a third color of red, identical to the first color, from a target point (1030-2) identified in a third captured image (910). The electronic device (100) can identify that at least one of the first object (920) and the second object (930) has rotated by identifying that the difference between the R, G, B pixel values ​​of the first color (red) and the R, G, B pixel values ​​of the third color (red) is less than a preset value.

[0151] FIG. 12 is a drawing for explaining a UI display process corresponding to camera position adjustment of an electronic device according to one or more embodiments.

[0152] According to one embodiment, the electronic device (100) can identify a first image (1210) corresponding to the center area of ​​a first captured image (10) and identify a second image (1220) corresponding to the first image identified in a second captured image (530) to identify the position of a camera (120) at the top of the electronic device (100).

[0153] According to one example, the electronic device (100) may identify an image as a first image (1210) that includes an area within a preset range in the center area of ​​the first captured image (10). The first image (1210) may be an image that includes an image that is not excluded based on the first captured image (10) from an image captured through a camera (120) while the display (110) is rotating.

[0154] Referring to FIG. 12, the camera (120) may be located at the top left (120-1) or right (120-2) of the display (110). When the camera (120) is located at the top left (120-1) of the display, the electronic device (100) can identify a second image (1220-1) corresponding to the topmost first image (1210) in the second captured image (530). When the camera (120) is located at the top right (120-2) of the display, the electronic device (100) can identify a second image (1220-3) corresponding to the bottommost first image (1210) in the second captured image (530). When the camera (120) is positioned in the center of the display, the electronic device (100) can identify a second image (1220-2) corresponding to the first image (1210) in the center in the second captured image (530).

[0155] According to one embodiment, the electronic device (100) may display a UI (User Interface) through the display (110) that guides the position adjustment of the camera (120) based on the position of the second image (1220-1 to 1220-3) corresponding to the first image (1210) and the rotation direction of the display (110).

[0156] Referring to FIG. 12, when the camera (120) is positioned at the top right of the display (120-2), the electronic device (100) can display a guide UI through the display (110) that includes text such as “Please move the camera to the left” based on the position of the second image (1220-3) corresponding to the first image (1210) in the second captured image (530) and the rotation direction (clockwise) of the display (110).

[0157] FIG. 13 is a drawing for explaining the black screen display process of an electronic device according to one or more embodiments.

[0158] According to one embodiment, the electronic device (100) identifies that the display (110) is rotating when the rotational acceleration of the display (110) exceeds a preset value, and can display a black screen through the display (110) while the display (110) is rotating.

[0159] A black screen is intended to compensate for the delay that occurs during the process of rotating a captured image and may include a black screen. The black screen is not limited thereto and may be referred to in various ways, such as a screen saver or a dark screen; however, in this disclosure, it will be collectively referred to as a black screen.

[0160] According to one example, the electronic device (100) may identify that the display (110) has rotated even when the display (110) moves slightly left and right or up and down. In this case, the electronic device (100) calculates the rotational acceleration of the display (110) based on the method described above, and if the rotational acceleration exceeds a preset value, it may identify that the display (110) is rotating.

[0161] Referring to FIG. 13, when the display (110) is rotated 90 degrees, the electronic device (100) may display a black screen (1310, 1320) through the display (110) instead of the image captured when rotated 30 degrees and the image captured when rotated 60 degrees. When multiple images captured by the camera (120) are all displayed through the display (110) when the display (110) is rotated from 0 to 90 degrees, the user may feel dizzy or uncomfortable.

[0162] According to one embodiment, the electronic device (100) can display a converted image (1330) of a second captured image (530) through a display (110) based on the rotation direction and rotation angle.

[0163] According to one embodiment, the electronic device (100) can receive a user voice signal in various ways to identify whether the display (110) is rotated. According to one example, the electronic device (100) can receive a user voice signal through a microphone (170) included in the electronic device (100).

[0164] According to one example, the electronic device (100) may receive a user voice signal from an external device including a microphone. Here, the external device may refer to a remote control device or a smartphone, etc. Here, the received user voice signal may be a digital voice signal, but may be an analog voice signal depending on the implementation example. The electronic device (100) may receive the user voice signal through a wireless communication method such as Bluetooth or Wi-Fi.

[0165] According to one embodiment, the electronic device (100) can obtain text information corresponding to a user voice signal from an external server. Specifically, the electronic device (100) can transmit a user voice signal (audio signal or digital signal) to an external server. Here, the external server may refer to a speech recognition server. Here, the speech recognition server can convert the user voice signal into text information using STT (Speech To Text). Then, the external server can transmit the text information corresponding to the converted user voice signal to the electronic device (100).

[0166] According to one embodiment, the electronic device (100) can independently acquire text information corresponding to a user voice signal. Specifically, the electronic device (100) may directly apply a Speech To Text (STT) function to a digital voice signal to convert it into text information and transmit the converted text information to an external server.

[0167] According to one embodiment, the electronic device (100) may receive rotation information of the display (110) from an external server. For example, the electronic device (100) may receive information corresponding to the rotation direction or rotation angle of the display (110) from an external server, and may display a second captured image based on the received information through the display (110).

[0168] According to one embodiment, the electronic device (100) may receive text information corresponding to a user voice signal from an external server. Specifically, the external server may be a server that performs a voice recognition function that converts the user voice signal into text information.

[0169] According to one embodiment, an external server may transmit at least one of text information corresponding to a user voice signal or search result information corresponding to text information to an electronic device (100). Specifically, the external server may be a server that performs a search result providing function that provides search result information corresponding to text information, in addition to a voice recognition function that converts a user voice signal into text information. As an example, the external server may be a server that performs both a voice recognition function and a search result providing function. As another example, the external server may perform only a voice recognition function, and the search result providing function may be performed on a separate server. The external server may transmit text information to a separate server to obtain search results and obtain search results corresponding to text information from the separate server.

[0170] According to one embodiment, the electronic device (100) transmits a first captured image and a second captured image to an external server and receives rotation information of a display from the external server. The external server can identify rotation information between the captured images based on the received plurality of captured images.

[0171] For example, an external server can identify a first line and a second line based on points corresponding to objects included in a captured image from a plurality of received captured images. Based on the identified first line and second line, the external server can identify rotation information of a display and transmit it to an electronic device (100).

[0172] According to one embodiment, the electronic device (100) can rotate and output a second captured image based on rotation information of a display received from an external server.

[0173] According to one embodiment, if the electronic device (100) does not have a sensor that detects rotation within the electronic device (100), it can obtain rotation information of the electronic device (100) based on a plurality of captured images. Based on the embodiments described above, the electronic device (100) can obtain rotation information of a display and output captured images based on the obtained rotation information.

[0174] For example, when a user experiences web content through an electronic device (100), the user can rotate the display (110) from a horizontal shape to a vertical shape. In this case, the electronic device (100) can rotate and output a captured image based on the rotation direction and rotation angle of the display (110).

[0175] For example, if the display (110) rotates while the user is playing a streaming game through the electronic device (100), the electronic device (100) can rotate and output the captured image based on the rotation direction and rotation angle of the display (110).

[0176] For example, if the display (110) is rotated while the user is performing a video call through the camera (120), the electronic device (100) can rotate and output the captured image based on the rotation direction and rotation angle of the display (110).

[0177] According to one embodiment, if the electronic device (100) does not have a sensor (e.g., an accelerometer) for detecting the degree of rotation within the electronic device (100), it can obtain a rotation direction and a rotation angle based on a first line and a second line identified from a plurality of captured images. The electronic device (100) can obtain rotation information of a display based on the obtained rotation direction and rotation angle, and can output a captured image based on the obtained rotation information.

[0178] FIG. 14 is a flowchart illustrating the overall operation of an electronic device according to one or more embodiments.

[0179] Referring to FIG. 14, in operation 1410, the electronic device (100) can input a captured image taken through a camera (120) into an artificial intelligence model.

[0180] In operation 1420, the electronic device (100) can identify multiple objects within a captured image through an artificial intelligence model.

[0181] In operation 1430, the electronic device (100) can classify static / dynamic objects among a plurality of objects based on points corresponding to the objects.

[0182] In operation 1440, the electronic device (100) can identify point coordinates corresponding to a static object and identify a second line based on the coordinates of the identified point.

[0183] In operation 1450, the electronic device (100) can identify point coordinates corresponding to a static object in a previously captured image and identify a first line based on the coordinates of the identified point.

[0184] In operation 1460, the electronic device (100) can identify whether the display (110) is rotated by comparing the first line and the second line.

[0185] In operation 1470, the electronic device (100) can display a captured image based on rotation information through a display (110).

[0186] FIG. 15 is a flowchart illustrating the operation of an electronic device according to one or more embodiments.

[0187] Referring to FIG. 15, in operation 1510, the electronic device (100) can identify a first object (310) and a second object (320) in a first captured image (10) obtained through a camera (120).

[0188] In operation 1520, when the electronic device (100) identifies a first point (310-3) corresponding to a first object (310) and a second point (320-3) corresponding to a second object (320), it can identify a first line (710) connecting the first point (310-3) and the second point (320-3).

[0189] In operation 1530, the electronic device (100) can identify a second line (720) connecting a first point (310-3) and a second point (320-3) in a second captured image (530) obtained through a camera (120).

[0190] In operation 1540, the electronic device (100) can identify rotation information of the display (110) based on the first line (710) and the second line (720).

[0191]

[0192] *In operation 1550, the electronic device (100) can display a second captured image (530) based on rotation information.

[0193] As the method for identifying the first line and the second line, and identifying rotation information of the display based on the first line and the second line, has been specifically explained in the various embodiments described above, a redundant explanation is omitted.

[0194] The control method described in FIG. 15 can be performed by an electronic device (100) having the configuration of FIG. 2 described above, but is not necessarily limited thereto and can be performed by an electronic device having various configurations.

[0195] The various embodiments described above may be implemented as individual embodiments, or at least one embodiment may be combined with one another, either wholly or partially, to be implemented together in a single device.

[0196] According to the various embodiments described above, the electronic device (100) can identify rotation information of the display (110) based on a point corresponding to an object and provide a captured image based on the rotation information, thereby enabling smoother image processing.

[0197] Meanwhile, the various embodiments described above may be applied to a product as embodiments alone, but at least some of their contents may be combined with other embodiments of the present disclosure to be implemented together.

[0198] The various embodiments described above may be implemented as software containing instructions stored on a machine-readable storage medium (e.g., computer). The machine may include an electronic device (e.g., electronic device (100)) according to the disclosed embodiments, which is a device capable of calling instructions stored from the storage medium and operating according to the called instructions. When instructions are executed by a processor, the processor may perform a function corresponding to the instructions directly or by using other components under the control of the processor. Instructions may include code generated or executed by a compiler or an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory computer-readable storage medium. Here, "non-transitory" means only that the storage medium does not contain a signal and is tangible, and does not distinguish whether data is stored semi-permanently or temporarily in the storage medium.

[0199] In addition, according to one embodiment of the present disclosure, the method according to the various embodiments described above may be provided by being included in a computer program product.

[0200] Specifically, a non-transient readable storage medium or computer program product may be provided that stores a computer instruction to perform an operation including: identifying a first object and a second object in a first captured image acquired through a camera; identifying a first point corresponding to the first object and a second point corresponding to the second object when they are identified; identifying a second line connecting the first point and the second point in a second captured image acquired through a camera; identifying rotation information of a display based on the first line and the second line; and displaying the second captured image based on the rotation information.

[0201] Computer program products may be distributed in the form of device-readable storage media (e.g., compact disc read-only memory (CD-ROM)) or online through an application store (e.g., Play Store™). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created on a storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0202] In addition, computer instructions or programs for performing the control method of an electronic device according to the various embodiments described above may be stored on a non-transitory computer-readable medium. When computer instructions stored on such a non-transitory computer-readable medium are executed by the processor of a specific device, they cause the specific device to perform a processing operation according to the various embodiments described above. A non-transitory computer-readable medium refers to a medium that stores data semi-permanently and is readable by a device, rather than a medium that stores data for a short period of time, such as a register, cache, or memory. Specific examples of a non-transitory computer-readable medium may include CDs, DVDs, hard disks, Blu-ray discs, USBs, memory cards, ROMs, etc.

[0203] Although preferred embodiments of the present disclosure have been illustrated and described above, the present disclosure is not limited to the specific embodiments described above. It is understood that various modifications can be made by those skilled in the art without departing from the essence of the present disclosure as claimed in the claims, and such modifications should not be understood individually from the technical spirit or perspective of the present disclosure.

Claims

1. In an electronic device, display; camera; Memory for storing instructions; and One or more processors including processing circuitry; and The above one or more processors, When the above instructions are executed individually or collectively, Identifying a first line connecting a first point corresponding to a first object included in a first captured image acquired through the camera and a second point corresponding to a second object included in the first captured image, and Identifying a second line connecting the first point and the second point in the second captured image obtained through the camera, and An electronic device that controls the display to output the second captured image based on rotation information of the display obtained based on the first line and the second line.

2. In Paragraph 1, The above one or more processors, When the above instructions are executed individually or collectively, Identifying a plurality of objects and a plurality of points corresponding to each of the plurality of objects in a plurality of first captured images acquired through the camera, and An electronic device that identifies a plurality of static objects among a plurality of objects based on position information of a plurality of points corresponding to each of the plurality of objects in the plurality of first captured images, and identifies the identified plurality of static objects as the first object and the second object.

3. In Paragraph 1, The above one or more processors, When the above instructions are executed individually or collectively, Rotation information of the display is obtained based on the angle formed by the first line and the second line, and The rotation information of the above display is, An electronic device comprising at least one of the rotation direction, rotation angle, and rotation acceleration of the electronic device.

4. In Paragraph 1, The above one or more processors, When the above instructions are executed individually or collectively, When a plurality of candidate points are identified in the first captured image such that the distance from each of the first object and the second object is greater than or equal to a threshold distance, at least one target point among the plurality of candidate points is identified based on the sum of the distance between each candidate point and the first point and the distance between each candidate point and the second point. An electronic device that identifies whether the display is rotated based on the target point identified in the first captured image and the target point identified in the second captured image.

5. In Paragraph 4, The above one or more processors, When the above instructions are executed individually or collectively, Identifying the first color of the target point identified in the first captured image and the second color of the target point identified in the second captured image, and If the difference between the first color and the second color is less than a preset value, it is identified that at least one of the first object and the second object has rotated, and An electronic device that identifies that the display has rotated if the difference between the first color and the second color is greater than or equal to the preset value.

6. In Paragraph 1, The above one or more processors, When the above instructions are executed individually or collectively, An electronic device that identifies the first point and the second point within an area excluding a pre-set margin area based on the boundary of the first captured image.

7. In Paragraph 1, The above one or more processors, When the above instructions are executed individually or collectively, When a first image corresponding to the central area of ​​the first captured image is identified, a second image corresponding to the identified first image is identified in the second captured image, and An electronic device that controls the display to output a UI (User Interface) corresponding to the position adjustment of the camera based on the rotation information and the position of the second image in the second captured image.

8. In Paragraph 1, The above first point and the above second point are, An electronic device identified based on at least one of the type or point distance between the first object and the second object.

9. In Paragraph 3, The above one or more processors, When the above instructions are executed individually or collectively, An electronic device that controls the display to output a preset screen corresponding to the rotation of the display when the rotational acceleration of the display exceeds a preset value.

10. In a method for controlling an electronic device, A step of identifying a first line connecting a first point corresponding to a first object included in a first captured image acquired through a camera and a second point corresponding to a second object included in the first captured image; A step of identifying a second line connecting the first point and the second point in a second captured image obtained through the camera; and A control method comprising the step of outputting the second captured image based on rotation information of a display obtained based on the first line and the second line.

11. In Paragraph 10, A step of identifying a plurality of objects and a plurality of points corresponding to each of the plurality of objects in a plurality of first captured images acquired through the camera; and A control method comprising: a step of identifying a plurality of static objects among a plurality of objects based on position information of a plurality of points corresponding to each of a plurality of objects in a plurality of first captured images, and identifying the identified plurality of static objects as the first object and the second object.

12. In Paragraph 10, The step of obtaining rotation information of the above display is, The method further includes the step of obtaining rotation information of the display based on the angle formed by the first line and the second line; The rotation information of the above display is, A control method comprising at least one of the rotation direction, rotation angle, and rotation acceleration of the electronic device.

13. In Paragraph 10, When a plurality of candidate points are identified in the first captured image such that the distance from each of the first object and the second object is greater than or equal to a threshold distance, a step of identifying at least one target point among the plurality of candidate points based on the sum of the distance between each candidate point and the first point and the distance between each candidate point and the second point; and A control method comprising: a step of identifying whether the display is rotated based on the target point identified in the first captured image and the target point identified in the second captured image.

14. In Paragraph 13, The step of identifying whether the above display is rotated is, A step of identifying a first color of the target point identified in the first captured image and a second color of the target point identified in the second captured image; A step of identifying that at least one of the first object and the second object has rotated if the difference between the first color and the second color is less than a preset value; and A control method further comprising the step of identifying that the display has rotated if the difference between the first color and the second color is greater than or equal to the preset value.

15. A non-transient computer-readable storage medium storing computer instructions that cause said electronic device to perform an operation when executed by a processor of said electronic device, wherein said operation is, A step of identifying a first line connecting a first point corresponding to a first object included in a first captured image acquired through a camera and a second point corresponding to a second object included in the first captured image; A step of identifying a second line connecting the first point and the second point in a second captured image obtained through the camera; and A non-transient computer-readable storage medium comprising: a step of outputting the second captured image based on rotation information of a display obtained based on the first line and the second line.

Images