Display device for expanding screen area and control method thereof

Abstract

A display device according to the present invention comprises: a display; a communication interface; a memory configured to store instructions; and one or more processors that are connected to the display, the communication interface, and the memory and execute the instructions. The instructions can be executed by the processor so that the display device displays a first image on the display, identifies a motion of an object included in the first image, acquires a second image including the object on the basis of the identified motion, and streams the second image to other display devices via the communication interface.

Description

Display device for expanding screen area and control method thereof

[0001] The present disclosure relates to a display device and a control method thereof, and more particularly, to a display device that expands a screen area and a control method thereof.

[0002] Advances in electronic technology are leading to the development of electronic devices offering a variety of functions. In particular, users are now able to utilize a variety of display devices, enhancing their convenience.

[0003] For example, various display devices such as desktop PCs, tablet PCs, and smartphones are provided to users, providing more diverse user experiences.

[0004] According to one embodiment of the present disclosure to achieve the above object, a display device includes a display, a communication interface, a memory configured to store a command, and at least one processor operably connected to the display, the communication interface, and the memory and configured to execute the command, wherein the command is executed by the processor to cause the display device to display a first image on the display, identify a motion of an object included in the first image, obtain a second image including the object based on the identified motion, and stream the second image to another display device through the communication interface.

[0005] In addition, the command is executed by the processor, and the display device can stream the second image through the communication interface to a plurality of other display devices positioned around the display device, the other display device being positioned at a position corresponding to the identified motion.

[0006] And, when the command is executed by the processor, the display device can communicate with the other display device through the communication interface to identify the location of the other display device, and obtain the second image based on the identified motion and the location of the other display device.

[0007] In addition, the command is executed by the processor, and the display device can communicate with the other display device through the UWB (ultra-wideband) communication standard to identify the location of the other display device.

[0008] And, when the command is executed by the processor, the display device can transmit a signal requesting information about another display included in the other display device to the other display device through the communication interface, obtain information about the other display from the other display device through the communication interface, and obtain the second image based on the identified motion and information about the other display.

[0009] Additionally, the information about the other display may include at least one of the size, resolution, aspect ratio, or form factor of the other display.

[0010] And, when the command is executed by the processor, the display device can obtain the second image by outpainting the background area of ​​the object.

[0011] Additionally, the command is executed by the processor so that the display device can generate the sound of the second image based on the sound of the object included in the first image.

[0012] And, the command is executed by the processor so that the display device can identify the alignment status of the display device and the other display device, and obtain the second image based on the alignment status.

[0013] In addition, when the command is executed by the processor and guidance information indicating that the display device uses the other display device is received from the other display device through the communication interface, the acquisition operation and streaming operation of the second image can be stopped.

[0014] Meanwhile, according to one embodiment of the present disclosure, a method for controlling a display device includes the steps of displaying a first image, identifying a motion of an object included in the first image, obtaining a second image including the object based on the identified motion, and streaming the second image to another display device.

[0015] Additionally, the streaming step may stream the second image to a plurality of other display devices positioned around the display device, the other display device being positioned at a position corresponding to the identified motion.

[0016] And, it further includes a step of performing communication with the other display device to identify the location of the other display device, and the obtaining step can obtain the second image based on the identified motion and the location of the other display device.

[0017] In addition, the step of identifying the location of the other display device can identify the location of the other display device by performing communication with the other display device through a UWB (ultra-wideband) communication standard.

[0018] And, the acquiring step may transmit a request signal for information on another display included in the other display device to the other display device, acquire information on the other display from the other display device, and acquire the second image based on the identified motion and information on the other display.

[0019] According to one embodiment of the present disclosure, a display device includes at least one memory configured to store instructions, which instructions, when executed by one or more processors, cause the display device to: display a first image; identify a motion of an object included in the first image; obtain a second image including the object based on the identified motion; and stream the second image to another display device.

[0020] The above command, when executed by one or more processors, may cause a second image to be streamed to a plurality of other display devices located around the display device at a position corresponding to the identified motion.

[0021] The above instructions, when executed by the one or more processors, are configured to cause the display device to perform communication with the other display device to identify a location of the other display device; and to obtain a second image based on the identified motion and the location of the other display device.

[0022] Additionally, when the above command is executed by one or more processors, the display device can identify the location of another display device by communicating with the other display device through the UWB (ultra-wideband) communication standard.

[0023] The instructions, when executed by one or more processors, cause the display device to: transmit a signal to the other display device requesting information about another display included in the other display device; and the other display of the other display device; and obtain a second image based on the identified motion and the information of the other display.

[0024] The above and other aspects, features and advantages of specific embodiments of the present disclosure will become more apparent from the following description taken in conjunction with the accompanying drawings.

[0025] FIG. 1 is a diagram showing the configuration of a display system according to one embodiment of the present disclosure.

[0026] FIG. 2 is a block diagram showing the configuration of a display device according to one embodiment of the present disclosure.

[0027] FIG. 3 is a block diagram showing a detailed configuration of a display device according to an embodiment of the present disclosure.

[0028] FIG. 4 is a drawing for explaining expansion of a screen area to multiple display devices according to one embodiment of the present disclosure.

[0029] FIG. 5 is a drawing for explaining an embodiment of a 3D space according to an embodiment of the present disclosure.

[0030] FIG. 6 is a diagram illustrating a method of using information on another display according to one embodiment of the present disclosure.

[0031] FIGS. 7 and 8 are drawings for explaining an operation that takes into account the alignment state of display devices according to one embodiment of the present disclosure.

[0032] FIG. 9 is a diagram for explaining operation on a single screen according to one embodiment of the present disclosure.

[0033] FIG. 10 is a flowchart for explaining a method for controlling a display device according to an embodiment of the present disclosure.

[0034] The exemplary embodiments of the present disclosure are subject to various modifications. Accordingly, specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, it should be understood that the present disclosure is not limited to specific embodiments and encompasses all modifications, equivalents, and alternatives within the scope of the technical spirit of the present disclosure. Furthermore, detailed descriptions of well-known functions or configurations may be omitted as unnecessary details may obscure the contents of the present disclosure.

[0035] The purpose of the present disclosure is to provide a display device and a control method thereof for providing a more immersive and three-dimensional experience to a user by expanding the screen area.

[0036] Hereinafter, the present disclosure will be described in detail with reference to the attached drawings.

[0037] The terms used in connection with the embodiments of this disclosure are generally and widely used terms, selected in consideration of the functions of this disclosure. However, these terms may vary depending on the intentions of those skilled in the art, precedents, the emergence of new technologies, etc. Furthermore, in certain cases, terms may be arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the description of the relevant disclosure. Therefore, the terms used in this disclosure should not be defined simply as names of terms, but rather based on the meanings of the terms and the overall content of this disclosure.

[0038] In this disclosure, expressions such as “has,” “can have,” “includes,” or “may include” indicate the presence of a corresponding feature (e.g., a component such as a number, function, operation, or part), and do not exclude the presence of additional features.

[0039] Throughout this disclosure, the expression “at least one of a, b or c” means a only, b only, c only, both a and b, both a and c, both b and c, or all of a, b and c.

[0040] The expressions “first,” “second,” “first,” or “second,” etc., used in this disclosure can describe various components, regardless of order and / or importance, and are only used to distinguish one component from another, but do not limit the components.

[0041] Singular expressions include plural expressions unless the context clearly dictates otherwise. In this disclosure, terms such as "comprise" or "consist of" are intended to indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in this disclosure, but should be understood not to preclude the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0042] In this disclosure, the term user may refer to a person using an electronic device or a device using an electronic device (e.g., an artificial intelligence electronic device).

[0043] Various embodiments of the present disclosure will be described in more detail with reference to the attached drawings below.

[0044] FIG. 1 is a diagram illustrating the configuration of a display system (1000) according to one embodiment of the present disclosure. As illustrated in FIG. 1, the electronic system (1000) may include multiple display devices. For example, the electronic system (1000) may include a laptop and a smartphone.

[0045] A laptop can display a video and provide a portion of the video to a smartphone. For example, the laptop can display a first video, identify the motion of an object included in the first video, acquire a second video containing the object based on the identified motion, and stream the second video to the smartphone.

[0046] The smartphone can display a second image received from the laptop.

[0047] This action allows users to watch videos not only on their laptops but also on their smartphones, increasing the immersive and three-dimensional experience of watching videos.

[0048] FIG. 2 is a block diagram showing the configuration of a display device (100) according to one embodiment of the present disclosure.

[0049] The display device (100) can display an image and provide the image to another display device (200). For example, the display device (100) may be a device equipped with a display, such as a TV, a desktop PC, a laptop, a smartphone, a tablet PC, etc., and displaying an image through the display and providing the image to another display device (200). However, the display device (100) is not limited thereto, and any device may be used as long as it can display an image and provide the image to another device.

[0050] The display device (100) may provide an image to another display device (200), but is not limited thereto. For example, the display device (100) may provide a portion of the image to another display device (200). Alternatively, the display device (100) may provide only a portion of the image frame to another display device (200). Alternatively, the display device (100) may generate a separate image based on an object included in the image and provide the generated image to another display device (200).

[0051] According to FIG. 2, the display device (100) includes a display (110), a communication interface (120), and a processor (130).

[0052] The display (110) is a configuration that displays an image and can be implemented as a display of various forms such as an LCD (Liquid Crystal Display), an OLED (Organic Light Emitting Diodes) display, a PDP (Plasma Display Panel), etc. The display (110) may also include a driving circuit, a backlight unit, etc. that can be implemented as a form such as an a-si TFT, an LTPS (low temperature poly silicon) TFT, an OTFT (organic TFT), etc. Meanwhile, the display (110) may be implemented as a touch screen combined with a touch sensor, a flexible display, a 3D display, etc.

[0053] The communication interface (120) is a component that performs communication with various types of external devices according to various types of communication methods. For example, the display device (100) can perform communication with another display device (200) through the communication interface (120). For example, the display device (100) can perform communication with another display device (200) through the UWB (ultra-wideband) communication standard to identify the location of the other display device (200).

[0054] The communication interface (120) may include a Wi-Fi module, a Bluetooth module, an infrared communication module, a wireless communication module, etc. Here, each communication module may be implemented in the form of at least one hardware chip.

[0055] Wi-Fi and Bluetooth modules communicate via Wi-Fi and Bluetooth, respectively. When using a Wi-Fi or Bluetooth module, connection information, such as the SSID and session key, is first transmitted and received. This information is then used to establish a communication connection before various other information can be transmitted and received. Infrared communication modules use infrared data association (IrDA) technology, which wirelessly transmits data over short distances using infrared light, which lies between visible light and millimeter waves.

[0056] In addition to the above-described communication method, the wireless communication module may include at least one communication chip that performs communication according to various wireless communication standards such as zigbee, 3G (3rd Generation), 3GPP (3rd Generation Partnership Project), LTE (Long Term Evolution), LTE-A (LTE Advanced), 4G (4th Generation), 5G (5th Generation), etc.

[0057] Alternatively, the communication interface (120) may include a wired communication interface such as HDMI, DP, Thunderbolt, USB, RGB, D-SUB, DVI, etc.

[0058] In addition, the communication interface (120) may include at least one of a LAN (Local Area Network) module, an Ethernet module, or a wired communication module that performs communication using a pair cable, a coaxial cable, or an optical fiber cable.

[0059] The processor (130) controls the overall operation of the display device (100). Specifically, the processor (130) is connected to each component of the display device (100) and can control the overall operation of the display device (100). For example, the processor (130) is connected to components such as the display (110), the communication interface (120), and the memory and can control the operation of the display device (100).

[0060] The one or more processors (130) may include one or more of a CPU, a GPU (Graphics Processing Unit), an APU (Accelerated Processing Unit), a MIC (Many Integrated Core), an NPU (Neural Processing Unit), a hardware accelerator, or a machine learning accelerator. The one or more processors (130) may control one or any combination of other components of the display device (100) and perform operations related to communication or data processing. The one or more processors (130) may execute one or more programs or instructions stored in a memory. For example, the one or more processors (130) may perform a method according to an embodiment of the present disclosure by executing one or more instructions stored in a memory.

[0061] When a method according to an embodiment of the present disclosure includes multiple operations, the multiple operations may be performed by one processor or by multiple processors. For example, when a first operation, a second operation, and a third operation are performed by a method according to an embodiment, the first operation, the second operation, and the third operation may all be performed by the first processor, or the first operation and the second operation may be performed by the first processor (e.g., a general-purpose processor) and the third operation may be performed by the second processor (e.g., an artificial intelligence-specific processor).

[0062] One or more processors (130) may be implemented as a single core processor including one core, or may be implemented as one or more multicore processors including multiple cores (e.g., homogeneous multicores or heterogeneous multicores). When one or more processors (130) are implemented as a multicore processor, each of the multiple cores included in the multicore processor may include an internal processor memory, such as a cache memory or an on-chip memory, and a common cache shared by the multiple cores may be included in the multicore processor. In addition, each of the multiple cores (or some of the multiple cores) included in the multicore processor may independently read and execute a program instruction for implementing a method according to an embodiment of the present disclosure, or all (or some) of the multiple cores may be linked to read and execute a program instruction for implementing a method according to an embodiment of the present disclosure.

[0063] When a method according to an embodiment of the present disclosure includes a plurality of operations, the plurality of operations may be performed by one core among the plurality of cores included in a multi-core processor, or may be performed by the plurality of cores. For example, when a first operation, a second operation, and a third operation are performed by a method according to an embodiment, the first operation, the second operation, and the third operation may all be performed by a first core included in the multi-core processor, or the first operation and the second operation may be performed by a first core included in the multi-core processor, and the third operation may be performed by a second core included in the multi-core processor.

[0064] In embodiments of the present disclosure, one or more processors (130) may refer to a system on a chip (SoC) in which one or more processors and other electronic components are integrated, a single-core processor, a multi-core processor, or a core included in a single-core processor or a multi-core processor, wherein the core may be implemented as a CPU, a GPU, an APU, a MIC, an NPU, a hardware accelerator, or a machine learning accelerator, but the embodiments of the present disclosure are not limited thereto. However, for convenience of explanation, the operation of the display device (100) is described below using the expression processor (130).

[0065] The processor (130) controls the display (110) to display a first image and can identify the motion of an object included in the first image. For example, the processor (130) can control the display (110) to sequentially display a plurality of frames included in the first image, identify an object in each frame, and compare the positions of the objects in each frame to identify the motion of the object.

[0066] The processor (130) may acquire a second image including an object based on the identified motion. For example, if a person dancing in place is identified as an object in the first image, the processor (130) may crop the area including the person in each frame to acquire a second image. Alternatively, if a car moving to the right in the first image is identified as an object, the processor (130) may crop the area including the car in each frame to acquire a second image. In other words, the processor (130) may change the cropped area in each frame depending on the motion of the object.

[0067] The processor (130) can control the communication interface (120) to stream the second image to another display device (200) positioned around the display device (100). For example, the processor (130) can control the communication interface (120) to stream the second image, in which the area including the object in each frame is cropped, to another display device (200).

[0068] The processor (130) can control the communication interface (120) to stream a second image to another display device (200) positioned at a position corresponding to the identified motion among a plurality of other display devices positioned around the display device (100).

[0069] For example, in a state where a first display device is arranged on the left side of the display device (100) and a second display device is arranged on the right side of the display device (100), when a car moving to the right is identified in the first image, the processor (130) can crop an area including the car in each frame to obtain a second image, and provide the second image to the second display device.

[0070] The processor (130) may communicate with the other display device (200) through the communication interface (120) to identify the location of the other display device (200), and obtain a second image based on the identified motion and the location of the other display device (200). For example, if a car moving to the right is identified in the first image and the other display device (200) is identified as being arranged to the right of the display device (100), the processor (130) may crop an area including the car in each frame to obtain a second image, and provide the second image to the second display device. Alternatively, if a car moving to the right is identified in the first image and the other display device (200) is identified as being arranged to the left of the display device (100), the processor (130) may not obtain the second image.

[0071] The processor (130) can identify the transmission time of the second image based on the location of the other display device (200). For example, the processor (130) can transmit the second image later when the other display device (200) is 20 cm away from the display device (100) than when the other display device (200) is 10 cm away from the display device (100). However, the present invention is not limited thereto, and the processor (130) can also change the generation time of the second image based on the location of the other display device (200). For example, the processor (130) can generate the second image later when the other display device (200) is 20 cm away from the display device (100) than when the other display device (200) is 10 cm away from the display device (100).

[0072] The processor (130) can identify the location of another display device (200) by communicating with the other display device (200) through the UWB (ultra-wideband) communication standard.

[0073] The above is only an example, and the processor (130) may identify the location of the other display device (200) using any number of different methods.

[0074] The processor (130) controls the communication interface (120) to transmit a signal requesting information about another display included in the other display device (200) to the other display device (200), receives information about the other display from the other display device (200) through the communication interface (120), and acquires a second image based on the identified motion and information about the other display. Here, the information about the other display may include at least one of the size, resolution, screen ratio, or form factor of the other display.

[0075] For example, if the other display has a 4:3 ratio, the processor (130) may obtain a second image by cropping the first image to a 4:3 ratio. Alternatively, if the other display is foldable or rollable, the processor (130) may obtain the second image based on a size smaller than the overall size of the other display.

[0076] However, it is not limited thereto, and the processor (130) may also acquire the second image by taking into consideration the injection rate of the other display, the communication status with the other display device (200), etc.

[0077] The processor (130) can acquire a second image by outpainting the background area of ​​an object. For example, if a car moving to the right is identified in the first image, the processor (130) can crop only the car from the first image and outpaint the background area of ​​the car to acquire a second image.

[0078] The processor (130) can generate the sound of the second image based on the sound of the object included in the first image.

[0079] The processor (130) can identify the alignment status of the display device (100) and another display device (200) and obtain a second image based on the alignment status.

[0080] When the processor (130) receives guidance information from the other display device (200) via the communication interface (120) indicating that another display device (200) is to be used, the processor (130) may stop the acquisition operation and streaming operation of the second image. For example, when the user uses the other display device (200), even if the display device (100) provides the second image, the other display device (200) may not display the second image. Accordingly, when the processor (130) receives guidance information from the other display device (200) indicating that another display device (200) is to be used, the processor (130) may stop the acquisition operation and streaming operation of the second image to reduce power consumption.

[0081] However, the present invention is not limited thereto, and if the processor (130) receives guidance information from the other display device (200) through the communication interface (120) that another display device (200) is to be used, the processor (130) may lower the resolution of the second image during the process of acquiring the second image. For example, even if the user uses another display device (200), the second image received from the display device (100) may be displayed in a pop-up form. In this case, even if guidance information from the other display device (200) that another display device (200) is received, the processor (130) may acquire the second image with a lowered resolution, and stream the second image with a lowered resolution to the other display device (200), thereby reducing power consumption.

[0082] FIG. 3 is a block diagram illustrating a detailed configuration of a display device (100) according to an embodiment of the present disclosure. The display device (100) may include a display (110), a communication interface (120), and a processor (130). In addition, according to FIG. 3, the display device (100) may further include a memory (140), a user interface (150), a microphone (160), a speaker (170), and a camera (180). For components illustrated in FIG. 3 that overlap with those illustrated in FIG. 2, a detailed description thereof will be omitted.

[0083] Memory (140) may refer to hardware that stores information such as data in an electrical or magnetic form so that a processor (130) or the like can access it. To this end, memory (140) may be implemented with one or more of non-volatile memory, volatile memory, flash memory, hard disk drive (HDD), solid state drive (SSD), RAM, ROM, etc.

[0084] The memory (140) may store at least one instruction required for the operation of the display device (100) or the processor (130). Here, the instruction is a unit of code that instructs the operation of the display device (100) or the processor (130), and may be written in machine language, which is a language that a computer can understand. Alternatively, the memory (140) may store a plurality of instructions for performing a specific task of the display device (100) or the processor (130) as an instruction set.

[0085] The memory (140) may store data in bit or byte units that can represent characters, numbers, images, etc. For example, the memory (140) may store location information of another display device (200), an image processing module, etc.

[0086] The memory (140) is accessed by the processor (130), and reading / writing / modifying / deleting / updating instructions, instruction sets, or data can be performed by the processor (130).

[0087] The user interface (150) may be implemented with buttons, a touch pad, a mouse, a keyboard, etc., or may be implemented with a touch screen capable of performing both display and operation input functions. Here, the buttons may be various types of buttons, such as mechanical buttons, touch pads, wheels, etc., formed on any area of ​​the front, side, or back of the main body of the display device (100).

[0088] The microphone (160) is configured to receive sound and convert it into an audio signal. The microphone (160) is electrically connected to the processor (130) and can receive sound under the control of the processor (130).

[0089] For example, the microphone (160) may be formed as an integrated unit integrated into the upper side, front side, side direction, etc. of the display device (100). Alternatively, the microphone (160) may be provided in a remote control separate from the display device (100). In this case, the remote control may receive sound through the microphone (160) and provide the received sound to the display device (100).

[0090] The microphone (160) may include various configurations such as a microphone that collects analog sound, an amplifier circuit that amplifies the collected sound, an A / D conversion circuit that samples the amplified sound and converts it into a digital signal, and a filter circuit that removes noise components from the converted digital signal.

[0091] The microphone (160) may be implemented in the form of a sound sensor, and any method that can collect sound may be used.

[0092] The speaker (170) is a component that outputs various audio data processed by the processor (130) as well as various notification sounds and voice messages.

[0093] The camera (180) is configured to capture still images or moving images. The camera (180) can capture still images at a specific point in time, but can also capture still images continuously.

[0094] The camera (180) can capture the front of the display device (100) to capture the actual environment in front of the display device (100). The processor (130) can also identify another display device (200) through the camera (180).

[0095] The camera (180) includes a lens, a shutter, an aperture, a solid-state image sensor, an AFE (Analog Front End), and a TG (Timing Generator). The shutter controls the time at which light reflected from a subject enters the camera (180), and the aperture mechanically increases or decreases the size of the opening through which light enters to control the amount of light incident on the lens. When the solid-state image sensor accumulates light reflected from a subject as a photocharge, the image generated by the photocharge is output as an electrical signal. The TG outputs a timing signal for reading out pixel data of the solid-state image sensor, and the AFE samples and digitizes the electrical signal output from the solid-state image sensor.

[0096] As described above, the display device (100) can provide a more immersive and three-dimensional experience to the user by extending the screen area to other display devices (200) arranged around it.

[0097] Hereinafter, the operation of the display device (100) will be described in more detail with reference to FIGS. 4 to 9. For convenience of explanation, individual embodiments are described in FIGS. 4 to 9. However, the individual embodiments of FIGS. 4 to 9 may be implemented in any combination.

[0098] FIG. 4 is a drawing for explaining screen area expansion to multiple display devices according to one embodiment of the present disclosure.

[0099] The processor (130) communicates with a first display device (200-1) and a second display device (200-2) around the display device (100), and can identify the locations of the first display device (200-1) and the second display device (200-2). For example, as illustrated in FIG. 4, the processor (130) communicates with the first display device (200-1) and the second display device (200-2) through a UWB (ultra-wideband) communication standard, and can identify the locations of the first display device (200-1) and the second display device (200-2).

[0100] When the processor (130) establishes communication with the first display device (200-1) and the second display device (200-2) through the UWB communication standard, the processor (130) can identify the locations of the first display device (200-1) and the second display device (200-2). Alternatively, when the processor (130) establishes communication with the first display device (200-1) and the second display device (200-2) through the UWB communication standard, the processor (130) can identify the locations of the first display device (200-1) and the second display device (200-2) at preset time intervals. Alternatively, when a preset application is executed, the processor (130) can identify the location of the display device (200) that is connected to the display device (100) through the UWB communication standard.

[0101] When the processor (130) is connected to the first display device (200-1) and the second display device (200-2) through the UWB communication standard, the processor (130) requests the use status of the first display device (200-1) and the second display device (200-2), and when the first display device (200-1) and the second display device (200-2) are not in use, the processor (130) can transmit an image related to an image being displayed on the display device (100) to the first display device (200-1) and the second display device (200-2).

[0102] For example, the processor (130) controls the display (110) to display a first image, identifies the motion of an object included in the first image, obtains a second image including the object based on the identified motion, and transmits the second image to the first display device (200-1) and the second display device (200-2) when the first display device (200-1) and the second display device (200-2) are not in use.

[0103] The processor (130) may determine at least one of a method for acquiring a second image or a method for transmitting the second image based on the motion of the object in the first image. For example, if motion is identified in the first image without movement of the object, the processor (130) may acquire a second image including the object, as illustrated in FIG. 4, and transmit the second image to the first display device (200-1) and the second display device (200-2).

[0104] Alternatively, when movement of an object is identified in the first image, the processor (130) may acquire a second image or transmit the acquired second image based on at least one of the position of the object in the frame, the direction of movement, or the position of a surrounding display device. For example, when an object moves to the right in the first image, the processor (130) may acquire the time it takes for the object to move from the position of the object in the frame to the position of the second display device (200-2) based on the movement speed of the object, and transmit the second image including the object to the second display device (200-2) after the acquired time has elapsed. In addition, the processor (130) may not transmit the second image to the first display device (200-1) because the object moves to the right.

[0105] FIG. 5 is a drawing for explaining an embodiment of a 3D space according to an embodiment of the present disclosure.

[0106] As shown in the upper part of Fig. 5, a user can view an image formed in 3D space through 3D glasses. In this case, the 3D glasses can display only a portion of the image formed in 3D space corresponding to the direction in which the user's face is facing.

[0107] The display device (100) can store an image formed in a 3D space, as shown in the upper part of Fig. 5. The processor (130) can control the display (110) to display a portion of the image formed in a 3D space.

[0108] The processor (130) identifies the positions of the first display device (200-1) and the second display device (200-2), and when the first display device (200-1) and the second display device (200-2) are not in use, transmits an image of an area corresponding to the position of the first display device (200-1) in an image formed in 3D space to the first display device (200-1), and transmits an image of an area corresponding to the position of the second display device (200-2) in an image formed in 3D space to the second display device (200-2).

[0109] The processor (130) can identify the positions of the first display device (200-1) and the second display device (200-2) at preset time intervals, and when the positions change, provide an image of an area corresponding to the changed position in an image formed in 3D space.

[0110] FIG. 6 is a diagram illustrating a method of using information on another display according to one embodiment of the present disclosure.

[0111] The processor (130) controls the communication interface (120) to transmit a signal requesting information about another display included in the other display device to the other display device, receives information about the other display from the other display device through the communication interface (120), and can obtain a second image based on the identified motion and information about the other display.

[0112] For example, as illustrated in FIG. 6, the processor (130) controls the communication interface (120) to transmit a signal requesting information about the first display included in the first display device (200-1) to the first display device (200-1), receives information about the first display from the first display device (200-1) through the communication interface (120), and obtains a second image 1 based on the identified motion and information about the first display. For example, the processor (130) can obtain a second image 1 corresponding to the resolution and screen ratio of the first display.

[0113] In addition, the processor (130) controls the communication interface (120) to transmit a signal requesting information about a second display included in the second display device (200-2) to the second display device (200-2), as illustrated in FIG. 6, and receives information about the second display from the second display device (200-2) through the communication interface (120), and obtains a second image 2 based on the identified motion and information about the second display. For example, the processor (130) can obtain a second image 2 corresponding to the resolution and screen ratio of the second display.

[0114] Accordingly, the second video 1 may be a video with an aspect ratio of 16:9, and the second video 2 may be a video with an aspect ratio of 9:16.

[0115] The processor (130) may perform scaling to obtain the second image 2 when the object included in the first image is not included in the resolution of the second display. For example, if the resolution of the second display is 480×800 and the resolution of the rectangular area (620) including the motorcycle on the left side of FIG. 6 is 960×1600, the processor (130) may downscale the resolution of the rectangular area including the motorcycle to obtain the second image 2.

[0116] The processor (130) may also obtain the second image based on the form factor of the second display device. For example, if the second display device (200-2) is a foldable device and the upper display and the lower display of the second display device (200-2) do not form a 180-degree angle, the processor (130) may obtain the second image 2 of 480×400 instead of the second image 2 of 480×800 corresponding to the resolution of the upper display and the lower display. When the second image 2 is received from the display device (100), the second display device (200-2) may display the second image 2 on one of the upper display and the lower display.

[0117] When multiple objects are identified in the first image, the processor (130) may acquire a second image including at least one of the multiple objects based on at least one of the sizes or motions of each of the multiple objects. For example, as illustrated in FIG. 6, the processor (130) may identify a motorcycle and an airplane in the first image, and since the motorcycle is larger than the airplane, the processor (130) may acquire a second image including the motorcycle.

[0118] However, the present invention is not limited thereto, and the processor (130) may acquire the second image by further considering information about the other display. For example, the processor (130) may acquire the second image 2 that includes only the motorcycle based on the screen ratio of the second display. However, if the processor (130) determines that the screen ratio of the first display can also include the airplane, the processor (130) may acquire the second image 1 that includes the motorcycle and the airplane.

[0119] For convenience of explanation, in FIG. 6, the display device (100), the first display device (200-1), and the second display device (200-2) are illustrated as simultaneously displaying images including the same object even when the object moves, but this is not limited thereto. For example, if the display device (100) displays a motorcycle and the motorcycle moves to the right and disappears through the right edge of the display device (100), the second display device (200-1) may also display the motorcycle.

[0120] FIGS. 7 and 8 are drawings for explaining an operation that takes into account the alignment state of display devices according to one embodiment of the present disclosure.

[0121] Fig. 7 illustrates a display device (100) in the center, a first display device on the left side of the display device (100), and a second display device on the right side of the display device (100). In addition, Fig. 7 assumes that the upper edges of each of the display device (100), the first display device, and the second display device are arranged on the same line, and the lower edges of each of the display device (100), the first display device, and the second display device are arranged on the same line.

[0122] However, in some cases, as illustrated in FIG. 8, the upper and lower edges of each display device may not be arranged on the same line. In this case, the processor (130) may identify the alignment status of the display device (100), the first display device, and the second display device, and acquire the second image based on the alignment status.

[0123] For example, the processor (130) can identify the alignment status of the display device (100), the first display device, and the second display device, and if the upper edge and the lower edge of the first display device are higher than the upper edge and the lower edge of the display device (100), respectively, outpaint an area (810) higher than the upper edge of the display device (100) to obtain a second image 1. If the upper edge and the lower edge of the second display device are lower than the upper edge and the lower edge of the display device (100), respectively, outpaint an area (820) lower than the upper edge of the display device (100) to obtain a second image 2.

[0124] Through this operation, the sense of incongruity between the first image and the second image output through the display device (100), the first display device, and the second display device can be minimized.

[0125] FIG. 9 is a diagram for explaining operation on a single screen according to one embodiment of the present disclosure.

[0126] The processor (130) can change the screen ratio of the first image to correspond to the screen ratio of the display (110) through outpainting. For example, the processor (130) can display the first image of 1.85:1 as shown in the upper part of FIG. 9, in which case the areas 910-1 and 910-2 can be displayed in black. The processor (130) can identify that the screen ratio of the first image is different from the screen ratio of the display (110) and can change the first image of the display (110) to 16:9 as shown in the lower part of FIG. 9. In this case, the processor (130) can change the first image of 1.85:1 to a first image of 16:9 by outpainting the areas 910-1 and 910-2, such as the areas 920-1 and 920-2, respectively.

[0127] However, the present invention is not limited thereto, and the processor (130) may also outpaint the first image by further considering the object. For example, if an airplane moves upward in the first image of the upper part of FIG. 9 and a part of the airplane is not visible through the area 910-1, the processor (130) may perform outpainting as shown in the lower part of FIG. 9. In this case, a part of the airplane that was not visible in the area 920-1 may be expressed based on the shape of the airplane in the previous frame.

[0128] FIG. 10 is a flowchart for explaining a method for controlling a display device according to an embodiment of the present disclosure.

[0129] First, a first image is displayed (S1010). Then, the motion of an object included in the first image is identified (S1020). Then, a second image including the object is acquired based on the identified motion (S1030). Then, the second image is streamed to another display device positioned around the display device (S1040).

[0130] Additionally, the streaming step (S1040) can stream a second image to another display device positioned at a position corresponding to the identified motion among a plurality of other display devices positioned around the display device.

[0131] And, it further includes a step of identifying the location of the other display device by performing communication with the other display device, and the obtaining step (S1030) can obtain a second image based on the identified motion and the location of the other display device.

[0132] Additionally, the step of identifying the location of the other display device can identify the location of the other display device by performing communication with the other display device through the UWB (ultra-wideband) communication standard.

[0133] And, the acquiring step (S1030) can transmit a signal requesting information about another display included in another display device to the other display device, receive information about the other display from the other display device, and acquire a second image based on the identified motion and information about the other display.

[0134] Additionally, information about the other display may include at least one of the size, resolution, aspect ratio, or form factor of the other display.

[0135] And, the acquiring step (S1030) can acquire a second image by outpainting the background area of ​​the object.

[0136] Additionally, the acquiring step (S1030) can generate the sound of the second image based on the sound of the object included in the first image.

[0137] And, it further includes a step of identifying the alignment status of the display device and other display devices, and the obtaining step (S1030) can obtain a second image based on the alignment status.

[0138] Additionally, when guidance information indicating that another display device is being used is received from the other display device, a step of stopping the acquisition operation and streaming operation of the second image may be further included.

[0139] According to various embodiments of the present disclosure as described above, the display device can extend the screen area to other display devices arranged around it to provide a more immersive and three-dimensional experience to the user.

[0140] According to an exemplary embodiment of the present disclosure, the various embodiments described above may be implemented as software including instructions stored in a machine-readable storage medium that can be read by a machine (e.g., a computer). The device may include an electronic device (e.g., electronic device (A)) according to the disclosed embodiments, which is a device that can call instructions stored in the storage medium and operate according to the called instructions. When an instruction is executed by a processor, the processor may directly or under the control of the processor perform a function corresponding to the instruction using other components. The instruction may include code generated or executed by a compiler or interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' means that the storage medium does not contain a signal and is tangible, but does not distinguish between data being stored semi-permanently or temporarily in the storage medium.

[0141] Furthermore, according to one embodiment of the present disclosure, the method according to the various embodiments described above may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or online through an application store (e.g., Play Store™). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0142] Furthermore, according to one embodiment of the present disclosure, the various embodiments described above may be implemented in a computer-readable recording medium or a similar device using software, hardware, or a combination thereof. In some cases, the embodiments described in the present disclosure may be implemented by the processor itself. In a software implementation, embodiments such as the procedures and functions described in the present disclosure may be implemented as separate software. Each software may perform one or more functions and operations described in the present disclosure.

[0143] Computer instructions for performing processing operations of a device according to the various embodiments described above may be stored in a non-transitory computer-readable medium. The computer instructions stored in such a non-transitory computer-readable medium, when executed by a processor of a specific device, cause the specific device to perform processing operations of the device according to the various embodiments described above. A non-transitory computer-readable medium refers to a medium that permanently stores data and can be read by a device, rather than a medium that stores data for a short period of time, such as a register, cache, or memory. Specific examples of non-transitory computer-readable media may include a CD, DVD, hard disk, Blu-ray disk, USB, memory card, or ROM.

[0144] In addition, each of the components (e.g., modules or programs) according to the various embodiments described above may be composed of a single or multiple entities, and some of the corresponding sub-components described above may be omitted, or other sub-components may be further included in various embodiments. Alternatively or additionally, some components (e.g., modules or programs) may be integrated into a single entity, which may perform the same or similar functions as those performed by each of the corresponding components prior to integration. Operations performed by modules, programs or other components according to various embodiments may be executed sequentially, in parallel, iteratively or heuristically, or at least some operations may be executed in a different order, omitted, or other operations may be added.

[0145] Although the preferred embodiments of the present disclosure have been illustrated and described above, the present disclosure is not limited to the above-described embodiments, and various modifications may be made by a person having ordinary skill in the art to which the present disclosure pertains without departing from the gist of the present disclosure as claimed in the claims. Furthermore, such modifications should not be understood individually from the technical idea or prospect of the present disclosure.

Claims

1. In the display device, display; communication interface; a memory configured to store instructions; and comprising one or more processors operably connected to the display, the communication interface and the memory, and configured to execute the instructions; The display device is configured such that the command is executed by the processor. Display the first image on the above display, Identifying the motion of an object included in the first image above, Acquire a second image containing the object based on the identified motion, A display device that streams the second image to another display device through the communication interface.

2. In paragraph 1, The display device is configured such that the command is executed by the processor. A display device that streams the second image through the communication interface to a plurality of other display devices positioned around the display device, the other display device being positioned at a position corresponding to the identified motion.

3. In paragraph 1, The display device is configured such that the command is executed by the processor. By performing communication with the other display device through the above communication interface, the location of the other display device is identified, A display device that acquires the second image based on the identified motion and the location of the other display device.

4. In paragraph 3, The display device is configured such that the command is executed by the processor. A display device that identifies the location of another display device by communicating with the other display device through a UWB (ultra-wideband) communication standard.

5. In paragraph 1, The display device is configured such that the command is executed by the processor. Transmitting a signal requesting information about another display included in the other display device to the other display device through the above communication interface, Obtaining information about the other display from the other display device through the above communication interface, A display device that obtains the second image based on the identified motion and information about the other display.

6. In paragraph 5, Information about the above displays is: A display device comprising at least one of the size, resolution, aspect ratio or form factor of the above display.

7. In paragraph 1, The display device is configured such that the command is executed by the processor. An electronic device that obtains the second image by outpainting a background area of ​​the object.

8. In paragraph 1, The display device is configured such that the command is executed by the processor. A display device that generates the sound of the second image based on the sound of the object included in the first image.

9. In paragraph 1, The display device is configured such that the command is executed by the processor. Identify the alignment status of the above display device and the other display device, A display device that obtains the second image based on the above alignment state.

10. In paragraph 1, The display device is configured such that the command is executed by the processor. A display device that stops the acquisition operation and streaming operation of the second image when guidance information indicating that the other display device is to be used is received from the other display device through the communication interface.

11. In a method for controlling a display device, Step of displaying the first image; A step of identifying the motion of an object included in the first image; A step of obtaining a second image including the object based on the identified motion; and A control method comprising the step of streaming the second image to another display device.

12. In paragraph 11, The above streaming steps are: A control method for streaming the second image to a plurality of other display devices positioned around the display device, the other display device being positioned at a position corresponding to the identified motion.

13. In paragraph 11, Further comprising a step of identifying a location of the other display device by performing communication with the other display device; The above obtaining steps are: A control method for obtaining the second image based on the identified motion and the position of the other display device.

14. In paragraph 13, The step of identifying the location of the above display device is: A control method for identifying the location of another display device by performing communication with the other display device through a UWB (ultra-wideband) communication standard.

15. In paragraph 11, The above obtaining steps are: Transmitting a signal requesting information about another display included in the other display device to the other display device, Obtaining information about the other display from the other display device, A control method for obtaining the second image based on the identified motion and information about the other display.

Images