Electronic device and method of operating the same

Abstract

An electronic device and a method of operating the same are provided. The electronic device includes: at least one memory storing one or more instructions; and one or more processors. When the one or more instructions are executed by the one or more processors, the electronic device performs the following operations: acquiring a first captured image of a display device that outputs a first image to a screen area; acquiring a second captured image of the display device when the screen is off; acquiring a second operational image from which a luminance component has been extracted from the first captured image; acquiring a third operational image from which a luminance component has been extracted from the second captured image; and identifying a screen area based on a result image obtained by performing a second operation on the second operational image and the third operational image.

Description

Technical Field

[0001] This disclosure relates to an electronic device for identifying a screen area of ​​a display device, a method for operating the electronic device, and a recording medium. Background Technology

[0002] As display devices become larger and have higher resolutions, the number of display modules that make up the display device is increasing. For example, multiple display modules can be connected to each other to form a screen for the display device.

[0003] When installing display devices, it is difficult for installers to manually input the position, layout (or arrangement relationship) and connection relationship of each display module, so there is a possibility of installation errors.

[0004] A method is being researched to automatically identify the location of display modules and the arrangement or connection relationship between display modules during the installation of display devices, and to complete the setup without separate setup work. Summary of the Invention

[0005] Solution to the problem

[0006] According to one aspect of this disclosure, an electronic device includes: at least one memory storing one or more instructions; and one or more processors configured to execute one or more instructions.

[0007] According to one aspect of this disclosure, one or more instructions, when executed by one or more processors, cause an electronic device to obtain a first captured image of a display device in a state of outputting a first image to a screen area.

[0008] According to one aspect of this disclosure, one or more instructions, when executed by one or more processors, cause an electronic device to obtain a second captured image of a display device in a state where the screen area is turned off.

[0009] According to one aspect of this disclosure, one or more instructions, when executed by one or more processors, cause the electronic device to obtain a second operational image from which value components have been extracted from a first captured image.

[0010] According to one aspect of this disclosure, one or more instructions, when executed by one or more processors, cause the electronic device to obtain a third operational image from which value components have been extracted from a second captured image.

[0011] According to one aspect of this disclosure, one or more instructions, when executed by one or more processors, cause the electronic device to identify a screen region based on a result image obtained by performing a second operation on a second operation image and a third operation image.

[0012] According to one aspect of this disclosure, a method of operating an electronic device includes: obtaining a first captured image of a display device in a state of outputting a first image to a screen area; obtaining a second captured image of the display device in a state where the screen area is turned off; obtaining a second operating image by extracting value components from the first captured image; obtaining a third operating image from which value components have been extracted from the second captured image; and identifying a screen area based on a result image obtained by performing a second operation on the second operating image and the third operating image.

[0013] According to one aspect of this disclosure, a non-transitory computer-readable medium may be provided having a program recorded thereon for performing at least one of the methods of operating an electronic device on a computer. Attached Figure Description

[0014] The above and other aspects and features of certain embodiments of this disclosure will become clearer from the following description taken in conjunction with the accompanying drawings, in which:

[0015] Figure 1 The figures illustrate a display device and an electronic device according to embodiments of the present disclosure;

[0016] Figure 2 This is a diagram illustrating a display device before and after setup according to an embodiment of the present disclosure;

[0017] Figure 3 This is a diagram illustrating a screen area of ​​a display device recognized by an electronic device according to an embodiment of the present disclosure;

[0018] Figure 4 This is a flowchart illustrating a method for setting up a display device performed by an electronic device according to an embodiment of the present disclosure;

[0019] Figure 5 This is a block diagram illustrating the configuration of an electronic device and a display device according to embodiments of the present disclosure;

[0020] Figure 6 This is a diagram illustrating the operation of obtaining a first captured image according to an embodiment of the present disclosure;

[0021] Figure 7 This is a diagram illustrating the operation of obtaining a second captured image according to an embodiment of the present disclosure;

[0022] Figure 8 This is a diagram illustrating a first operational image according to an embodiment of the present disclosure;

[0023] Figure 9 This is a diagram illustrating a second operational image according to an embodiment of the present disclosure;

[0024] Figure 10 This is a diagram illustrating a third operational image according to an embodiment of the present disclosure;

[0025] Figure 11 This is a diagram showing a result image according to an embodiment of the present disclosure;

[0026] Figure 12 This is a diagram illustrating the operation of the peripheral region in a processed result image according to embodiments of the present disclosure;

[0027] Figure 13a This is a flowchart illustrating the operation of an electronic device according to an embodiment of the present disclosure;

[0028] Figure 13b This is a diagram illustrating the operation of an electronic device obtaining a resulting image according to an embodiment of the present disclosure;

[0029] Figure 14a This is a flowchart illustrating the operation of an electronic device according to an embodiment of the present disclosure;

[0030] Figure 14b This is a diagram illustrating the operation of an electronic device obtaining a resulting image according to an embodiment of the present disclosure;

[0031] Figure 15 This is a flowchart illustrating a method for automatically identifying a display device performed by an electronic device according to an embodiment of the present disclosure;

[0032] Figure 16 and Figure 17 This is a diagram illustrating the position information of the display module according to an embodiment of the present disclosure;

[0033] Figure 18 This is a flowchart illustrating the operation of an electronic device, a source device, and a display device according to embodiments of the present disclosure; and

[0034] Figure 19 This is a diagram illustrating the red, green, and blue (RGB) color space and the hue, saturation, and value (HSV) color space according to embodiments of the present disclosure. Detailed Implementation

[0035] Throughout this disclosure, the expression "at least one of a, b or c" means only a, only b, only c, both a and b, both a and c, both b and c, all of a, b and c, or variations thereof.

[0036] One or more embodiments of this disclosure will be described more fully below with reference to the accompanying drawings, enabling those skilled in the art to implement one or more embodiments. However, this disclosure may be implemented in different forms and should not be construed as limited to the embodiments set forth herein.

[0037] The terms used herein may be general terms that are currently widely used in the art based on the functions described in this disclosure, but may refer to various other terms depending on the intent of those skilled in the art, precedent, the emergence of new technologies, etc. Therefore, the terms used herein should not be defined by their simple names, but rather based on the meaning of the terms and the overall description of this disclosure.

[0038] Furthermore, the terminology used herein is only for describing specific embodiments of this disclosure and is not intended to limit this disclosure.

[0039] Throughout this specification, it should be understood that when a component is referred to as being “connected” or “coupled” to another component, it can be “directly connected” or “electrically coupled” to that other component, with one or more intermediate elements in between.

[0040] In this specification (especially in the appended claims), the use of the term "described" and similar designations should be interpreted to cover both the singular and plural. Furthermore, unless the order of operations is expressly specified herein, the operations of the methods of this disclosure described herein may be performed in any suitable order. The embodiments are not limited to the described order of operations.

[0041] Representations such as “in the embodiments of this disclosure” described in various parts of this specification do not necessarily refer to the same embodiments.

[0042] Embodiments of this disclosure may be described in terms of functional block components and various processing operations. Some or all of such functional blocks may be implemented by any number of hardware and / or software components performing a specific function. For example, functional blocks of this disclosure may be implemented by one or more microprocessors or circuit components for performing a specific function. Furthermore, functional blocks according to this disclosure may be implemented using various programming languages ​​or scripting languages, for example. Functional blocks may be implemented using various algorithms executed by one or more processors. Furthermore, this disclosure may employ techniques from related technical fields for electronic configuration, signal processing, and / or data processing. Terms such as “mechanism,” “element,” “device,” and “configuration” may be used broadly and are not limited to mechanical or physical components.

[0043] Furthermore, the connecting lines or connectors shown in the various figures are intended to illustrate exemplary functional relationships and / or physical or logical couplings between the components in the figures. In actual devices, connections between components may be represented by a number of alternative or additional functional relationships, physical connections, or logical connections.

[0044] As used herein, the term "unit" or "module" refers to a unit for performing at least one function or operation, and can be implemented using hardware, software, or a combination of hardware and software.

[0045] Furthermore, in this specification, the term "user" refers to a person who uses the image processing equipment, and may include consumers, evaluators, viewers, administrators, or installation engineers. Additionally, in this specification, "manufacturer" or "supplier" may refer to the manufacturer of the electronic equipment and / or the components included in the electronic equipment.

[0046] As used in this article, “image” can refer to a still image, picture, frame, moving image composed of multiple consecutive still images, or video.

[0047] As used herein, a processor may include various processing circuitry and / or multiple processors. For example, as used herein (including the claims), the term "processor" may include various processing circuitry, including at least one processor, wherein one or more of the at least one processor may be configured individually and / or collectively in a distributed manner to perform the various functions described herein. As used herein, when "processor," "at least one processor," and "one or more processors" are described as being configured to perform multiple functions, these terms cover, for example, but not limited to, a situation where one processor performs some of the functions and another processor performs other functions, and also cover the situation where a single processor can perform all the described functions. Additionally, at least one processor may include, for example, a combination of processors performing various described / disclosed functions in a distributed manner. At least one processor may execute program instructions to implement or perform various functions.

[0048] Figure 1 This is a diagram illustrating a display device 100 and an electronic device 200 according to embodiments of the present disclosure.

[0049] refer to Figure 1 The display device 100 can display video data. The display device 100 can be implemented as a TV, but embodiments are not limited to this, and any device with display capabilities (e.g., video wall, large-format display (LFD), digital signage, digital information display (DID), or projection display) can be used without limitation. Furthermore, the display device 100 can be implemented as various types of displays, such as liquid crystal display (LCD), organic light-emitting diode (OLED), liquid crystal on silicon (LCoS), digital light processing (DLP), quantum dot (QD) display panel, or quantum light-emitting diode (QLED).

[0050] Display device 100 may include a plurality of display modules 10, which may include, for example, display modules 10-1 to 10-n. The plurality of display modules 10 may be combined to implement a display device (e.g., display device 100).

[0051] According to embodiments of this disclosure, a plurality of display modules 10 disposed in a display device 100 may be interconnected. Each of the plurality of display modules 10 may output an image corresponding to video data received from a source device. Here, the source device may include various devices that provide content to the display device 100. For example, the source device may be an electronic device 200, but is not limited thereto, and may be implemented as a device separate from the electronic device 200.

[0052] Display device 100 may include multiple display modules, and the multiple display modules may be combined with each other to form a display screen. To this end, each of the multiple display modules needs to receive video data from a source device and also check which part of the video data needs to be displayed.

[0053] For example, at least one display module that receives control signals, video data, etc. from a source device can send the received control signals, video data, etc. to another display module connected in series with it, and correspondingly, send the control signals, video data, etc. to all the plurality of display modules 10 in sequence.

[0054] Alternatively, for example, a processor provided in the display device 100 can send control signals, video data, etc., corresponding to the position of each of the plurality of display modules 10, based on the position information of each of the plurality of display modules 10. Therefore, the display device 100 can output an image corresponding to the video data received from the source device.

[0055] As described above, in a multi-screen display where multiple display modules 10 constitute a complete display screen, each of the multiple display modules 10 can display a corresponding portion of an image divided from an image. As described above, in order to display the corresponding portion of the image corresponding to each of the multiple display modules 10, it is necessary to obtain the position and arrangement relationship of each of the multiple display modules 10 constituting the multi-screen display. The position and arrangement relationship of each of the multiple display modules 10 constituting the multi-screen display can be obtained through manual user settings or automatic identification.

[0056] According to the example, the position and arrangement of each of the multiple display modules 10 constituting a multi-screen display can be obtained through manual settings by the user. However, as the resolution, size, etc., of the display device 100 increases, the number of multiple display modules 10 constituting the display device 100 also increases. Therefore, it may be difficult for the user (or the installer of the display device 100) to manually set the position, arrangement, communication connection, etc., of each of the multiple display modules 10 in the display device 100.

[0057] According to the example, the position and arrangement of each of the multiple display modules 10 constituting a multi-screen display can be obtained through automatic identification performed by the user. For example, electronic device 200 can automatically identify the position, arrangement, communication connection, etc., of each of the multiple display modules 10 constituting display device 100 by analyzing images obtained by capturing images of display device 100. Electronic device 200 can automatically obtain information related to the position of each of the multiple display modules 10 of display device 100 without manual input by the user and provide this information to display device 100. Therefore, the installation accuracy and user convenience of display device 100 can be improved.

[0058] As described above, the electronic device 200 can obtain information about the position of each of the plurality of display modules 10 based on the results of analyzing the image obtained by capturing the display device 100 (hereinafter referred to as the captured image), thereby obtaining the position of each of the plurality of display modules 10 more accurately, and this automatic recognition process does not require user intervention, thereby improving user convenience.

[0059] Meanwhile, in order for the electronic device 200 to accurately obtain the position and arrangement relationship of the display modules in the above-described automatic identification method, it is necessary to accurately detect the screen area configured by the multiple display modules 10. For example, the electronic device 200 obtains information related to the position of each of the multiple display modules 10 by using the aspect ratio information of the identified screen area (e.g., the ratio of width W to height H), therefore, it is necessary to accurately detect the screen area.

[0060] When display device 100 displays an image in a captured image, electronic device 200 can identify the area of ​​output light as the screen area. For example, electronic device 200 can identify the screen area based on the luminance component (or value component) present in the captured image. However, when there is peripheral illumination or reflected light from display device 100, electronic device 200 has difficulty distinguishing between light output by display device 100 and light output from peripheral illumination, etc., making it difficult to accurately identify the screen area. Therefore, the user must perform additional tasks, such as blocking the peripheral illumination of display device 100 or adjusting the camera angle of electronic device 200, in order to accurately identify the screen area of ​​display device 100 through electronic device 200.

[0061] In embodiments of this disclosure, examples of methods performed by electronic device 200 for accurately detecting screen areas of display device 100 without additional work by the user are described below.

[0062] Figure 2 This is a diagram showing a display device 100 before and after setup according to an embodiment of the present disclosure.

[0063] Figure 2 For ease of illustration, it is assumed that a total of 18 display modules (e.g., 10-1, ..., 10-18) constitute a display device 100.

[0064] refer to Figure 2 In step 201, before setting position information indicating where each of the multiple display modules is located in the display device 100 and layout information of the multiple display modules 10, the image output by the display device 100 is different from the image sent to the display device 100 by the source device.

[0065] Layout information refers to the arrangement relationship information of multiple display modules 10 or N×M information (N: columns and M: rows). For example, layout information indicates that a total of 18 display modules are arranged in a 6×3 layout.

[0066] For example, before setting the position information and layout information of each of the multiple display modules 10, the following problem exists: each of the multiple display modules 10 does not display the portion of the image corresponding to the position of each of the multiple display modules 10, but instead... Figure 2 As shown in 201, the same portion of the entire image is output.

[0067] For example, since each of the multiple display modules 10 is initially set to the (0,0) coordinates of the output image, the installation position of each of the multiple display modules 10 needs to be set when the display device 100 is installed.

[0068] According to embodiments of this disclosure, even if the user does not manually input at least one of the position information of each of the plurality of display modules 10 or the layout information of the plurality of display modules 10 into the display device 100, the electronic device 200 can automatically obtain at least one of the position information or layout information, and send at least one of the position information or layout information to the display device 100.

[0069] For example, electronic device 200 can identify the screen area of ​​a display device 100 configured with multiple display modules 10 based on a captured image 400 received by a camera. Electronic device 200 can identify that the multiple display modules 10 are arranged in a 6×3 layout based on the identified screen area. Electronic device 200 can assign the (0,0) coordinate to the first display module 10-1 located on the upper left side of the identified screen area, and assign the (2,5) coordinate to the 18th display module 10-18 located on the lower right side of the screen area (see...). Figure 16 and Figure 17The electronic device 200 can send the position information of each of the multiple display modules 10 and the layout information of the multiple display modules 10 to the display device 100.

[0070] refer to Figure 2 In step 202, when position information and layout information of each of the plurality of display modules 10 are set in the display device 100, the display device 100 can output a partial image corresponding to the position of each of the plurality of display modules 10 based on the position information of each of the plurality of display modules 10. After setting, the display device 100 can output the image received from the source device (i.e., the entire image). The image output by the display device 100 can be the same as the image sent to the display device 100 by the source device.

[0071] Figure 3 This is a diagram illustrating the screen area of ​​a display device 100 recognized by an electronic device 200 according to an embodiment of the present disclosure.

[0072] refer to Figure 3 According to embodiments of the present disclosure, the electronic device 200 can use information relating to the screen area 50 of the display device 100 to identify at least one of the position information of each of the plurality of display modules or the layout information of the plurality of display modules. The screen area 50 is an area configured by the plurality of display modules and may correspond to the area where the plurality of display modules output images. The screen area 50 may be the display area of ​​the display device 100.

[0073] Electronic device 200 can identify screen area 50 based on captured image 400 received by camera. Electronic device 200 can identify the position of each display module within screen area 50, assign coordinates corresponding to the position of each display module, and send the assigned coordinates to display device 100.

[0074] When the display device 100 displays the image in the captured image 400, the electronic device 200 can identify the area of ​​output light as the screen area 50. The electronic device 200 can identify the screen area 50 based on the luminance component (or value component) present in the captured image 400.

[0075] However, when a user captures a picture of a display device 100 using electronic device 200, there may be illumination or reflected light in the peripheral areas 60 and 70 of the display device 100 unless the user performs additional operations (such as covering the peripheral lighting around the display device 100 or adjusting the camera angle of electronic device 200). For example, in addition to the light output from screen area 50, the captured image 400 may include peripheral light present in the periphery of the display device 100, and in this case, electronic device 200 may have difficulty accurately distinguishing screen area 50 from peripheral areas 60 and 70.

[0076] Peripheral areas 60 and 70 may correspond to areas located around the periphery of screen area 50. Peripheral light may be caused by reflected light, external light sources, internal lighting, etc., present in the surrounding environment of display device 100 (e.g., walls, floor, ceiling, lighting, etc.), and may exist in peripheral areas 60 and 70.

[0077] For example, when electronic device 200 identifies a screen area based on value components in captured image 400 without removing the influence of peripheral light present in peripheral areas 60 and 70, electronic device 200 may identify the first identification area 410 as the screen area. Since the first identification area 410 differs from the actual screen area 50, installation errors of display device 100 may occur when setting the position information of each of the plurality of display modules or the layout information of the plurality of display modules based on the first identification area 410.

[0078] For example, electronic device 200 can perform image processing on captured image 400 to remove the influence of peripheral light present in peripheral regions 60 and 70 of captured image 400. Electronic device 200 can identify the area where light exists by using captured image 400 with the peripheral light removed after image processing, thereby identifying second identification area 420 as screen area. Since second identification area 420 is the same as actual screen area 50, the recognition accuracy of screen area 50 is improved, and the installation error of display device 100 can be minimized.

[0079] Figure 4 This is a flowchart illustrating a method performed by an electronic device 200 for setting up a display device 100 according to an embodiment of the present disclosure.

[0080] refer to Figure 4 In operation 401, the electronic device 200 according to an embodiment of the present disclosure can capture the display device 100. The electronic device 200 can perform capture on a plurality of display modules constituting the display device 100 via a camera. The electronic device 200 can obtain a captured image by capturing the display device 100.

[0081] For example, electronic device 200 can obtain a first captured image by capturing a display device 100 with its screen on. In the first captured image, the value data of the screen area and the value data of the peripheral area can be relatively large. Electronic device 200 can obtain a second captured image by capturing a display device 100 with its screen off. In the second captured image, the value data of the screen area can be relatively small, and the value data of the peripheral area can be relatively large.

[0082] In operation 402, the electronic device 200 according to an embodiment of the present disclosure can identify a screen area configured by a plurality of display modules based on a captured image.

[0083] In the captured image according to an embodiment of the present disclosure, in addition to the light emitted from the screen area of ​​the display device 100, there may also be light from the peripheral area of ​​the display device 100.

[0084] An electronic device 200 according to an embodiment of the present disclosure can remove data in a captured image that corresponds to light in a peripheral region of a display device 100 by performing image processing on the captured image.

[0085] For example, electronic device 200 can obtain an image (hereinafter referred to as second operational image 900) with larger value data and an image (hereinafter referred to as third operational image 1000) with smaller value data for a screen region by using a first captured image and a second captured image. Electronic device 200 can obtain an image (hereinafter referred to as result image 1100) in one of the second operational image 900 and the third operational image 1000 that includes only the larger value data of the screen region by performing an XOR operation on the second operational image 900 and the third operational image 1000. Since value data of regions other than the screen region (e.g., peripheral regions) can be removed from the result image 1100, electronic device 200 can accurately distinguish the screen region from the peripheral region in the result image 1100. Electronic device 200 can determine the region with larger value data as the screen region in the result image 1100. In an embodiment, the region with larger value data may include regions in the result image with value data larger than a predetermined threshold.

[0086] refer to Figures 6 to 1 4. An example that describes this situation in more detail.

[0087] In operation 403, the electronic device 200 according to an embodiment of the present disclosure may obtain at least one of layout information or position information of the display device 100 based on captured images.

[0088] An electronic device 200 according to embodiments of the present disclosure can identify the ratio of a screen area based on the identified screen area. For example, the electronic device 200 can identify the ratio of the width to the height of the screen area or the aspect ratio of the screen area.

[0089] An electronic device 200 according to an embodiment of the present disclosure can obtain N×M layout information of a plurality of display modules by using aspect ratio information of the screen area.

[0090] An electronic device 200 according to an embodiment of the present disclosure can assign the coordinates of a first display module located on the upper left side of an identified screen area to a last display module using layout information. The electronic device 200 can obtain position information for each of the plurality of display modules by assigning coordinates to each of the plurality of display modules.

[0091] refer to Figure 15 An example describing this situation in more detail.

[0092] In operation 404, the electronic device 200 according to an embodiment of the present disclosure may send at least one of the layout information or position information of the display device 100 to the display device 100.

[0093] In operation 405, the display device 100 according to embodiments of the present disclosure can control a plurality of display modules based on at least one of layout information or position information. For example, the display device 100 can set the position of each of the plurality of display modules based on at least one of layout information or position information. The display device 100 can display each portion of the image corresponding to the position of each of the plurality of display modules in the entire image. The image output by the display device 100 can be the same as the image sent to the display device 100 by the source device.

[0094] Figure 5 This is a block diagram illustrating the configuration of an electronic device 200 and a display device 100 according to embodiments of the present disclosure.

[0095] refer to Figure 5 The electronic device 200 may include at least one of the following: smartphone, tablet PC, mobile phone, video phone, e-book reader, desktop PC, laptop PC, netbook computer, workstation, server, PDA, portable multimedia player (PMP), MP3 player, medical device, camera, virtual reality (VR) device or wearable device.

[0096] Electronic device 200 may include processor 210, memory 220, communication module 230, and camera 240. However, Figure 5 The components shown are not all necessary. Electronic device 200 can be composed of more than [missing components]. Figure 5The components shown can be implemented with more components, or the electronic device 200 can be implemented with fewer components.

[0097] Processor 210 can control the overall operation of electronic device 200. Processor 210 can be implemented as one or more processors. Processor 210 can execute instructions or commands stored in memory 220 to perform specific operations. For example, processor 210 can execute one or more instructions of a program stored in memory 220 to control the overall operation of electronic device 200 to identify the screen area of ​​display device 100.

[0098] Processor 210 may include at least one of a central processing unit (CPU), a graphics processing unit (GPU), an accelerated processing unit (APU), an integrated many-core processor (MIC), a digital signal processor (DSP), or a neural processing unit (NPU). Processor 210 may be implemented as an integrated system-on-a-chip (SoC) including one or more electronic components. Each of the one or more processors may be implemented as separate hardware.

[0099] Memory 220 can store various information, data, instructions, and programs necessary for the operation of electronic device 200. Memory 220 may include at least one of volatile memory or non-volatile memory or a combination thereof. Memory 220 may include at least one type of storage medium selected from flash memory, hard disk, micro multimedia card, card type memory (e.g., SD or XD memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic storage, magnetic disk, or optical disk.

[0100] The communication module 230 can communicate with the display device 100. For example, the communication module 230 can send control signals to the display device 100 wirelessly. For example, the communication module 230 can send at least one of the position information of each of the plurality of display modules or the layout information of the plurality of display modules to the display device 100. Alternatively, for example, the communication module 230 can send at least one of the position information of each of the plurality of display modules or the layout information of the plurality of display modules to the display device 100 through a source device.

[0101] Communication module 230 may include at least one or a combination of short-range and long-range communication modules. Communication module 230 may include at least one antenna for wireless communication with other devices. Short-range wireless communication modules may include, but are not limited to, Bluetooth communication modules, Bluetooth Low Energy (BLE) communication modules, near-field communication modules, WLAN (Wi-Fi) communication modules, Zigbee communication modules, Infrared Data Association (IrDA) communication modules, Wi-Fi Direct (WFD) communication modules, ultra-wideband (UWB) communication modules, Anti+ communication modules, uWave communication modules, etc.

[0102] The camera 240 can be located at a certain distance from the display device 100 and can capture images of multiple display modules constituting the display device 100. For example, the camera 240 can send images obtained by capturing images of the display device 100 to the processor 210.

[0103] In embodiments of this disclosure, camera 240 is shown as a component disposed in electronic device 200, but is not limited thereto, and may be implemented in an external electronic device different from electronic device 200.

[0104] According to embodiments of the present disclosure, the processor 210 can identify a screen area of ​​the display device 100 by capturing an image obtained from the camera 240 capturing the display device 100. Hereinafter, reference is made to... Figures 6 to 12 This section describes in detail an example of how the processor 210 identifies the screen area of ​​the display device 100.

[0105] Display device 100 may include processor 110, memory 120, communication module 130, and display 140. However, Figure 5 Not all of the components shown are necessary. The display device 100 can be composed of components such as... Figure 5 The components shown can be implemented with more components, or the display device 100 can be implemented with fewer components.

[0106] Processor 110 can control the overall operation of display device 100. Processor 110 can be implemented as one or more processors. Processor 110 can execute instructions or commands stored in memory 120 to perform specific operations. For example, processor 110 can execute one or more instructions of a program stored in memory 120 to control the overall operation of display device 100 to identify screen areas of display device 100.

[0107] Processor 210 may include at least one of a CPU, GPU, APU, MIC, DS, or NPU. Processor 110 may be implemented as an integrated SoC including one or more electronic components. Each of the one or more processors may be implemented as separate hardware.

[0108] The memory 120 can store various information, data, instructions, and programs required for the operation of the display device 100. The memory 120 may include at least one of volatile memory or non-volatile memory or a combination thereof.

[0109] The communication module 130 can communicate with the electronic device 200. For example, the communication module 130 can send control signals to the electronic device 200 wirelessly. For example, the communication module 130 can receive at least one of the following from the electronic device 200: position information of each of the plurality of display modules or layout information of the plurality of display modules.

[0110] The display 140 may include a plurality of display modules. Each of the plurality of display modules according to embodiments of the present disclosure may include a plurality of self-emissive elements. Here, the self-emissive elements may be at least one of light-emitting diodes (LEDs) or micro LEDs.

[0111] Furthermore, each of the multiple display modules can be implemented as an LED cabinet comprising multiple LED elements. Here, the LED elements can be implemented as RGB LEDs, and RGB LEDs can include red LEDs, green LEDs, and blue LEDs. In addition to RGB LEDs, the LED elements can also include white LEDs.

[0112] LED elements can be implemented as micro-LEDs. Here, a micro-LED is an LED with a size of about 5 micrometers to about 100 micrometers, and is an ultra-small light-emitting element that emits light on its own without a color filter.

[0113] According to embodiments of the present disclosure, the processor 110 can receive, via the communication module 130, at least one of the position information of each of a plurality of display modules or the layout information of the plurality of display modules from the electronic device 200.

[0114] Figure 6 This is a diagram illustrating the operation of obtaining a first captured image 600 according to an embodiment of the present disclosure. Figure 6 In this context, the first captured image 600 can be represented as RGB data.

[0115] refer to Figure 6 According to an embodiment of the present disclosure, the processor 210 can receive a first captured image 600 obtained by the camera 240 capturing the display device 100.

[0116] The display device 100 according to an embodiment of the present disclosure can be in a state of outputting a first image 610. The first image 610 can be an image formed by multiple display modules outputting the same image. For example, multiple display modules can output images with the same pattern or color, so that the display device 100 can output a first image 610 with a single pattern or a first image 610 with a single color.

[0117] According to embodiments of the present disclosure, the display device 100 can output a first image 610 on a screen area 50. The screen area 50 may correspond to the area on which the display device 100 displays the first image 610. The first image 610 may be an image received from a source device.

[0118] According to an embodiment of the present disclosure, the processor 210 can receive a first captured image 600 obtained by capturing a display device 100 in a state of outputting a first image 610.

[0119] The first captured image 600 according to embodiments of the present disclosure may include a display device 100 that is outputting the first image 610 and the surrounding environment (e.g., walls, floor, ceiling, lighting, etc.) where the display device 100 is located. For example, the first captured image 600 may include a screen area 50 of the display device 100 and peripheral areas 60 and 70 corresponding to the periphery of the screen area 50.

[0120] For example, the light output from the display device 100 may exist in the screen area 50 of the first captured image 600. The screen area 50 may correspond to the area where the first image 610 is output. For example, when the RGB data of the first image 610 is a single color (255,0,0), the RGB data of the screen area 50 of the first captured image 600 may be (255,0,0).

[0121] For example, peripheral light generated by reflected light, external light sources, internal lighting, etc., present in the periphery of the display device 100 may exist in the peripheral regions 60 and 70 of the first captured image 600. For example, illumination reflected from the peripheral walls and ceiling of the display device 100 (e.g., fluorescent lighting, external light sources, etc.) may exist in the peripheral region 60. For example, reflected light reflected from the bottom periphery of the display device 100 may exist in the peripheral region 70. For example, the RGB data of the peripheral regions 60 and 70 of the first captured image 600 may be (255, 255, 255).

[0122] According to embodiments of this disclosure, when the RGB data present in the first captured image 600 is converted into value V data in Hue, Saturation, and Value (HSV) format, the first captured image 600 may include value components of screen region 50 and value components of peripheral regions 60 and 70. The value components included in the first captured image 600 may be represented as described below. Figure 9 As shown.

[0123] Here, this value can indicate the level of brightness. This value can be represented as a component, a data value, or a total value. The presence of this component corresponds to a relatively large data value, and the absence of this component corresponds to a relatively small data value.

[0124] Figure 6 The first image 610 shown, output by the display device 100, is merely an example, and the display device 100 can output a first image 610 in which various colors are combined or various patterns are combined. For example, multiple display modules can output images with different patterns or different colors.

[0125] Figure 7 This is a diagram illustrating the operation of obtaining a second captured image 700 according to an embodiment of the present disclosure. Figure 7 In this context, the second captured image 700 can be represented as RGB data.

[0126] refer to Figure 7 According to an embodiment of the present disclosure, the processor 210 can receive a second captured image 700 obtained by the camera 240 capturing the display device 100.

[0127] The display device 100 according to embodiments of the present disclosure can be in a state where it does not output images. The display device 100 can also be in a state where the screen is turned off.

[0128] According to an embodiment of the present disclosure, the processor 210 can receive a second captured image 700 obtained by capturing a display device 100 in a state where the screen is off.

[0129] The second captured image 700 according to an embodiment of the present disclosure may include a display device 100 in a state of not outputting an image and the surrounding environment in which the display device 100 is located.

[0130] For example, the light output from the display device 100 may not be present in the screen area 50 of the second captured image 700. For example, the screen area 50 of the second captured image 700 may have black data in which the R, G, and B values ​​are (0,0,0).

[0131] For example, peripheral light caused by reflected light, external light sources, internal lighting, etc., present in the periphery of the display device 100 may exist in the peripheral regions 60 and 70 of the second captured image 700. For example, internal lighting (e.g., fluorescent lighting, external light sources, etc.) reflected from the peripheral walls and ceiling of the display device 100 may exist in the peripheral region 60. For example, reflected light reflected from the bottom of the display device 100 may exist in the peripheral region 70. For example, the RGB data of the peripheral regions 60 and 70 of the second captured image 700 may be (255, 255, 255).

[0132] According to embodiments of this disclosure, when the RGB data present in the second captured image 700 is converted into HSV format value data, the second captured image 700 may not include the value data of the screen area 50, but may include the value data of the peripheral areas 60 and 70. The value data included in the second captured image 700 can be represented as described below. Figure 10 As shown.

[0133] However, the embodiments are not limited to this, and light may also be present in the screen area 50 of the second captured image 700 according to the embodiments of this disclosure. For example, in the screen area 50 of the second captured image 700, no light is generated from the screen of the display device 100, but there is reflected light from illumination in the surrounding environment. In this case, the screen area 50 of the second captured image 700 may at least include a portion of the area where the R, G, and B values ​​are data other than (0,0,0) (e.g., (30,30,30)).

[0134] In this case, the processor 210 can obtain an image that has explicitly removed the value components present in the screen region 50 through specific operations on the first captured image 600 and the second captured image 700. This specific operation can be the first operation (see...). Figure 8 The first operation can be performed to improve the recognition accuracy of the screen area, or it can be omitted for ease of operation.

[0135] According to an embodiment of the present disclosure, the processor 210 can generate a third operation image 1000 by extracting value data from the second captured image 700 or the first operation image 800.

[0136] Figure 8 This is a diagram illustrating a first operational image 800 according to an embodiment of the present disclosure. Figure 8 In this context, the first operational image 800 can be represented as RGB data.

[0137] refer to Figure 8According to an embodiment of the present disclosure, the processor 210 can obtain a first operation image 800 by performing a first operation on the first captured image 600 and the second captured image 700.

[0138] For example, processor 210 can perform an AND operation on a first captured image 600 with larger RGB data for screen region 50 and peripheral regions 60 and 70, and a second captured image 700 with larger RGB data for peripheral regions 60 and 70. Processor 210 can obtain a first operational image 800 with larger RGB data for peripheral regions 60 and 70 by performing an AND operation on the first captured image 600 and the second captured image 700. In the first operational image 800, screen region 50 may have smaller RGB data, or may have no RGB data. For example, in the first operational image 800, the RGB data of screen region 50 may be (30,0,0) or (0,0,0), while the RGB data of peripheral regions 60 and 70 may be (255,255,255). However, the first operation is not limited to an AND operation and can be replaced by various operations capable of removing the value components of screen region 50.

[0139] The AND operation is an operator that produces a result of 1 only if all operands have the same value, and a result of 0 even if one operand has a different value.

[0140] because Figure 8 A first operation image 800 is shown, obtained by performing a first operation using a second captured image 700 in which reflected light exists in screen area 50. Therefore, some RGB data may exist in screen area 50. However, the embodiment is not limited to this, and when the first operation is performed using a second captured image 700 in which no reflected light exists in screen area 50, RGB data may not exist in screen area 50 of the first operation image 800.

[0141] According to embodiments of this disclosure, when the RGB data present in the first operational image 800 is converted into HSV format value data, the first operational image 800 may not include the value data of the screen area 50 and may include the value data of the peripheral areas 60 and 70. This is because the RGB data of the screen area 50 of the first operational image 800 has relatively small value components or no value components. The value data included in the first operational image 800 can be represented as described below. Figure 10 As shown.

[0142] For example, even if some RGB data exists in screen area 50 of the first operating image 800, the value data may not be extracted because the RGB data is converted to HSV data.

[0143] Therefore, by performing an AND operation on the first captured image 600 and the second captured image 700, an image can be generated that has had the value data of the screen area 50 removed and only the value data of the outer areas 60 and 70 remain.

[0144] According to an embodiment of the present disclosure, the processor 210 can obtain an image in which the value data of the screen region 50 is explicitly removed by performing a first operation on the first captured image 600 and the second captured image 700.

[0145] For example, processor 210 can explicitly remove the reflected light component present in screen region 50 by performing a first operation on a first captured image 600 and a second captured image 700 in which reflected light exists in screen region 50. Processor 210 can generate a third operated image 1000 by explicitly extracting value data of peripheral regions 60 and 70 using the first operated image 800.

[0146] Alternatively, the processor 210 according to an embodiment of the present disclosure can generate a third operation image 1000 by omitting the first operation process for ease of operation and extracting value data from the second captured image 700.

[0147] Figure 9 This is a diagram illustrating a second operational image 900 according to an embodiment of the present disclosure. Figure 9 In this context, the second operational image 900 can be represented as V data in HSV format.

[0148] refer to Figure 9 According to an embodiment of the present disclosure, the processor 210 can obtain the second operation image 900 by extracting the value components of the first captured image 600.

[0149] The processor 210 according to embodiments of this disclosure can convert an RGB format image to an HSV format image and extract value components (V components). For example, the processor 210 can convert a first captured image 600 in RGB format to a first captured image 600 in HSV format. The processor 210 can obtain a second operational image 900 from which value components have been extracted from the first captured image 600 converted to HSV format. The processor 210 can represent the value components present in the first captured image 600 as value data. In the second operational image 900, each pixel can be represented as value data.

[0150] RGB format is a three-channel format where the pixel values ​​of an image are represented as red data (R), green data (G), and blue data (B). For example, when each data is represented as 8 bits, red, green, and blue each have one of 256 levels. Colors can be represented by mixing red, green, and blue, each represented at its respective level.

[0151] Like the RGB format, the HSV format can display images in three channels. The HSV format represents the pixel values ​​of an image as hue (H) data, saturation (S) data, and value (V) data. The darker the image, the smaller the value data, and the brighter the image, the larger the value data.

[0152] In embodiments of this disclosure, reference is made to Figure 19 An example of a method for obtaining value components from an RGB format image is described.

[0153] According to an embodiment of this disclosure, the processor 210 can extract value components from a first captured image 600, which includes a screen region 50 containing the value components and peripheral regions 60 and 70 containing the value components. The processor 210 can obtain a second operational image 900 containing the value components of the screen region 50 and the peripheral regions 60 and 70.

[0154] Since the hue and saturation components have been removed from the second operational image 900, each region of the second operational image 900 can be classified according to the magnitude of the value data. For example, in the second operational image 900, regions with larger value data and regions with smaller value data can be classified. For example, in the second operational image 900, screen region 50 with larger value data, as well as peripheral regions 60 and 70, can be displayed as white, while other regions can be displayed entirely as black.

[0155] Figure 10 This is a diagram illustrating a third operational image 1000 according to an embodiment of the present disclosure. Figure 10 In this context, the third operation image 1000 can be represented as V data in HSV format.

[0156] refer to Figure 10 According to an embodiment of the present disclosure, the processor 210 can obtain the third operation image 1000 by extracting the value components of the second captured image 700.

[0157] Alternatively, the processor 210 according to an embodiment of the present disclosure can obtain the third operation image 1000 by extracting the value components of the first operation image 800.

[0158] In the following text, Figure 10 The value data of the third operational image 1000 is shown to correspond to the value data of the second captured image 700.

[0159] The processor 210 according to embodiments of this disclosure can convert an RGB format image to an HSV format image and extract value components (V components). For example, the processor 210 can convert a second captured image 700 in RGB format to a second captured image 700 in HSV format. The processor 210 can obtain a third operational image 1000 from which value components have been extracted from the second captured image 700 converted to HSV format. The processor 210 can represent the value components present in the second captured image 700 as value data. In the third operational image 1000, each pixel can be represented as value data.

[0160] According to an embodiment of this disclosure, the processor 210 can extract value components from a second captured image 700, the first captured image 600 including a screen region 50 where no value components exist and peripheral regions 60 and 70 where value components exist. The processor 210 can obtain a third operational image 1000 that retains the value components of the peripheral regions 60 and 70, but excludes the screen region 50. The third operational image 1000 may not contain the value components of the screen region 50.

[0161] Since the hue and saturation components have been removed from the third operational image 1000, each region of the third operational image 1000 can be classified according to the magnitude of the value data. For example, in the third operational image 1000, the outer regions 60 and 70 with larger value data can be displayed as white, while other regions can be displayed as black. For example, in the third operational image 1000, the screen region 50 with smaller value data can be displayed as black.

[0162] Meanwhile, in the embodiments of this disclosure, when a first operation is added to obtain the first operation image 800, the value data of the third operation image 1000 can correspond to the value data of the first operation image 800.

[0163] Figure 11 This is a diagram showing a resulting image 1100 according to an embodiment of the present disclosure.

[0164] refer to Figure 11 The processor 210 can obtain the result image 1100 by performing a second operation on the second operation image 900 and the third operation image 1000. When the result image 1100 is obtained, the processor 210 can identify the screen area 50 of the display device 100.

[0165] For example, processor 210 can perform an XOR operation on a second operational image 900 that retains only the value components of screen region 50 and the value components of peripheral regions 60 and 70, and perform an XOR operation on a third operational image 1000 that retains only the value components of peripheral regions 60 and 70. Processor 210 can obtain a result image 1100 that includes the value components of screen region 50 by performing XOR operations on the second operational image 900 and the third operational image 1000.

[0166] The XOR operation is an operator that produces a result of 1 only if the two operands have different values, and a result of 0 even if the two operands have the same value.

[0167] Processor 210 can remove residual data by performing an XOR operation on the second operational image 900 and the third operational image 1000, leaving only the data in the region with the larger value in one of the two operational images. For example, processor 210 can leave only the value data in screen region 50 corresponding to the region with the larger value between the second operational image 900 and the third operational image 1000. Processor 210 can also remove the value data in the peripheral regions 60 and 70 that are typically present in the second operational image 900 and the third operational image 1000.

[0168] According to embodiments of this disclosure, the resulting image 1100 obtained from the second operation may include the value components of the screen region 50, but not the value components of the peripheral regions 60 and 70.

[0169] Each region of the resulting image 1100 can be categorized based on the magnitude of its value data. For example, in the resulting image 1100, screen region 50 with larger value data can be displayed in white, while other regions can be displayed entirely in black. For example, in the resulting image 1100, the outer regions 60 and 70 with smaller value data can be displayed in black.

[0170] According to embodiments of the present disclosure, the processor 210 can determine the region with larger value data as the screen region 50 in the resulting image 1100. The processor 210 can distinguish the screen region 50 in the resulting image 1100 from the peripheral regions 60 and 70 based on the magnitude of the value data.

[0171] For example, when the value data of the resulting image 1100 is greater than or equal to a threshold, the processor 210 can determine that the value data is large, and when the value data of the resulting image 1100 is less than the threshold, the processor 210 can determine that the value data is small. However, the embodiments are not limited to this.

[0172] Processor 210 can remove the influence of peripheral light present in peripheral regions 60 and 70 through a second operation, thus distinguishing screen region 50 in display device 100 from peripheral regions 60 and 70. Processor 210 can accurately identify screen region 50 included in the resulting image 1100.

[0173] Figure 12 This is a diagram illustrating the operation of the peripheral region in the processing result image 1100 according to an embodiment of the present disclosure.

[0174] refer to Figure 12 The value data of the peripheral region can be partially retained in the resulting image 1100. A resulting image 1100 in which the value data of the peripheral region 70 is retained is shown. Even when the value data of the peripheral region 70 is retained in the resulting image 1100, the processor 210 can still distinguish the screen region 50 from the peripheral region 70.

[0175] The resulting image 1100 may include first value data corresponding to the larger value data and second value data corresponding to the smaller value data. For example, the first value data may have a value of 1 or white, and the second value data may have a value of 0 or black.

[0176] According to an embodiment of the present disclosure, the processor 210 can read value data from the resulting image 1100, each comprising a predefined number of pixels. For example, the processor 210 can read pixel value data from the center of the image in the unit 1220 along the top, bottom, left, and right directions in a unit comprising a predefined number (e.g., 3, 5, etc.) of pixels.

[0177] When the processor 210 according to an embodiment of the present disclosure identifies a unit 1220 including a black pixel and identifies the unit 1220 having a black pixel repeatedly to a certain extent, the processor 210 can determine the position of the last pixel other than the black pixel as one end of the screen area 50.

[0178] When a second value data is identified in any cell, the processor 210 according to an embodiment of the present disclosure can identify the value data of the next cell adjacent to that cell. When the second value data is also identified in the next cell, the processor 210 can determine the position of the last cell in which the first value data can be identified as one end of the screen area 50.

[0179] For example, processor 210 can read the value data of unit 1220 from the center of image 1210 in a downward direction. Processor 210 can identify the value data of third unit 1223 as having a value of 1. Processor 210 can identify the value data of first unit 1221 corresponding to any unit as not having a value of 1. When a unit whose value data does not have a value of 1 is identified as repeated to a certain degree or greater, processor 210 can determine the position of third unit 1223 corresponding to the last unit whose value data has a value of 1 as the lower end 51 of screen area 50. For example, when processor 210 identifies a unit whose value data does not have a value of 1 even in second unit 1222, which is the next unit adjacent to first unit 1221, processor 210 can identify the unit whose value data does not have a value of 1 as repeated to a certain degree.

[0180] The processor 210 can determine the left, right and top edges of the screen area 50 in the same way as it determines the lower edge 51 of the screen area 50.

[0181] The processor 210 can identify the screen region 50 by determining the top, left, right and bottom 51 of the screen region 50 in the result image 1100.

[0182] Figure 13a This is a flowchart illustrating the operation of an electronic device 200 according to an embodiment of the present disclosure. Figure 13b This is a diagram illustrating the operation of an electronic device 200 obtaining a resulting image 1100 according to an embodiment of the present disclosure.

[0183] refer to Figure 13a and Figure 13b In operation 1310, the electronic device 200 according to an embodiment of the present disclosure can obtain a first captured image 600 by capturing a display device 100 in a state of outputting a first image 610 to a screen area 50.

[0184] According to an embodiment of the present disclosure, the electronic device 200 can capture the display device 100 with its screen turned on using a camera 240. According to an embodiment of the present disclosure, the display device 100 can be in a state of outputting a first image 610. The first image 610 can be an image formed by outputting the same image through multiple display modules.

[0185] The first captured image 600 according to an embodiment of the present disclosure may include value components present in screen region 50 and value components present in peripheral regions 60 and 70. When the RGB data of the first captured image 600 is converted into value data, the value data of screen region 50 and peripheral regions 60 and 70 may be large.

[0186] In operation 1320, the electronic device 200 according to an embodiment of the present disclosure can obtain a second captured image 700 by capturing the display device 100 in a state where the screen is off.

[0187] According to an embodiment of the present disclosure, the electronic device 200 can capture the display device 100 in a state where the screen is off using the camera 240. According to an embodiment of the present disclosure, the display device 100 can be in a state where it does not output images.

[0188] The second captured image 700 according to embodiments of the present disclosure may include value components present in peripheral regions 60 and 70. When the RGB data of the second captured image 700 is converted into value data, the value data of screen region 50 may be smaller, while the value data of peripheral regions 60 and 70 may be larger.

[0189] In operation 1330, the electronic device 200 according to an embodiment of the present disclosure can obtain a second operational image 900 from which value components have been extracted from the first captured image 600.

[0190] An electronic device 200 according to an embodiment of the present disclosure can convert a first captured image 600 in RGB format to a first captured image 600 in HSV format. The electronic device 200 can obtain a second operational image 900 from which value components have been extracted from the first captured image 600 converted to HSV format. In the second operational image 900, each pixel can be represented as value data.

[0191] An electronic device 200 according to an embodiment of the present disclosure can obtain a second operational image 900 containing value components of screen region 50 and peripheral regions 60 and 70 by extracting value components from a first captured image 600. For example, the second operational image 900 may be displayed as black, except for screen region 50 and peripheral regions 60 and 70 which have larger value data.

[0192] In operation 1340, the electronic device 200 according to an embodiment of the present disclosure can obtain a third operational image 1000 from which value components have been extracted from the second captured image 700.

[0193] According to embodiments of the present disclosure, the electronic device 200 can convert a second captured image 700 in RGB format to a second captured image 700 in HSV format. The electronic device 200 can obtain a third operational image 1000 from which value components have been extracted from the second captured image 700 converted to HSV format. In the third operational image 1000, each pixel can be represented as value data.

[0194] An electronic device 200 according to an embodiment of the present disclosure can obtain a second operational image 900 containing value components of peripheral regions 60 and 70 (excluding screen region 50) by extracting value components from a second captured image 700. For example, a third operational image 1000 may be displayed as black, except for peripheral regions 60 and 70 which have larger value data. In the third operational image 1000, screen region 50 which has smaller value data may be displayed as black.

[0195] In operation 1350, the electronic device 200 according to an embodiment of the present disclosure can identify the screen region 50 by the result image 1100 obtained by performing a second operation on the second operation image 900 and the third operation image 1000.

[0196] According to embodiments of this disclosure, the electronic device 200 can perform an XOR operation on the second operational image 900 and the third operational image 1000 to remove residual data, leaving only the data of the region with the larger value in one of the two operational images. For example, the electronic device 200 can retain only the value data corresponding to the region with the larger value in the screen area 50 of the second operational image 900. The electronic device 200 can also remove the value data of the peripheral regions 60 and 70 that are typically present in the second operational image 900 and the third operational image 1000.

[0197] The resulting image 1100 may include the value components of screen region 50, but may not include the value components of peripheral regions 60 and 70.

[0198] Each region of the resulting image 1100 can be categorized based on the magnitude of the value data. For example, the resulting image 1100 can be entirely displayed as black, except for screen region 50 with larger value data. For instance, in the resulting image 1100, the outer regions 60 and 70 with smaller value data can be displayed as black.

[0199] According to embodiments of the present disclosure, the electronic device 200 can identify screen region 50 by distinguishing screen regions 50 in the resulting image 1100 corresponding to regions with larger data values ​​from screen regions 50 corresponding to regions with smaller data values. The electronic device 200 can determine the region with larger data values ​​as screen region 50 in the resulting image 1100.

[0200] Electronic device 200 can accurately identify screen area 50 by performing processing on it to remove light present in the surrounding environment of display device 100. Based on the accurately identified screen area 50, electronic device 200 can obtain the position information of each of the plurality of display modules of display device 100 and the layout information of the plurality of display modules. Electronic device 200 can complete the installation of display device 100 by automatically identifying information about display device 100 and sending that information to display device 100.

[0201] Figure 14a This is a flowchart illustrating the operation of an electronic device 200 according to an embodiment of the present disclosure. Figure 14b This is a diagram illustrating the operation of an electronic device 200 obtaining a resulting image 1100 according to an embodiment of the present disclosure. Figure 14a and Figure 14b In addition, the electronic device 200 according to embodiments of the present disclosure may further include a first operation, and the part related to... Figure 13a The description is a repetitive description.

[0202] refer to Figure 14a and Figure 14b Operation 1410 corresponds to Figure 13a Operation 1310. Operation 1420 corresponds to... Figure 13a Operation 1320.

[0203] On the other hand, in the screen area 50 of the second captured image 700, no light is generated from the screen of the display device 100, but there is a case where light is reflected from the illumination of the surrounding environment. In this case, some RGB data other than (0,0,0) may exist in the screen area 50 of the second captured image 700.

[0204] In operation 1430, an electronic device 200 according to an embodiment of the present disclosure can obtain a first operation image 800 by performing a first operation on a first captured image 600 and a second captured image 700.

[0205] An electronic device 200 according to an embodiment of the present disclosure can obtain RGB data common to the first captured image 600 and the second captured image 700 by performing an AND operation on the first captured image 600 and the second captured image 700.

[0206] For example, electronic device 200 can obtain a first operational image 800 by performing an AND operation on the first captured image 600 and the second captured image 700 to obtain a first operational image 800 with larger RGB data in the peripheral regions 60 and 70 and smaller RGB data or no RGB data in the screen region 50. When the RGB data of the first operational image 800 is converted into value data, the peripheral regions 60 and 70 may have larger value data, and the screen region 50 may have smaller value data or no value data.

[0207] In operation 1440, the electronic device 200 according to an embodiment of the present disclosure can obtain a second operational image 900 from which value components have been extracted from the first captured image 600. Operation 1440 corresponds to Figure 13a Operation 1330.

[0208] In operation 1450, the electronic device 200 according to an embodiment of the present disclosure can obtain a third operation image 1000 from which value components have been extracted from the first operation image 800.

[0209] According to embodiments of the present disclosure, the electronic device 200 can convert a first operational image 800 in RGB format to a first operational image 800 in HSV format. The electronic device 200 can obtain a third operational image 1000 from which value components have been extracted from the first operational image 800 converted to HSV format. In the third operational image 1000, each pixel can be represented as value data.

[0210] An electronic device 200 according to an embodiment of the present disclosure can obtain a second operational image 900 containing value components of peripheral regions 60 and 70, except for screen region 50, by extracting value components from a first operational image 800. For example, a third operational image 1000 can be displayed as black, except for peripheral regions 60 and 70 with larger value data. In the third operational image 1000, screen region 50 with smaller value data can be displayed as black.

[0211] In operation 1460, the electronic device 200 according to an embodiment of the present disclosure can identify the screen region 50 by the result image 1100 obtained by performing a second operation on the second operation image 900 and the third operation image 1000. Operation 1460 corresponds to Figure 13a Operation 1350.

[0212] Figure 15 This is a flowchart illustrating a method for automatically identifying a display device 100 performed by an electronic device 200 according to an embodiment of the present disclosure. Figure 16 and Figure 17 This is a diagram illustrating the position information of the display module according to an embodiment of the present disclosure.

[0213] refer to Figure 15 In operation 1510, electronic device 200 can acquire the captured image from display device 100. Operation 1510 can correspond to... Figure 4 Operation 401.

[0214] In operation 1520, electronic device 200 can identify a screen area of ​​display device 100 based on the captured image. The captured image may include a first captured image 600 and a second captured image 700. Operation 1520 may correspond to Figure 4 Operation 402.

[0215] In operation 1530, electronic device 200 can obtain aspect ratio information of display device 100 based on the size information of the screen area in the captured image. The captured image may include at least one of a first captured image 600, a second captured image 700, a third captured image 1600, or a fourth captured image 1700. The third captured image 1600 may be the same as or different from the first captured image 600.

[0216] refer to Figure 16 In the third captured image 1600, the processor 210 can obtain aspect ratio information of the display device 100 based on the identified screen area 50. For example, the processor 210 can obtain aspect ratio information of the display device 100 by using the size information of the screen area 50, such as the ratio of the width W to the height H of the display device 100 or the aspect ratio of the display device 100.

[0217] For example, the processor 210 can identify the display device 100 as having a ratio of 16:9, or 1.778, based on the identified screen area 50.

[0218] In operation 1540, electronic device 200 can obtain layout information of multiple display modules 10 based on aspect ratio information of display device 100 and information about multiple display modules 10.

[0219] In embodiments of this disclosure, the plurality of display modules 10 may include at least one of the following: quantity information of the plurality of display modules 10 or size information of each of the plurality of display modules 10.

[0220] For example, refer to Figure 16 The plurality of display modules 10 may include 18 display modules, and the width W × height H of each of the plurality of display modules 10 may be 320 × 360.

[0221] The processor 210 can obtain multiple candidate layout information based on the number of display modules 10 (e.g., 18). For example, when the processor 210 identifies 18 display modules constituting the display device 100, the processor 210 can obtain multiple candidate layout information by identifying configurable N×M layouts (where N and M are integers). For example, the processor 210 can obtain a total of six candidate layout information, such as 1×18, 2×9, 3×6, 6×3, 9×2, and 18×1.

[0222] The processor 210 can obtain candidate ratios by multiplying each line of the obtained layout information by the size of the display module (e.g., 320×360). For example, when the size of the display module is 320×360, the processor 210 can obtain multiple candidate ratios (N×width:M×height) corresponding to multiple candidate layouts.

[0223] The processor 210 can identify candidate layouts with a ratio that is the same as or similar to 1.778 (which is the aspect ratio information of the display device 100) among the six candidate ratios corresponding to the six candidate layouts as the layout information of the display device 100. For example, the candidate ratio of the 6×3 layout among multiple candidate layouts could be (6×320px):(3×360px)=1920:1080=1.778. The processor 210 can identify the layout information of the display device 100 as 6×3.

[0224] In operation 1550, processor 210 can obtain position information of the region corresponding to each of the plurality of display modules 10 based on layout information. For example, processor 210 can assign (0,0) to the region corresponding to the first display module 10-1 and (2,5) to the region corresponding to the 18th display module 10-18. The specific figures are merely examples for illustrative purposes and are not limited thereto.

[0225] refer to Figure 17 The processor 210 can obtain the position of the display modules by performing a capture operation on the display device 100, which includes any display module that outputs different images (e.g., colors or patterns) (see fourth captured image 1700). Among the plurality of display modules 10 of the display device 100, any one display module (e.g., the fifteenth display module 10-15) can be in a state where it outputs an image different from the other display modules. The processor 210 can obtain the fourth captured image 1700 by capturing the display device 100 in a state where each display module outputs an image different from the other display modules.

[0226] The processor 210 can identify the area where any display module (e.g., the fifteenth display module 10-15) that outputs an image different from the fourth captured image 1700 is located.

[0227] The processor 210 can compare the third captured image 1600 with the fourth captured image 1700 to identify the region in the third captured image 1600 that is identified in the fourth captured image 1700 (i.e., the region on which a different image is displayed).

[0228] When a region identified in the fourth captured image 1700 (i.e., the region on which a different image is displayed) is identified as similar to (or matching) a region in the third captured image 1600 corresponding to the fifteenth display module 10-15, the processor 210 can obtain the position information assigned to the fifteenth display module 10-15 (e.g., (2,2)) as the position information of the fifteenth display module 10-15. The processor 210 can then send the obtained position information to the display device 100.

[0229] The processor 210 can obtain the position information of each display module by repeatedly performing a capture operation on the display device 100, which is in a state of repeatedly turning on and off different images on the screen of each display module.

[0230] The processor 210 can send at least one of the position information of each of the plurality of display modules 10 or the layout information of the plurality of display modules 10 to the display device 100 via the communication module 230. The processor 210 can complete the installation of the display device 100 by automatically identifying information about the display device 100 and sending that information to the display device 100. Figure 18 This is a flowchart illustrating the operation of electronic device 20, source device 300, and display device 100 according to embodiments of the present disclosure. Figure 18 The source device 300 according to an embodiment of the present disclosure is shown to be implemented as a device separate from the electronic device 200.

[0231] In operation 1810, the source device 300 can send video data to the display device 100.

[0232] For example, a user can automatically configure multiple displays by running an application installed on electronic device 200.

[0233] Electronic device 200 can control source device 300 to send data corresponding to the first image (e.g., 610), or provide a signal to display device 100 to turn off the screen, so as to automatically identify multi-screen displays.

[0234] The source device 300 can send data corresponding to the first image 610 or a signal for turning off the screen to the display device 100 under the control of the electronic device 200.

[0235] In operation 1820, electronic device 200 can capture display device 100. Operation 1820 can correspond to Figure 4 Operation 401.

[0236] In operation 1830, electronic device 200 can identify a screen area configured by multiple display modules based on the captured image. Operation 1830 can correspond to Figure 4 Operation 402.

[0237] In operation 1840, source device 300 may send display module information to electronic device 200. The display module information may include at least one of the following: quantity information of multiple display modules or size information of each of the multiple display modules.

[0238] In operation 1850, electronic device 200 can obtain at least one of layout information or position information of display device 100 based on the captured image. Operation 1850 can correspond to Figure 4 Operation 403 or Figure 15 Operation 1540.

[0239] In operation 1860, electronic device 200 may send at least one of layout information or position information of display device 100 to display device 100. Operation 1860 may correspond to Figure 4 Operation 404.

[0240] In operation 1870, the source device 300 can generate control commands for the display device 100 based on layout information and position information.

[0241] In operation 1880, source device 300 can control display device 100 by sending control commands to display device 100.

[0242] The display device 100, which receives control commands, can set the position of each of the plurality of display modules based on at least one of layout information or position information. The display device 100 can display each portion of the image corresponding to the position of each of the plurality of display modules in the entire image.

[0243] Figure 19 This is a diagram illustrating the RGB color space 1910 and the HSV color space 1920 according to embodiments of the present disclosure.

[0244] refer to Figure 19According to an embodiment of the present disclosure, the processor 210 can extract value components from pixels in RGB format in an image using Equation 1.

[0245] Equation 1:

[0246]

[0247]

[0248] V=Cmax

[0249] According to Equation 1, processor 210 can calculate the values ​​of R', G', and B' based on the values ​​of R, G, and B given from 0 to 255. Processor 210 can set the maximum value among the values ​​of R', G', and B' as the maximum value Cmax, and can obtain the maximum value Cmax as value data.

[0250] However, the methods for extracting value components from pixels in RGB format are merely examples and are not limited to the examples described above.

[0251] According to one aspect of this disclosure, an electronic device includes: at least one memory storing one or more instructions; and one or more processors configured to execute one or more instructions.

[0252] According to one aspect of this disclosure, one or more instructions, when executed by one or more processors, cause an electronic device to obtain a second captured image of a display device in a state where the screen area is turned off.

[0253] According to one aspect of this disclosure, one or more instructions, when executed by one or more processors, cause an electronic device to obtain a second operational image from which value components have been extracted from a first captured image.

[0254] According to one aspect of this disclosure, one or more instructions, when executed by one or more processors, cause an electronic device to obtain a third operational image from which value components have been extracted from a second captured image.

[0255] According to one aspect of this disclosure, one or more instructions, when executed by one or more processors, cause an electronic device to identify a screen region based on a result image obtained by performing a second operation on a second operation image and a third operation image.

[0256] According to one aspect of this disclosure, an electronic device can obtain a resulting image by performing image processing on an image acquired by capturing a display device, thereby removing light from the peripheral regions of the display device. The electronic device can use the resulting image to identify screen areas of the display device.

[0257] According to one aspect of this disclosure, the first captured image may include value components of a screen region and value components of a peripheral region corresponding to the periphery of the screen region, and the second captured image may include value components of the peripheral region.

[0258] According to one aspect of this disclosure, the display device may include multiple display modules.

[0259] According to one aspect of this disclosure, the first image may include an image formed by outputting the same pattern or the same color through multiple display modules.

[0260] According to one aspect of this disclosure, one or more instructions, when executed by one or more processors, can enable an electronic device to obtain a first operational image with value components of the screen area removed by performing an AND operation on a first captured image and a second captured image.

[0261] According to one aspect of this disclosure, one or more instructions, when executed by one or more processors, enable an electronic device to: obtain a second operational image by extracting value components of a screen region and value components of a peripheral region from a first captured image; obtain a third operational image by extracting value components of a peripheral region from the second captured image; and obtain a result image by performing an XOR operation on the second operational image and the third operational image, wherein the result image does not include the value components of the peripheral region.

[0262] According to one aspect of this disclosure, when executed by one or more processors, an electronic device may: identify a portion of a resulting image that includes a value component larger than a predetermined threshold as a screen region.

[0263] According to one aspect of this disclosure, one or more instructions, when executed by one or more processors, enable an electronic device to: obtain value data from a resulting image, wherein the value data corresponds to one or more cells of the resulting image, wherein each of the one or more cells may include a predefined number of pixels of the resulting image, and wherein the value data may include first value data and second value data; identify value data of a next cell adjacent to the cell based on the identification of the second value data in a cell of the one or more cells; and determine the position of the last cell where the first value data has been identified as one end of a screen area based on the identification of the second value data in the next cell.

[0264] According to one aspect of this disclosure, when executed by one or more processors, one or more instructions can enable an electronic device to obtain screen area size information and aspect ratio information based on the resulting image.

[0265] The display device may include multiple display modules, the first image may include an image formed by the multiple display modules, and one or more instructions, when executed by one or more processors, may enable the electronic device to: obtain display module information related to the multiple display modules from the display device; identify layout information of the display device based on aspect ratio information and display module information; and obtain position information of multiple regions in the first captured image corresponding to the multiple display modules based on the layout information.

[0266] According to one aspect of this disclosure, the display module information may include at least one of the number of multiple display modules or the size information of each of the multiple display modules, and one or more instructions, when executed by one or more processors, may enable the electronic device to: obtain candidate layout information based on the number of multiple display modules; and identify the layout included in the candidate layout information as layout information based on the size information and aspect ratio information.

[0267] According to one aspect of this disclosure, the electronic device may further include a communication module, and one or more instructions, when executed by one or more processors, may cause the electronic device to send at least one of location information or layout information to a display device via the communication module.

[0268] According to one aspect of this disclosure, a method of operating an electronic device includes: obtaining a first captured image of a display device in a state of outputting a first image to a screen area; obtaining a second captured image of the display device in a state where the screen area is turned off; obtaining a second operating image by extracting value components from the first captured image; obtaining a third operating image from which value components have been extracted from the second captured image; and identifying a screen area based on a result image obtained by performing a second operation on the second operating image and the third operating image.

[0269] The first captured image may include value components of the screen area and value components of the peripheral area corresponding to the periphery of the screen area, and the second captured image may include value components of the peripheral area.

[0270] The display device may include multiple display modules, and the first image may include an image formed by outputting the same pattern or the same color through multiple display modules.

[0271] The method may further include: obtaining a first manipulated image by performing an AND operation on the first captured image and the second captured image to obtain a first manipulated image from which the value components of the screen region have been removed.

[0272] Obtaining the second operation image may further include extracting the value components of the screen region and the value components of the peripheral region from the first capture image. Obtaining the third operation image may further include extracting the value components of the peripheral region from the second capture image. Identifying the screen region based on the result image may further include obtaining the result image by performing an XOR operation on the second operation image and the third operation image, and the result image may not include the value components of the peripheral region.

[0273] The method for identifying screen regions based on the result image may also include: identifying the portion of the result image that includes a value component larger than a predetermined threshold as a screen region.

[0274] The method may further include: obtaining value data from a result image, wherein the value data corresponds to one or more units of the result image, wherein each of the one or more units may include a predefined number of pixels of the result image, and wherein the value data may include first value data and second value data; identifying value data of a next unit adjacent to the unit based on the identification of second value data in a unit of the one or more units; and determining the position of the last unit where the first value data has been identified as one end of a screen area based on the identification of second value data in the next unit.

[0275] The display device may include multiple display modules, and the method may include: obtaining screen area size information and aspect ratio information based on the result image; identifying layout information of the display device based on the aspect ratio information and display module information obtained from the display device; and obtaining position information of multiple regions in the first captured image corresponding to each of the multiple display modules based on the layout information.

[0276] Machine-readable storage media may be provided in the form of non-transitory storage media. In this respect, the term "non-transitory storage media" simply means that the storage media does not include signals (e.g., electromagnetic waves) and is a tangible device, and the term does not distinguish between cases where data is stored semi-permanently in the storage medium and cases where data is temporarily stored in the storage medium. For example, "non-transitory storage media" may include buffers for temporarily storing data.

[0277] According to embodiments of this disclosure, a method according to embodiments of this disclosure may be included in a computer program product when providing such a method. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., a compact disc (CD)-ROM), or distributed online via an app store (e.g., downloaded or uploaded), or distributed directly between two user devices (e.g., smartphones). For online distribution, at least a portion of the computer program product (e.g., a downloadable application) may be stored at least temporarily in a machine-readable storage medium (e.g., the memory of a manufacturer's server, an app store's server, or a relay server) or temporarily generated on such a readable storage medium.

Claims

1. An electronic device (200), comprising: At least one memory (220) stores one or more instructions; as well as One or more processors (210) are configured to execute the one or more instructions, wherein, when executed by the one or more processors (210), the one or more instructions cause the electronic device (200) to perform the following operations: A first captured image (600) of a display device (100) in a state of outputting a first image (610) to a screen area (50) is obtained. A second captured image (700) of the display device (100) is obtained when the screen area (50) is turned off. Obtain a second operational image (900) from which the value components have been extracted from the first captured image (600); Obtain a third operational image (1000) from which the value components have been extracted from the second captured image (700); and The screen region (50) is identified based on the resulting image (1100) obtained by performing a second operation on the second operation image (900) and the third operation image (1000).

2. The electronic device (200) according to claim 1. in, The first captured image (600) includes value components of the screen region (50) and value components of peripheral regions (60, 70) corresponding to the periphery of the screen region (50), and The second captured image (700) includes the value components of the peripheral regions (60, 70).

3. The electronic device (200) according to claim 1 or 2. in, The display device (100) includes multiple display modules (10), and The first image (610) includes an image formed by outputting the same pattern or the same color through the plurality of display modules (10).

4. The electronic device (200) according to any one of claims 1 to 3, wherein, When executed by the one or more processors (210), the one or more instructions cause the electronic device (200) to perform the following operations: A first operational image (800) with the value components of the screen region (50) removed is obtained by performing an AND operation on the first captured image (600) and the second captured image (700).

5. The electronic device (200) according to any one of claims 1 to 4, wherein, When executed by the one or more processors (210), the one or more instructions cause the electronic device (200) to perform the following operations: The second operation image (900) is obtained by extracting the value components of the screen region (50) and the value components of the peripheral regions (60, 70) from the first captured image (600); The third operational image (1000) is obtained by extracting the value components of the peripheral regions (60, 70) from the second captured image (700); and The result image (1100) is obtained by performing an XOR operation on the second operation image (900) and the third operation image (1000), wherein the result image (1100) does not include the value components of the outer regions (60, 70).

6. The electronic device (200) according to any one of claims 1 to 5, wherein, When executed by the one or more processors (210), the one or more instructions cause the electronic device (200) to perform the following operations: The portion of the resulting image (1100) that includes a value component larger than a predetermined threshold is identified as the screen region (50).

7. The electronic device (200) according to any one of claims 1 to 6, wherein, When executed by the one or more processors (210), the one or more instructions cause the electronic device (200) to perform the following operations: Value data is obtained from the result image (1100), wherein the value data corresponds to each of one or more units of the result image (1100), wherein each of the one or more units includes a predetermined number of pixels of the result image (1100), and wherein the value data includes first value data and second value data; Based on the identification of the second value data in a cell among the one or more cells, the value data of the next cell adjacent to that cell is identified; and Based on the identification of the second value data in the next unit, the position of the last unit where the first value data has been identified is determined as one end of the screen area (50).

8. The electronic device (200) according to any one of claims 1 to 7, wherein, When executed by the one or more processors (210), the one or more instructions cause the electronic device (200) to perform the following operations: Based on the resulting image (1100), the size information and aspect ratio information of the screen area (50) are obtained.

9. The electronic device (200) according to claim 8. in, The first image (610) comprises an image formed by a plurality of display modules (10) of the display device (100), and When executed by the one or more processors (210), the one or more instructions cause the electronic device (200) to perform the following operations: Obtain display module information related to the plurality of display modules (10) from the display device (100); The layout information of the display device (100) is identified based on the aspect ratio information of the screen area (50) and the display module information; and Based on the layout information, the position information of multiple regions in the first captured image (600) corresponding to the multiple display modules (10) is obtained.

10. The electronic device (200) according to claim 9. in, The display module information includes at least one of the following: the number of the plurality of display modules (10) or the size information of each of the plurality of display modules (10). When executed by the one or more processors (210), the one or more instructions cause the electronic device (200) to perform the following operations: Based on the number of the plurality of display modules (10), candidate layout information is obtained; and Based on the size information and the aspect ratio information, the layouts included in the candidate layout information are identified as the layout information.

11. The electronic device (200) according to claim 9 or 10, further comprising: Communication module (230). When executed by the one or more processors (210), the one or more instructions cause the electronic device (200) to perform the following operations: The communication module (230) sends at least one of the location information or the layout information to the display device (100).

12. A method of operating an electronic device (200), the method comprising: Obtain (1310) the first captured image (600) of the display device (100) in the state of outputting the first image (610) to the screen area (50); A second captured image (700) of the display device (100) is obtained (1320) while the screen area (50) is turned off. The second operational image (900) is obtained by extracting the value components from the first captured image (600); The third operational image (1000) from which the value components have been extracted (1340) from the second captured image (700) has been obtained; and Based on the resulting image (1100) obtained by performing a second operation on the second operation image (900) and the third operation image (1000), the screen region (50) is identified (1350).

13. The method according to claim 12, in, Obtaining the second operation image (900) further includes extracting the value components of the screen region (50) and the value components of the peripheral regions (60, 70) from the first captured image (600). The process of obtaining the third operational image (1000) further includes extracting the value components of the peripheral regions (60, 70) from the second captured image (700), and The identification of the screen region (50) based on the result image (1100) further includes obtaining the result image (1100) by performing an XOR operation on the second operation image (900) and the third operation image (1000), wherein the result image (1100) does not include the value components of the peripheral regions (60, 70).

14. The method according to claim 12 or 13, wherein, Identifying the screen region (50) based on the result image (1100) further includes identifying the portion of the result image (1100) that includes a value component larger than a predetermined threshold as the screen region (50).

15. A non-transitory computer-readable medium storing instructions that, when executed by at least one processor, cause the at least one processor to perform a method of operating an electronic device, the method comprising: Obtain (1310) the first captured image (600) of the display device (100) in the state of outputting the first image (610) to the screen area (50); A second captured image (700) of the display device (100) is obtained (1320) while the screen area (50) is turned off. The second operational image (900) is obtained by extracting the value components from the first captured image (600); The third operational image (1000) from which the value components have been extracted (1340) from the second captured image (700) has been obtained; and Based on the resulting image (1100) obtained by performing a second operation on the second operation image (900) and the third operation image (1000), the screen region (50) is identified (1350).

Images