Electronic device and operation method thereof

Abstract

Provided are an electronic device and an operation method thereof. The electronic device includes: at least one memory storing one or more instructions; and at least one processor. When the one or more instructions are executed by the at least one processor, the electronic device obtains a first captured image of a display device outputting a first image to a screen area, obtains a second captured image of the display device when the screen is off, obtains a second operation image in which a brightness component has been extracted from the first captured image, obtains a third operation image in which a brightness component has been extracted from the second captured image, and identifies the screen area on the basis of a result image obtained by performing a second operation on the second operation image and the third operation image.

Description

Electronic device and method of operation thereof

[0001] One embodiment of the present disclosure relates to an electronic device for identifying a screen area of ​​a display device, a method of operating the electronic device, and a recording medium.

[0002] As display devices become larger and higher-resolution, the number of display modules that make up a display device is increasing. For example, display modules can be interconnected to form a single screen on a display device.

[0003] When a display device is installed, it is difficult for the installer to manually input the location, layout (or arrangement relationship), and connection relationship of each display module, and there is a possibility of installation errors occurring.

[0004] When installing a display device, research is being conducted on a method to automatically identify the location of display modules and the arrangement and connection relationships between display modules, thereby completing setup without separate setup work.

[0005] An electronic device according to one embodiment of the present disclosure includes at least one memory storing one or more instructions, and one or more processors executing the one or more instructions.

[0006] By executing the one or more instructions by the one or more processors according to one embodiment of the present disclosure, the electronic device obtains a first captured image by capturing a display device in a state of outputting a first image to a screen area.

[0007] By executing the one or more instructions by the one or more processors according to one embodiment of the present disclosure, the electronic device obtains a second captured image by capturing a display device with the screen turned off.

[0008] By executing the one or more instructions by the one or more processors according to one embodiment of the present disclosure, the electronic device obtains a second operation image in which a brightness component is extracted from a first captured image.

[0009] By executing the one or more instructions by the one or more processors according to one embodiment of the present disclosure, the electronic device obtains a third operation image in which a brightness component is extracted from a second captured image.

[0010] By executing the one or more instructions by the one or more processors according to one embodiment of the present disclosure, the electronic device identifies a screen area through a result image obtained by performing a second operation image and a third operation image by performing a second operation.

[0011] An operating method of an electronic device according to one embodiment of the present disclosure includes a step of obtaining a first captured image by capturing a display device in a state of outputting a first image to a screen area, a step of obtaining a second captured image by capturing a display device in a state of having the screen turned off, a step of obtaining a second computed image in which a brightness component is extracted from the first captured image, a step of obtaining a third computed image in which a brightness component is extracted from the second captured image, and a step of identifying a screen area through a result image obtained by performing a second operation on the second computed image and the third computed image.

[0012] According to one embodiment of the present disclosure, a computer-readable recording medium having recorded thereon a program for performing at least one of the operating methods of an electronic device on a computer may be provided.

[0013] Features of one embodiment of the present disclosure will become more apparent from the accompanying drawings and the following description.

[0014] FIG. 1 is a drawing for explaining a display device and an electronic device according to one embodiment of the present disclosure.

[0015] FIG. 2 is a drawing for explaining a display device before and after setting according to one embodiment of the present disclosure.

[0016] FIG. 3 is a drawing for explaining a screen area of ​​a display device identified by an electronic device according to one embodiment of the present disclosure.

[0017] FIG. 4 is a flowchart illustrating a method for an electronic device to set a display device according to one embodiment of the present disclosure.

[0018] FIG. 5 is a block diagram illustrating the configuration of an electronic device and a display device according to one embodiment of the present disclosure.

[0019] FIG. 6 is a drawing for explaining an operation of acquiring a first captured image according to one embodiment of the present disclosure.

[0020] FIG. 7 is a drawing for explaining an operation of acquiring a second captured image according to one embodiment of the present disclosure.

[0021] FIG. 8 is a drawing for explaining a first operation image according to one embodiment of the present disclosure.

[0022] FIG. 9 is a drawing for explaining a second operation image according to one embodiment of the present disclosure.

[0023] FIG. 10 is a drawing for explaining a third operation image according to one embodiment of the present disclosure.

[0024] FIG. 11 is a drawing for explaining a result image according to one embodiment of the present disclosure.

[0025] FIG. 12 is a diagram for explaining an operation of processing a peripheral area in a result image according to one embodiment of the present disclosure.

[0026] FIG. 13A is a flowchart illustrating the operation of an electronic device according to one embodiment of the present disclosure.

[0027] FIG. 13b is a diagram for explaining an operation of an electronic device obtaining a result image according to one embodiment of the present disclosure.

[0028] FIG. 14A is a flowchart illustrating the operation of an electronic device according to one embodiment of the present disclosure.

[0029] FIG. 14b is a diagram for explaining an operation of an electronic device obtaining a result image according to one embodiment of the present disclosure.

[0030] FIG. 15 is a flowchart illustrating a method for an electronic device to automatically recognize a display device according to one embodiment of the present disclosure.

[0031] FIG. 16 and FIG. 17 are drawings for explaining position information of a display module according to one embodiment of the present disclosure.

[0032] FIG. 18 is a flowchart illustrating the operation of an electronic device, a source device, and a display device according to one embodiment of the present disclosure.

[0033] FIG. 19 is a diagram showing an RGB color space and an HSV color space according to one embodiment of the present disclosure.

[0034] In this disclosure, the expression “at least one of a, b or c” may refer to “a”, “b”, “c”, “a and b”, “a and c”, “b and c”, “all of a, b and c”, or variations thereof.

[0035] Below, embodiments of the present disclosure are described in detail with reference to the attached drawings so that those skilled in the art can easily implement the present disclosure. However, the present disclosure may be implemented in various different forms and is not limited to the embodiments described herein.

[0036] The terms used in this disclosure are described as currently common terms, taking into account the functions mentioned herein. However, these terms may mean various other terms depending on the intentions of those skilled in the art, precedents, the emergence of new technologies, etc. Therefore, the terms used in this disclosure should not be interpreted solely based on their names, but rather based on the meanings of the terms and the overall content of this disclosure.

[0037] Additionally, the terminology used in this disclosure is for the purpose of describing specific embodiments only and is not intended to limit the present disclosure.

[0038] Throughout the specification, when a part is said to be "connected" to another part, this includes not only the cases where it is "directly connected" but also the cases where it is "electrically connected" with another element in between.

[0039] As used herein, and particularly in the claims, the terms "above" and "above" and similar referents may refer to both the singular and the plural. Furthermore, unless the order of steps in a method according to the present disclosure is explicitly specified, the steps described may be performed in any appropriate order. The present disclosure is not limited by the order in which the steps are described.

[0040] The appearances of phrases such as “in some embodiments” or “in an embodiment” in various places throughout this specification are not necessarily all referring to the same embodiment.

[0041] Some embodiments of the present disclosure may be represented by functional block configurations and various processing steps. Some or all of these functional blocks may be implemented by various hardware and / or software configurations that perform specific functions. For example, the functional blocks of the present disclosure may be implemented by one or more microprocessors or by circuit configurations for a given function. Furthermore, for example, the functional blocks of the present disclosure may be implemented in various programming or scripting languages. The functional blocks may be implemented by algorithms that execute on one or more processors. Furthermore, the present disclosure may employ conventional techniques for electronic configuration, signal processing, and / or data processing. Terms such as “mechanism,” “element,” “means,” and “configuration” may be used broadly and are not limited to mechanical and physical configurations.

[0042] Additionally, the connecting lines or connecting members between components depicted in the drawings are merely exemplary representations of functional connections and / or physical or circuit connections. In an actual device, connections between components may be represented by various functional connections, physical connections, or circuit connections that may be replaced or added.

[0043] Additionally, terms such as “part”, “module”, etc. described in the specification mean a unit that processes at least one function or operation, which may be implemented as hardware or software, or a combination of hardware and software.

[0044] Additionally, the term "user" in this specification refers to a person who uses the display device, and may include a consumer, evaluator, viewer, administrator, or installer. The term "manufacturer" or "provider" in this specification may refer to a manufacturer that manufactures electronic devices and / or components included in electronic devices.

[0045] In the present disclosure, 'image' may include a still image, a graphic, a picture, a frame, a moving image composed of a plurality of consecutive still images, or a video.

[0046] In the present disclosure, a processor may include various processing circuits and / or multiple processors. For example, the term “processor” as used herein, including in the claims, may include various processing circuits, including at least one processor. At least one processor, one or more processors, may be configured to perform various functions described herein, individually and / or collectively, in a distributed fashion. As used herein, “processor,” “at least one processor,” and “one or more processors” may be configured to perform various functions. However, these terms encompass, without limitation, situations where one processor performs some of the functions and other processor(s) perform other parts of the functions, and situations where a single processor may perform all of the functions. Furthermore, the at least one processor may include a combination of processors that perform various functions of the disclosed functions in a distributed manner. At least one processor may execute program instructions to achieve or perform various functions.

[0047] FIG. 1 is a drawing for explaining a display device and an electronic device according to one embodiment of the present disclosure.

[0048] Referring to FIG. 1, a display device (100) can display video data. The display device (100) can be implemented as a TV, but is not limited thereto, and can be applied to any device having a display function, such as a video wall, a large format display (LFD), a digital signage, a digital information display (DID), a projector display, etc. In addition, the display device (100) can be implemented as various types of displays, such as a liquid crystal display (LCD), an organic light-emitting diode (OLED), a liquid crystal on silicon (LCoS), a digital light processing (DLP), a quantum dot (QD) display panel, a quantum dot light-emitting diodes (QLED), etc.

[0049] The display device (100) may include a plurality of display modules (10) including display modules (10-1) to (10-n). A plurality of display modules (10) may be combined to implement a single display device (i.e., display device (100)).

[0050] According to one embodiment, each of the plurality of display modules (10) provided in the display device (100) may be connected to each other. Each of the plurality of display modules (10) may output an image corresponding to video data received from a source device. Here, the source device may include various devices that provide content to the display device (100). For example, the source device may be an electronic device (200), but is not limited thereto, and may also be implemented as a separate device from the electronic device (200).

[0051] The display device (100) includes a plurality of display modules, which can be combined to form a single display screen. To this end, each of the plurality of display modules must receive video data from a source device and determine which portion of the video data should be displayed.

[0052] For example, at least one display module that receives a control signal, video data, etc. from a source device transmits the received control signal, video data, etc. to another display module connected in series with itself, and thus the control signal, video data, etc. can be sequentially transmitted to all of the plurality of display modules (10).

[0053] Alternatively, for example, a processor provided in a display device (100) may transmit a control signal, video data, etc. corresponding to a position to each of the plurality of display modules (10) based on the position information of each of the plurality of display modules (10). Accordingly, the display device (100) may output an image corresponding to the video data received from the source device.

[0054] In this way, in a multi-screen display in which a plurality of display modules (10) constitute one overall display screen, each of the plurality of display modules (10) can display a corresponding partial image divided from one image. In order to display a corresponding partial image on each of the plurality of display modules (10), it is necessary to obtain the position and arrangement relationship of each of the plurality of display modules constituting the multi-screen display. The position and arrangement relationship of each of the plurality of display modules (10) constituting the multi-screen display can be obtained by manual setting by the user or automatic recognition, etc.

[0055] For example, the position and arrangement relationship of each of the plurality of display modules (10) constituting a multi-screen display can be obtained through manual setting by the user. However, as the resolution, size, etc. of the display device (100) increase, the number of the plurality of display modules (10) constituting the display device (100) may increase. Therefore, it may become difficult for the user (or the installer of the display device (100) to manually set the position, arrangement relationship, and communication connection relationship of each of the plurality of display modules (10) within the display device (100).

[0056] For example, the position and arrangement relationship of each of the plurality of display modules (10) constituting the multi-screen display can be acquired by an automatic recognition method. For example, the electronic device (200) can automatically identify the position, arrangement relationship, and communication connection relationship of each of the plurality of display modules (10) constituting the display device (100) by analyzing an image captured of the display device (100). The electronic device (200) can automatically acquire information about the position of each of the plurality of display modules (10) of the display device (100) without manual input from the user and provide the information to the display device (100). Accordingly, the installation accuracy of the display device (100) and the convenience of the user can be improved.

[0057] In this way, the electronic device (200) obtains information on the positions of each of the plurality of display modules (10) based on the results of analyzing the image captured by the display device (100) (hereinafter, “captured image”), so that the positions of the display modules can be obtained more accurately, and since this automatic recognition process does not require additional intervention by the user, the convenience of the user can be increased.

[0058] Meanwhile, in order for the electronic device (200) to accurately obtain the position and arrangement relationship of the display modules in the automatic recognition method described above, it is necessary to accurately detect the screen area constituted by the plurality of display modules (10). For example, the electronic device (200) obtains information on the position and arrangement relationship of the plurality of display modules (10) by using the ratio information of the identified screen area (e.g., the ratio of the width (W, Width) and the height (H, Height)), and therefore it is necessary to accurately detect the screen area.

[0059] The electronic device (200) can identify the area where the display device (100) outputs light when displaying an image within a captured image as a screen area. For example, the electronic device (200) can identify the screen area based on a brightness component (or luminance component) present in the captured image. However, when there is ambient lighting or reflected light around the display device (100), the electronic device (200) may have difficulty distinguishing between the light output by the display device (100) and the light output from the ambient lighting, etc., and thus may have difficulty accurately identifying the screen area. Accordingly, the user must perform additional tasks, such as blocking the ambient lighting around the display device (100) or adjusting the camera angle of the electronic device (200), in order to accurately identify the screen area of ​​the display device (100) through the electronic device (200), which is inconvenient.

[0060] In one embodiment of the present disclosure, a method for an electronic device (200) to accurately detect a screen area of ​​a display device (100) without the need for additional actions by a user is described.

[0061] FIG. 2 is a drawing for explaining a display device before and after setting according to one embodiment of the present disclosure.

[0062] For convenience of explanation, FIG. 2 illustrates a case where a total of 18 display modules (e.g., 10-1, … 10-18) constitute one display device (100).

[0063] Referring to 201 of FIG. 2, since the position information indicating where each of the plurality of display modules (10) is located within the display device (100) and the layout information of the plurality of display modules (10) are set beforehand, the image output by the display device (100) is different from the image transmitted by the source device to the display device (100).

[0064] Layout information refers to arrangement relationship information of multiple display modules (10) or N x M information (N: column, M: row). For example, layout information indicates information indicating that a total of 18 display modules are arranged in a 6 x 3 layout.

[0065] For example, before the position information of each of the plurality of display modules (10) and the layout information of the plurality of display modules (10) are set, there is a problem in that each of the plurality of display modules (10) does not display a partial image corresponding to the position of each of the plurality of display modules (10), but outputs the same partial image of the entire image, as shown in 201 of FIG. 2.

[0066] For example, since each of the plurality of display modules (10) is initially set to output the (0, 0) coordinates of the image, it is necessary to set the position in which each of the plurality of display modules (10) is installed when installing the display device (100).

[0067] According to one embodiment, even if the user does not manually input location information of each of the plurality of display modules (10) and / or layout information of the plurality of display modules (10) into the display device (100), the electronic device (200) can automatically obtain location information and / or layout information and transmit the same to the display device (100).

[0068] For example, the electronic device (200) can identify a screen area of ​​a display device (100) composed of a plurality of display modules (10) based on a captured image (400) received through a camera. The electronic device (200) can identify that the plurality of display modules (10) are arranged in a 6 x 3 layout based on the identified screen area. The electronic device (200) can assign coordinates (0, 0) to the first display module (10-1) located at the upper left of the identified screen area, and can assign coordinates (2, 5) to the eighteenth display module (10-18) located at the lower right of the screen area (see FIGS. 16 and 17). The electronic device (200) can transmit location information of each of the plurality of display modules (10) and layout information of the plurality of display modules (10) to the display device (100).

[0069] Referring to 202 of FIG. 2, once the position information of each of the plurality of display modules (10) and the layout information of the plurality of display modules (10) are set in the display device (100), the display device (100) can output a partial image corresponding to the position of each of the plurality of display modules (10) based on the position information of each of the plurality of display modules (10). After the setting, the display device (100) can output an image (i.e., an entire image) received from the source device. The image output by the display device (100) can be the same as the image transmitted by the source device to the display device (100).

[0070] FIG. 3 is a drawing for explaining a screen area of ​​a display device identified by an electronic device according to one embodiment of the present disclosure.

[0071] Referring to FIG. 3, an electronic device (200) according to one embodiment may use information about a screen area (50) of a display device (100) to identify position information of each of a plurality of display modules and / or layout information of the plurality of display modules. The screen area (50) is an area formed by a plurality of display modules and may correspond to an area in which the plurality of display modules output images. The screen area (50) may be a display area of ​​the display device (100).

[0072] The electronic device (200) can identify a screen area (50) based on a captured image (400) received through a camera. The electronic device (200) can identify the location of each display module within the screen area (50), assign coordinates corresponding to the location of each display module, and transmit the assigned coordinates to the display device (100).

[0073] The electronic device (200) can identify the area where the display device (100) within the captured image (400) outputs light as an image as a screen area (50). The electronic device (200) can identify the screen area (50) based on a brightness component (or luminance component) present in the captured image (400).

[0074] However, when a user takes a picture of a display device (100) using an electronic device (200), unless additional actions are performed, such as blocking the surrounding light of the display device (100) or adjusting the camera angle of the electronic device (200), there may be light or reflected light in the surrounding area (60, 70) of the display device (100). For example, the captured image (400) may include ambient light output from the surrounding area of ​​the display device (100) in addition to the light output from the screen area (50), and in this case, it may be difficult for the electronic device (200) to accurately distinguish between the screen area (50) and the surrounding area (60, 70).

[0075] The surrounding area (60, 70) may correspond to an area located around the screen area (50). The surrounding area (60, 70) may include ambient light such as reflected light existing in the surrounding environment of the display device (100) (e.g., a wall, floor, ceiling, lighting, etc.), external light sources, and internal lighting.

[0076] For example, if the electronic device (200) identifies the screen area based on the brightness component within the captured image (400) without removing the influence of the ambient light present in the surrounding area (60, 70), the first identification area (410) can be identified as the screen area. Since the first identification area (410) is different from the actual screen area (50), if the position information of each of the plurality of display modules and / or the layout information of the plurality of display modules is set based on the first identification area (410), an installation error of the display device (100) may occur.

[0077] For example, the electronic device (200) may perform image processing on the captured image (400) to remove the influence of ambient light existing in the surrounding area (60, 70) within the captured image (400). The electronic device (200) may identify the area where light exists by using the captured image (400) that has been image-processed to remove the ambient light, thereby identifying the second identification area (420) as the screen area. Since the second identification area (420) is the same as the actual screen area (50), the identification accuracy of the screen area (50) is improved, and installation errors of the display device (100) may be minimized.

[0078] FIG. 4 is a flowchart illustrating a method for an electronic device to set a display device according to one embodiment of the present disclosure.

[0079] Referring to FIG. 4, in operation 401, an electronic device (200) according to one embodiment can photograph a display device (100). The electronic device (200) can photograph a plurality of display modules constituting the display device (100) through a camera. The electronic device (200) can obtain a photographed image of the display device (100).

[0080] For example, the electronic device (200) can obtain a first captured image of the display device (100) with the screen turned on. In the first captured image, the brightness data of the screen area and the brightness data of the surrounding area may be large. The electronic device (200) can obtain a second captured image of the display device (100) with the screen turned off. In the second captured image, the brightness data of the screen area may be small and the brightness data of the surrounding area may be large.

[0081] In operation 402, an electronic device (200) according to one embodiment can identify a screen area composed of a plurality of display modules based on a captured image.

[0082] In one embodiment, the captured image may include light from a peripheral area of ​​the display device (100) in addition to light emitted from the screen area of ​​the display device (100).

[0083] An electronic device (200) according to one embodiment can perform image processing on a captured image to remove data corresponding to light in a peripheral area of ​​a display device (100) within the captured image.

[0084] For example, the electronic device (200) can obtain an image (hereinafter, a second operation image (900)) having large brightness data in a screen area and an image (hereinafter, a third operation image (1000)) having small brightness data in a screen area by using a first captured image and a second captured image. The electronic device (200) can obtain an image (hereinafter, a result image (1100)) that includes brightness data in a screen area having large brightness data in only one of the two images by performing an XOR (exclusive OR) operation on the second operation image (900) and the third operation image (1000). Since brightness data in an area other than the screen area, for example, a peripheral area, can be removed from the result image (1100), the electronic device (200) can accurately distinguish the screen area and the peripheral area in the result image (1100). The electronic device (200) can determine an area having large brightness data in the result image (1100) as the screen area. In one embodiment, the area with large luminance data may include an area in the result image (1100) with luminance data greater than a predetermined threshold.

[0085] This will be explained in more detail with reference to FIGS. 6 to 14.

[0086] In operation 403, the electronic device (200) according to one embodiment can obtain layout information and / or location information of the display device (100) based on the captured image.

[0087] An electronic device (200) according to one embodiment can identify a ratio of a screen area based on the identified screen area. For example, the electronic device (200) can identify a ratio of a width and height of a screen area or an aspect ratio of the screen area.

[0088] An electronic device (200) according to one embodiment can obtain N x M layout information of a plurality of display modules by using ratio information of a screen area.

[0089] An electronic device (200) according to one embodiment can use layout information to assign coordinates to the first display module located at the upper left of an identified screen area, starting from the last display module. By assigning coordinates to each of the plurality of display modules, the electronic device (200) can obtain location information for each of the plurality of display modules.

[0090] This is explained in more detail in Fig. 15.

[0091] In operation 404, the electronic device (200) according to one embodiment may transmit layout information and / or position information of the display device (100) to the display device (100).

[0092] In operation 405, the display device (100) according to one embodiment can control a plurality of display modules based on layout information and / or position information. For example, the display device (100) can set the position of each of the plurality of display modules based on the layout information and / or position information. The display device (100) can display each partial image of the entire image corresponding to the position of each of the plurality of display modules. The image output by the display device (100) can be identical to the image transmitted to the display device (100) by the source device.

[0093] FIG. 5 is a block diagram illustrating the configuration of an electronic device and a display device according to one embodiment of the present disclosure.

[0094] Referring to FIG. 5, the electronic device (200) may include at least one of a smartphone, a tablet PC, a mobile phone, a video phone, an e-book reader, a desktop PC, a laptop PC, a netbook computer, a workstation, a server, a PDA, a portable multimedia player (PMP), an MP3 player, a medical device, a camera, a virtual reality (VR) implementation device, or a wearable device.

[0095] The electronic device (200) may include a processor (210), a memory (220), a communication module (230), and a camera (240). However, not all of the components illustrated in FIG. 5 are essential components. The electronic device (200) may be implemented with more components than the components illustrated in FIG. 5, or may be implemented with fewer components.

[0096] The processor (210) can control the overall operations of the electronic device (200). The processor (210) can be implemented as one or more processors. The processor (210) can perform a predetermined operation by executing instructions or commands stored in the memory (220). For example, the processor (210) can control the overall operations of the electronic device (200) to identify the screen area of ​​the display device (100) by executing one or more instructions of a program stored in the memory (220).

[0097] The processor (210) may include at least one of a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an APU (Accelerated Processing Unit), a MIC (Many Integrated Core), a DSP (Digital Signal Processor), and an NPU (Neural Processing Unit). The processor (210) may be implemented in the form of an integrated SoC (System on Chip) including one or more electronic components. Each of the one or more processors may be implemented as separate hardware.

[0098] The memory (220) stores various information, data, commands, programs, etc. required for the operation of the electronic device (200). The memory (220) may include at least one of volatile memory or non-volatile memory, or a combination thereof. The memory (220) may include at least one type of storage medium among a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory, etc.), a RAM (Random Access Memory), a SRAM (Static Random Access Memory), a ROM (Read-Only Memory), an EEPROM (Electrically Erasable Programmable Read-Only Memory), a PROM (Programmable Read-Only Memory), a magnetic memory, a magnetic disk, and an optical disk.

[0099] The communication module (230) can perform communication with the display device (100). For example, the communication module (230) can transmit a control signal to the display device (100) via wireless communication. For example, the communication module (230) can transmit position information of each of the plurality of display modules and / or layout information of the plurality of display modules to the display device (100). Alternatively, for example, the communication module (230) can transmit position information of each of the plurality of display modules and / or layout information of the plurality of display modules to the display device (100) via a source device.

[0100] The communication module (230) may include at least one of a short-range communication module or a long-range communication module, or a combination thereof. The communication module (230) may include at least one antenna for wirelessly communicating with another device. The short-range wireless communication module may include, but is not limited to, a Bluetooth communication module, a BLE (Bluetooth Low Energy) communication module, an NFC (Near Field Communication) module, a WLAN (Wi-Fi) communication module, a Zigbee communication module, an IrDA (Infrared Data Association) communication module, a WFD (Wi-Fi Direct) communication module, an UWB (ultrawideband) communication module, an Ant+ communication module, a microwave (uWave) communication module, etc.

[0101] The camera (240) is positioned at a certain distance from the display device (100) and can capture images of a plurality of display modules constituting the display device (100). For example, the camera (240) can transmit a captured image of the display device (100) to the processor (210).

[0102] In one embodiment, the camera (240) is illustrated as a component provided in the electronic device (200), but is not limited thereto and may be implemented in an external electronic device separate from the electronic device (200).

[0103] According to one embodiment, the processor (210) can identify the screen area of ​​the display device (100) through a captured image of the display device (100) captured by the camera (240). With reference to FIGS. 6 to 12 below, the operation of the processor (210) to identify the screen area of ​​the display device (100) will be described in detail.

[0104] The display device (100) may include a processor (110), a memory (120), a communication module (130), and a display (140). However, not all of the components illustrated in FIG. 5 are essential components. The display device (100) may be implemented with more components than the components illustrated in FIG. 5, or may be implemented with fewer components.

[0105] The processor (110) can control the overall operations of the display device (100). The processor (110) can be implemented as one or more processors. The processor (110) can perform a predetermined operation by executing instructions or commands stored in the memory (120). For example, the processor (110) can control the overall operations of the display device (100) to identify a screen area of ​​the display device (100) by executing one or more instructions of a program stored in the memory (120).

[0106] The processor (110) may include at least one of a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an APU (Accelerated Processing Unit), a MIC (Many Integrated Core), a DSP (Digital Signal Processor), and an NPU (Neural Processing Unit). The processor (110) may be implemented in the form of an integrated SoC (System on Chip) including one or more electronic components. Each of the one or more processors may be implemented as separate hardware.

[0107] The memory (120) stores various information, data, commands, programs, etc. required for the operation of the display device (100). The memory (120) may include at least one of volatile memory and non-volatile memory, or a combination thereof.

[0108] The communication module (130) can perform communication with the electronic device (200). For example, the communication module (130) can transmit a control signal to the electronic device (200) via wireless communication. For example, the communication module (130) can receive location information of each of the plurality of display modules and / or layout information of the plurality of display modules from the electronic device (200).

[0109] The display (140) may include a plurality of display modules. Each of the plurality of display modules according to one embodiment may include a plurality of self-luminous elements. Here, the self-luminous elements may be at least one of a light emitting diode (LED) or a micro LED.

[0110] Additionally, each of the plurality of display modules may be implemented as an LED cabinet including a plurality of LED elements. Here, the LED elements may be implemented as RGB LEDs, and the RGB LEDs may include red LEDs, green LEDs, and blue LEDs. Additionally, the LED elements may include white LEDs in addition to the RGB LEDs.

[0111] The LED element can be implemented as a micro LED. Here, a micro LED is an LED with a size of approximately 5 to 100 micrometers, and is an ultra-small light-emitting element that emits light on its own without a color filter.

[0112] According to one embodiment, the processor (110) can receive location information of each of the plurality of display modules and / or layout information of the plurality of display modules from the electronic device (200) through the communication module (130).

[0113] FIG. 6 is a diagram illustrating an operation for acquiring a first captured image according to one embodiment of the present disclosure. In FIG. 6, the first captured image (600) can be expressed as RGB data.

[0114] Referring to FIG. 6, a processor (210) according to one embodiment can receive a first captured image (600) of a display device (100) captured through a camera (240).

[0115] According to one embodiment, the display device (100) may be in a state of outputting a first image (610). The first image (610) may be an image in which each of a plurality of display modules outputs the same image. For example, each of the plurality of display modules may output an image of the same pattern or color, so that the display device (100) may output a first image (610) of a single pattern or a first image (610) of a single color.

[0116] A display device (100) according to one embodiment can output a first image (610) to a screen area (50). The screen area (50) can correspond to an area where the display device (100) displays the first image (610). The first image (610) can be an image received from a source device.

[0117] According to one embodiment, a processor (210) can receive a first captured image (600) that captures a display device (100) in a state of outputting a first image (610).

[0118] According to one embodiment, a first captured image (600) may include a display device (100) outputting the first image (610) and a surrounding environment (e.g., a wall, a floor, a ceiling, lighting, etc.) in which the display device (100) is placed. For example, the first captured image (600) may include a screen area (50) of the display device (100) and a surrounding area (60, 70) corresponding to the surrounding area of ​​the screen area (50).

[0119] For example, light output from the display device (100) may exist in the screen area (50) of the first captured image (600). The screen area (50) may correspond to the area where the first image (610) is output. For example, if the RGB data of the first image (610) is a single-color image of (255, 0, 0), the RGB data of the screen area (50) of the first captured image (600) may be (255, 0, 0).

[0120] For example, in the peripheral area (60, 70) of the first captured image (600), there may be ambient light such as reflected light, external light source, internal light, etc. existing around the display device (100). For example, in the peripheral area (60), there may be light (e.g., fluorescent light, external light source, etc.) reflected on the wall, ceiling, etc. around the display device (100). For example, in the peripheral area (70), there may be reflected light reflected on the floor around the display device (100). For example, the RGB data of the peripheral area (60, 70) of the first captured image (600) may be (255, 255, 255).

[0121] According to one embodiment, when RGB data present in the first captured image (600) is converted into brightness (V, Value) data in HSV format, the first captured image (600) may include a brightness component of the screen area (50) and a brightness component of the surrounding area (60, 70). The brightness component included in the first captured image (600) may be expressed as shown in FIG. 9, which will be described later.

[0122] Here, brightness can represent the degree of brightness. Brightness can be expressed as a brightness component, brightness data, or brightness value. The presence of a brightness component corresponds to relatively large brightness data, while the absence of a brightness component corresponds to relatively small brightness data.

[0123] The first image (610) output by the display device (100) illustrated in FIG. 6 is merely an example, and the display device (100) may also output a first image (610) in which various colors are combined or various patterns are combined. For example, each of the plurality of display modules may output images in different patterns or colors.

[0124] FIG. 7 is a diagram illustrating an operation for acquiring a second captured image according to one embodiment of the present disclosure. In FIG. 7, the second captured image (700) can be expressed as RGB data.

[0125] Referring to FIG. 7, a processor (210) according to one embodiment can receive a second captured image (700) of a display device (100) captured through a camera (240).

[0126] According to one embodiment, the display device (100) may be in a state where it does not output an image. The display device (100) may be in a state where the screen is turned off.

[0127] According to one embodiment, a processor (210) can receive a second captured image (700) captured by capturing a display device (100) with the screen turned off.

[0128] A second captured image (700) according to one embodiment may include a display device (100) that is not outputting an image and a surrounding environment in which the display device (100) is placed.

[0129] For example, there may be no light output from the display device (100) in the screen area (50) of the second captured image (700). For example, the screen area (50) of the second captured image (700) may have black data with R, G, and B values ​​of (0, 0, 0).

[0130] For example, in the peripheral area (60, 70) of the second captured image (700), there may be ambient light such as reflected light, external light sources, and internal light existing around the display device (100). For example, in the peripheral area (60), there may be internal light (e.g., fluorescent light, external light source, etc.) reflected on the surrounding walls and ceiling of the display device (100). For example, in the peripheral area (70), there may be reflected light reflected on the floor of the display device (100). For example, the RGB data of the peripheral area (60, 70) of the second captured image (700) may be (255, 255, 255).

[0131] According to one embodiment, when RGB data present in the second captured image (700) is converted into brightness data in HSV format, the second captured image (700) does not include brightness data of the screen area (50), but may include brightness data of the surrounding areas (60, 70). The brightness data included in the second captured image (700) may be expressed as shown in FIG. 10, which will be described later.

[0132] However, this is not limited thereto, and there may be cases where light exists in the screen area (50) of the second captured image (700) according to one embodiment. For example, there may be cases where the screen area (50) of the second captured image (700) does not have light generated from the screen of the display device (100), but there is light reflected from the lighting of the surrounding environment. In this case, the screen area (50) of the second captured image (700) may include at least a portion of the data in which the R, G, and B values ​​are not (0, 0, 0), for example, (30, 30, 30).

[0133] In this case, the processor (210) can obtain an image in which the brightness component existing in the screen area (50) is clearly removed through a predetermined operation on the first captured image (600) and the second captured image (700). The predetermined operation may be the first operation (see FIG. 8). The first operation may be performed to increase the accuracy of screen area identification, or may be omitted for convenience of operation.

[0134] According to one embodiment, the processor (210) can generate a third operation image (1000) by extracting brightness data of a second captured image (700) or a first operation image (800).

[0135] FIG. 8 is a diagram illustrating a first operation image according to one embodiment of the present disclosure. In FIG. 8, the first operation image (800) can be expressed as RGB data.

[0136] Referring to FIG. 8, a processor (210) according to one embodiment can obtain a first operation image (800) by performing a first operation on a first captured image (600) and a second captured image (700).

[0137] For example, the processor (210) can perform an AND (logical product) operation on the first captured image (600) having large RGB data of the screen area (50) and the peripheral area (60, 70) and the second captured image (700) having large RGB data of the peripheral area (60, 70). The processor (210) can obtain the first operation image (800) having large RGB data of the peripheral area (60, 70) by performing the AND operation on the first captured image (600) and the second captured image (700). In the first operation image (800), the RGB data of the screen area (50) may be small or absent. For example, in the first operation image (800), the RGB data of the screen area (50) may be (30, 0, 0) or (0, 0, 0), and the RGB data of the peripheral area (60, 70) may be (255, 255, 255). However, the first operation is not limited to the AND operation, and may be replaced with various operations capable of removing the brightness component of the screen area (50).

[0138] The AND operation is an operator whose result is 1 only when all operands have the same value, and whose result is 0 if even one of the operands is different.

[0139] The first operation image (800) illustrated in FIG. 8 exemplifies a case in which the first operation is performed using the second captured image (700) in which reflected light exists in the screen area (50), and therefore, predetermined RGB data may exist in the screen area (50). However, the present invention is not limited thereto, and in the case in which the first operation is performed using the second captured image (700) in which reflected light does not exist in the screen area (50), unlike what is illustrated, RGB data may not exist in the screen area (50) of the first operation image (800).

[0140] According to one embodiment, when RGB data present in the first operation image (800) is converted into brightness data in HSV format, the first operation image (800) does not include brightness data of the screen area (50), but may include brightness data of the surrounding areas (60, 70). This is because the RGB data of the screen area (50) of the first operation image (800) has a relatively small or no brightness component. The brightness data included in the first operation image (800) can be expressed as shown in FIG. 10, which will be described later.

[0141] For example, even if certain RGB data exists in the screen area (50) of the first operation image (800), brightness data may not be extracted as the RGB data is converted into HSV data.

[0142] Accordingly, the processor (210) can perform an AND operation on the first captured image (600) and the second captured image (700), thereby removing the brightness data of the screen area (50) and generating an image in which only the brightness data of the surrounding areas (60, 70) exist.

[0143] According to one embodiment, the processor (210) can obtain an image in which brightness data of the screen area (50) is clearly removed by performing a first operation on the first captured image (600) and the second captured image (700).

[0144] For example, the processor (210) can clearly remove the reflected light component existing in the screen area (50) by performing a first operation on the first captured image (600) and the second captured image (700) in which reflected light exists in the screen area (50). The processor (210) can clearly extract brightness data of the surrounding area (60, 70) using the first operation image (800), thereby generating the third operation image (1000).

[0145] Alternatively, the processor (210) according to one embodiment may omit the first operation process for convenience of operation and generate a third operation image (1000) by extracting brightness data from the second captured image (700).

[0146] FIG. 9 is a diagram for explaining a second operation image according to one embodiment of the present disclosure. In FIG. 9, the second operation image (900) can be expressed as V data in HSV format.

[0147] Referring to FIG. 9, a processor (210) according to one embodiment can obtain a second operation image (900) by extracting a brightness component for a first captured image (600).

[0148] According to one embodiment, the processor (210) can convert an image in RGB format into HSV format and extract a brightness component (V component). For example, the processor (210) can convert a first captured image (600) in RGB format into HSV format. The processor (210) can obtain a second operation image (900) in which a brightness component is extracted for the first captured image (600) converted into HSV format. The processor (210) can express a brightness component existing in the first captured image (600) as brightness data. Each pixel in the second operation image (900) can be expressed as brightness data.

[0149] The RGB format is a three-channel format that expresses image pixel values ​​as red data (R), green data (G), and blue data (B). For example, if each data is expressed in 8 bits, red, green, and blue each have one of 256 levels. Colors can be expressed by mixing the red, green, and blue colors expressed at their respective levels.

[0150] The HSV format, like the RGB format, can display images in three channels. HSV format expresses the pixel values ​​of an image as hue (H), saturation (S), and value (V). Darker images require less brightness data, while brighter images require more brightness data.

[0151] In one embodiment, a method for obtaining a brightness component from an image in RGB format is described in FIG. 19.

[0152] According to one embodiment, a processor (210) can extract a brightness component for a first captured image (600) including a screen area (50) in which a brightness component exists and a surrounding area (60, 70) in which a brightness component exists. The processor (210) can obtain a second operation image (900) in which the brightness components of the screen area (50) and the surrounding area (60, 70) exist.

[0153] Since the second operation image (900) has the hue and saturation components removed, each region of the second operation image (900) can be distinguished according to the size of the brightness data. For example, the second operation image (900) can be distinguished into a region with large brightness data and a region with small brightness data. For example, in the second operation image (900), the screen region (50) with large brightness data and the surrounding regions (60, 70) can be displayed in white, and the remaining regions can all be displayed in black.

[0154] FIG. 10 is a diagram for explaining a third operation image according to one embodiment of the present disclosure. In FIG. 10, the third operation image (1000) can be expressed as V data in HSV format.

[0155] Referring to FIG. 10, a processor (210) according to one embodiment can obtain a third operation image (1000) by extracting a brightness component for a second captured image (700).

[0156] Alternatively, the processor (210) according to one embodiment may obtain a third operation image (1000) by extracting a brightness component for the first operation image (800).

[0157] Hereinafter, in FIG. 10, the brightness data of the third operation image (1000) is exemplified as corresponding to the brightness data of the second captured image (700).

[0158] According to one embodiment, the processor (210) can convert an image in RGB format into HSV format and extract a brightness component (V component). For example, the processor (210) can convert a second captured image (700) in RGB format into HSV format. The processor (210) can obtain a third operation image (1000) in which a brightness component is extracted for the second captured image (700) converted into HSV format. The processor (210) can express a brightness component existing in the second captured image (700) as brightness data. Each pixel in the third operation image (1000) can be expressed as brightness data.

[0159] According to one embodiment, a processor (210) may extract a brightness component for a second captured image (700) that includes a screen area (50) where no brightness component exists and a peripheral area (60, 70) where a brightness component exists. The processor (210) may obtain a third operation image (1000) in which the brightness component of the peripheral area (60, 70) excluding the screen area (50) remains. The brightness component of the screen area (50) may not exist in the third operation image (1000).

[0160] Since the hue and saturation components have been removed from the third operation image (1000), each region of the third operation image (1000) can be distinguished according to the size of the brightness data. For example, in the third operation image (1000), the peripheral regions (60, 70) with large brightness data can be displayed in white, and the remaining regions can be displayed in black. For example, in the third operation image (1000), the screen region (50) with small brightness data can be displayed in black.

[0161] Meanwhile, in one embodiment, when a first operation operation for obtaining a first operation image (800) is added, the brightness data of the third operation image (1000) may correspond to the brightness data of the first operation image (800).

[0162] FIG. 11 is a drawing for explaining a result image according to one embodiment of the present disclosure.

[0163] Referring to FIG. 11, the processor (210) can obtain a result image (1100) by performing a second operation on the second operation image (900) and the third operation image (1000). As the processor (210) obtains the result image (1100), it can identify the screen area (50) of the display device (100).

[0164] For example, the processor (210) can perform an XOR (exclusive OR) operation on a second operation image (900) in which only the brightness component of the screen area (50) and the brightness component of the surrounding area (60, 70) remain, and a third operation image (1000) in which only the brightness component of the surrounding area (60, 70) remains. By performing the XOR operation on the second operation image (900) and the third operation image (1000), the processor (210) can obtain a result image (1100) including the brightness component of the screen area (50).

[0165] The XOR operation is an operator that outputs 1 only when the two operands have different values, and 0 if they are the same.

[0166] The processor (210) can remove the rest, leaving only the data of the area where the brightness data is large in only one of the two operation images by performing an XOR operation on the second operation image (900) and the third operation image (1000). For example, the processor (210) can leave the brightness data of the screen area (50) corresponding to the area where the brightness data is large in only the second operation image (900) among the second operation image (900) and the third operation image (1000). The processor (210) can remove the brightness data of the peripheral areas (60, 70) that exist in common in the second operation image (900) and the third operation image (1000).

[0167] According to one embodiment, the result image (1100) obtained from the second operation may include a brightness component of the screen area (50) and may not include a brightness component of the surrounding area (60, 70).

[0168] The resulting image (1100) may be divided into different regions based on the size of the luminance data. For example, in the resulting image (1100), a screen region (50) with large luminance data may be displayed in white, while the remaining regions may be displayed in black. For example, in the resulting image (1100), surrounding regions (60, 70) with small luminance data may be displayed in black.

[0169] According to one embodiment, the processor (210) may determine an area with large brightness data in the result image (1100) as the screen area (50). The processor (210) may distinguish the screen area (50) and the surrounding areas (60, 70) of the result image (1100) according to the size of the brightness data.

[0170] For example, the processor (210) may determine that the brightness data of the result image (1100) is large if the brightness data is greater than or equal to a threshold value, and may determine that the brightness data is small if the brightness data is less than or equal to the threshold value. However, the present invention is not limited thereto.

[0171] The processor (210) can distinguish between the screen area (50) and the peripheral area (60, 70) of the display device (100) by removing the influence of ambient light present in the peripheral area (60, 70) through a second operation. The processor (210) can accurately identify the screen area (50) included in the result image (1100).

[0172] FIG. 12 is a diagram for explaining an operation of processing a peripheral area in a result image according to one embodiment of the present disclosure.

[0173] Referring to FIG. 12, the resulting image (1100) may contain some brightness data of the surrounding area. The resulting image (1100) is illustrated with brightness data of the surrounding area (70) remaining. Even if brightness data of the surrounding area (70) remains in the resulting image (1100), the processor (210) can distinguish the screen area (50) from the surrounding area (70).

[0174] The resulting image (1100) may have first luminance data corresponding to large luminance data and second luminance data corresponding to small luminance data. For example, the first luminance data may have a value of 1 or white, and the second luminance data may have a value of 0 or black.

[0175] According to one embodiment, the processor (210) can read brightness data in units consisting of a predetermined number of pixels from the result image (1100). For example, the processor (210) can read brightness data of pixels within a unit (1220) in the up, down, left, and right directions from the center of the image in units consisting of a predetermined number of pixels (e.g., 3, 5, etc.).

[0176] According to one embodiment, the processor (210) identifies a unit (1220) that includes a black pixel, and if it identifies that the unit (1220) having the black pixel is repeated to a certain degree, it can determine the location of the last pixel that is not a black pixel as one end of the screen area (50).

[0177] According to one embodiment, the processor (210) can identify the brightness data of the next unit adjacent to the one unit by identifying the second brightness data in one unit. If the second brightness data is also identified in the next unit, the processor (210) can determine the location of the last unit that was able to identify the first brightness data as one end of the screen area (50).

[0178] For example, the processor (210) can read the brightness data of the unit (1220) from the center of the image (1210) downward. The processor (210) can identify that the brightness data of the third unit (1223) has a value of 1. The processor (210) can identify that the brightness data of the first unit (1221) corresponding to any one unit does not have a value of 1. If a unit whose brightness data does not have a value of 1 is repeatedly identified to a certain degree or more, the processor (210) can determine the location of the third unit (1223) corresponding to the last unit whose brightness data has a value of 1 as the bottom (51) of the screen area (50). For example, if the processor (210) identifies that the brightness data of the second unit (1222), which is the next unit adjacent to the first unit (1221), also does not have a value of 1, the processor (210) can identify that a unit whose brightness data does not have a value of 1 is repeatedly identified to a certain degree or more.

[0179] The processor (210) can determine the left end, right end, and top end of the screen area (50) in the same manner as determining the bottom end (51) of the screen area (50).

[0180] The processor (210) can identify the screen area (50) by determining the top, left end, right end, and bottom (51) of the screen area (50) in the result image (1100).

[0181] FIG. 13a is a flowchart illustrating the operation of an electronic device according to one embodiment of the present disclosure. FIG. 13b is a diagram illustrating the operation of an electronic device acquiring a result image according to one embodiment of the present disclosure.

[0182] Referring to FIGS. 13a and 13b, in operation 1310, an electronic device (200) according to one embodiment can obtain a first captured image (600) that captures a display device (100) that outputs a first image (610) to a screen area (50).

[0183] An electronic device (200) according to one embodiment can capture an image of a display device (100) with its screen turned on through a camera (240). The display device (100) according to one embodiment can be in a state of outputting a first image (610). The first image (610) can be an image in which each of a plurality of display modules outputs the same image.

[0184] According to one embodiment, the first captured image (600) may include a brightness component existing in the screen area (50) and a brightness component existing in the surrounding areas (60, 70). When RGB data of the first captured image (600) is converted into brightness data, the brightness data of the screen area (50) and the surrounding areas (60, 70) may be large.

[0185] In operation 1320, an electronic device (200) according to one embodiment can obtain a second captured image (700) of a display device (100) with the screen turned off.

[0186] An electronic device (200) according to one embodiment can capture a picture of a display device (100) with its screen turned off through a camera (240). The display device (100) according to one embodiment can be in a state where it does not output an image.

[0187] The second captured image (700) according to one embodiment may include a brightness component present in the surrounding area (60, 70). When RGB data of the second captured image (700) is converted into brightness data, the brightness data of the screen area (50) may be small or absent, and the brightness data of the surrounding area (60, 70) may be large.

[0188] In operation 1330, an electronic device (200) according to one embodiment can obtain a second operation image (900) in which a brightness component is extracted from a first captured image (600).

[0189] An electronic device (200) according to one embodiment can convert a first captured image (600) in RGB format into an HSV format. The electronic device (200) can obtain a second operation image (900) from which a brightness component is extracted for the first captured image (600) converted into the HSV format. In the second operation image (900), each pixel can be expressed as brightness data.

[0190] An electronic device (200) according to one embodiment can obtain a second operation image (900) in which brightness components of a screen area (50) and a peripheral area (60, 70) exist by extracting a brightness component from a first captured image (600). For example, in the second operation image (900), the entire area except for the screen area (50) and the peripheral area (60, 70) with large brightness data can be displayed as black.

[0191] In operation 1340, an electronic device (200) according to one embodiment can obtain a third operation image (1000) in which a brightness component is extracted from a second captured image (700).

[0192] An electronic device (200) according to one embodiment can convert a second captured image (700) in RGB format into an HSV format. The electronic device (200) can obtain a third operation image (1000) in which a brightness component is extracted from the first operation image (800) converted into the HSV format. In the third operation image (1000), each pixel can be expressed as brightness data.

[0193] An electronic device (200) according to one embodiment can obtain a second operation image (900) in which brightness components of a peripheral area (60, 70) excluding a screen area (50) exist by extracting a brightness component from a second captured image (700). For example, a third operation image (1000) can be displayed as black in all areas except for a peripheral area (60, 70) with large brightness data. In the third operation image (1000), a screen area (50) with small brightness data can be displayed as black.

[0194] In operation 1350, an electronic device (200) according to one embodiment can identify a screen area (50) through a result image (1100) obtained by performing a second operation image (900) and a third operation image (1000) on the second operation image.

[0195] According to one embodiment, the electronic device (200) can remove brightness data of the remaining areas while leaving an area with large brightness data only in one of the two operation images by performing an XOR operation on the second operation image (900) and the third operation image (1000). For example, the electronic device (200) can leave brightness data of a screen area (50) corresponding to an area with large brightness data only in the second operation image (900) among the second operation image (900) and the third operation image (1000). The electronic device (200) can remove brightness data of peripheral areas (60, 70) that exist in common in the second operation image (900) and the third operation image (1000).

[0196] The resulting image (1100) includes the brightness component of the screen area (50) and may not include the brightness component of the surrounding area (60, 70).

[0197] The resulting image (1100) may be divided into different regions based on the size of the luminance data. For example, in the resulting image (1100), all regions except the screen region (50) with large luminance data may be displayed in black. For example, in the resulting image (1100), surrounding regions (60, 70) with small luminance data may be displayed in black.

[0198] An electronic device (200) according to one embodiment can identify a screen area (50) by distinguishing a screen area (50) corresponding to an area with large brightness data in a result image (1100) from an area with small brightness data. The electronic device (200) can determine an area with large brightness data in a result image (1100) as a screen area (50).

[0199] The electronic device (200) can accurately identify a screen area (50) using a captured image that has been image-processed to remove light existing in the surrounding environment of the display device (100). Based on the accurately identified screen area (50), the electronic device (200) can obtain position information of each of the plurality of display modules of the display device (100) and layout information of the plurality of display modules. The electronic device (200) can automatically identify information about the display device (100) and transmit it to the display device (100), thereby completing installation of the display device (100).

[0200] FIG. 14A is a flowchart illustrating the operation of an electronic device according to an embodiment of the present disclosure. FIG. 14B is a diagram illustrating the operation of an electronic device acquiring a result image according to an embodiment of the present disclosure. In FIGS. 14A and 14B, the electronic device (200) according to an embodiment may further include a first operation operation, and any overlapping content with FIG. 13A is omitted.

[0201] Referring to FIGS. 14a and 14b, operation 1410 corresponds to operation 1310 of FIG. 13a. Operation 1420 corresponds to operation 1320 of FIG. 13a.

[0202] Meanwhile, in the screen area (50) of the second captured image (700), there is no light generated from the screen of the display device (100), but there may be light reflected from the surrounding environment. In this case, certain RGB data other than (0, 0, 0) may exist in the screen area (50) of the second captured image (700).

[0203] In operation 1430, an electronic device (200) according to one embodiment can obtain a first operation image (800) by performing a first operation on a first captured image (600) and a second captured image (700).

[0204] An electronic device (200) according to one embodiment can obtain RGB data common to the first captured image (600) and the second captured image (700) by performing an AND operation on the first captured image (600) and the second captured image (700).

[0205] For example, the electronic device (200) can obtain a first operation image (800) in which RGB data of the peripheral area (60, 70) is large and RGB data of the screen area (50) is small or absent by performing an AND operation on the first captured image (600) and the second captured image (700). When the RGB data of the first operation image (800) is converted into brightness data, the brightness data of the peripheral area (60, 70) may be large and the brightness data of the screen area (50) may be small or absent.

[0206] In operation 1440, an electronic device (200) according to one embodiment may obtain a second operation image (900) in which a brightness component is extracted from a first captured image (600). Operation 1440 corresponds to operation 1330 of FIG. 13A.

[0207] In operation 1450, an electronic device (200) according to one embodiment can obtain a third operation image (1000) in which a brightness component is extracted from a first operation image (800).

[0208] An electronic device (200) according to one embodiment can convert a first operation image (800) in RGB format into an HSV format. The electronic device (200) can obtain a third operation image (1000) in which a brightness component is extracted from a second captured image (700) converted into an HSV format. Each pixel in the third operation image (1000) can be expressed as brightness data.

[0209] An electronic device (200) according to one embodiment can obtain a second operation image (900) in which a brightness component of a peripheral area (60, 70) excluding a screen area (50) exists by extracting a brightness component from a first operation image (800). For example, in a third operation image (1000), all areas except for the peripheral area (60, 70) with large brightness data can be displayed as black. In the third operation image (1000), a screen area (50) with small brightness data can be displayed as black.

[0210] In operation 1460, the electronic device (200) according to one embodiment can identify the screen area (50) through the result image (1100) obtained by performing a second operation on the second operation image (900) and the third operation image (1000). Operation 1460 corresponds to operation 1350 of FIG. 13A.

[0211] FIG. 15 is a flowchart illustrating a method for an electronic device to automatically recognize a display device according to an embodiment of the present disclosure. FIG. 16 and FIG. 17 are diagrams illustrating position information of a display module according to an embodiment of the present disclosure.

[0212] Referring to FIG. 15, in operation 1510, the electronic device (200) can obtain a captured image for the display device (100). Operation 1510 may correspond to operation 401 of FIG. 4.

[0213] In operation 1520, the electronic device (200) may identify a screen area of ​​the display device (100) based on the captured image. The captured image may include a first captured image (600) and a second captured image (700). Operation 1520 may correspond to operation 402 of FIG. 4.

[0214] In operation 1530, the electronic device (200) may obtain ratio information of the display device (100) based on size information of the screen area in the captured image. The captured image may include at least one of a first captured image (600), a second captured image (700), a third captured image (1600), and a fourth captured image (1700). The third captured image (1600) may be the same as or different from the first captured image (600).

[0215] Referring to the third captured image (1600) of FIG. 16, the processor (210) can obtain ratio information of the display device (100) based on the identified screen area (50). For example, the processor (210) can obtain ratio information of the display device (100), for example, the ratio of the width (W) and height (H) of the display device (100) or the aspect ratio of the display device (100), by using the size information of the screen area (50).

[0216] For example, the processor (210) can identify that the aspect ratio of the display device (100) is 16:9, i.e., 1.778, based on the identified screen area (50).

[0217] In operation 1540, the electronic device (200) can obtain layout information of a plurality of display modules (10) based on the ratio information of the display device (100) and information of a plurality of display modules (10).

[0218] In one embodiment, the information of the plurality of display modules (10) may include at least one of information on the number of the plurality of display modules (10) or information on the size of each of the plurality of display modules (10).

[0219] For example, referring to FIG. 16, the plurality of display modules (10) may include 18 display modules, and the width (W) x height (H) of each of the plurality of display modules (10) may be 320 x 360.

[0220] The processor (210) can obtain a plurality of candidate layout information based on the number information (e.g., 18) of the plurality of display modules (10). For example, if the processor (210) identifies that 18 display modules constitute the display device (100), the processor (210) can obtain a plurality of candidate layout information by identifying configurable N x M layouts (wherein, N and M are integers). As an example, the processor (210) can obtain a total of six pieces of candidate layout information, e.g., 1 x 18, 2 x 9, 3 x 6, 6 x 3, 9 x 2, and 18 x 1.

[0221] The processor (210) can obtain a candidate ratio by multiplying each of the acquired plurality of layout information by the size of the display module (e.g., 320 x 360). For example, when the size of the display module is 320 x 360, the processor (210) can obtain a plurality of candidate ratios (N x Width: M x Height) corresponding to a plurality of candidate layouts.

[0222] The processor (210) can identify a candidate layout having a candidate ratio that is identical or similar to 1.778, which is the ratio information of the display device (100), among the six candidate ratios corresponding to the six candidate layouts, as layout information of the display device (100). For example, the candidate ratio of a 6 x 3 layout among the plurality of candidate layouts may be (6 x 320px):(3 x 360px) = 1920:1080 = 1.778. The processor (210) can identify that the layout information of the display device (100) is 6 x 3.

[0223] In operation 1550, the processor (210) may obtain location information for an area corresponding to each of the plurality of display modules (10) based on the layout information. For example, the processor (210) may assign (0, 0) to an area corresponding to the first display module (10-1) and (2, 5) to an area corresponding to the eighteenth display module (10-18). Specific numbers are merely examples for convenience of explanation and are not limited thereto.

[0224] Referring to FIG. 17, the processor (210) can acquire the position of the display module by taking a picture of the display device (100) including one display module that outputs a different image (e.g., color or pattern, etc.) (see the fourth captured image (1700)). The display device (100) may be in a state where one display module (e.g., the fifteenth display module (10-15)) among the plurality of display modules (10) outputs a different image from the remaining display modules. The processor (210) can acquire the fourth captured image (1700) that takes a picture of the display device (100) in a state where one display module outputs a different image from the remaining display modules.

[0225] The processor (210) can identify an area where one of the display modules (e.g., the 15th display module (10-15)) that outputs a different image from the fourth captured image (1700) is located.

[0226] The processor (210) can compare the third captured image (1600) and the fourth captured image (1700) to identify an area (i.e., an area where a different image is displayed) identified in the fourth captured image (1700) within the third captured image (1600).

[0227] If the processor (210) identifies that an area (i.e., an area where a different image is displayed) identified in the fourth captured image (1700) is similar to (or identical to) an area corresponding to the 15th display module (10-15) in the third captured image (1600), the processor (210) may acquire location information (e.g., (2, 2)) assigned to the 15th display module (10-15) as location information of the 15th display module (10-15). The processor (210) may transmit the acquired location information to the display device (100).

[0228] The processor (210) can obtain location information of each display module by repeatedly performing shooting of the display device (100) in a state of repeatedly turning different images on and off on the screen for each display module.

[0229] The processor (210) can transmit location information of each of the plurality of display modules (10) and / or layout information of the plurality of display modules (10) to the display device (100) through the communication module (230). The processor (210) can automatically identify information about the display device (100) and transmit it to the display device (100), thereby completing installation of the display device (100). FIG. 18 is a flowchart for explaining the operation of an electronic device, a source device, and a display device according to one embodiment of the present disclosure. FIG. 18 exemplifies a case where the source device (300) according to one embodiment is implemented as a separate device from the electronic device (200).

[0230] In operation 1810, the source device (300) can transmit video data to the display device (100).

[0231] For example, a user can perform automatic settings for a multi-screen display by running a predetermined application installed on the electronic device (200).

[0232] The electronic device (200) can control the source device (300) to transmit data corresponding to the first image (e.g., 610) to the display device (100) for automatic recognition of the multi-screen display, or to provide a signal to turn off the screen.

[0233] The source device (300) can transmit data corresponding to the first image (610) to the display device (100) under the control of the electronic device (200) or transmit a signal to turn off the screen.

[0234] In operation 1820, the electronic device (200) can photograph the display device (100). Operation 1820 may correspond to operation 401 of FIG. 4.

[0235] In operation 1830, the electronic device (200) can identify a screen area composed of multiple display modules based on the captured image. Operation 1830 may correspond to operation 402 of FIG. 4.

[0236] In operation 1840, the source device (300) may transmit display module information to the electronic device (200). The display module information may include at least one of information on the number of a plurality of display modules or information on the size of each of the plurality of display modules.

[0237] In operation 1850, the electronic device (200) may obtain layout information and / or location information of the display device (100) based on the captured image. Operation 1850 may correspond to operation 403 of FIG. 4 or operation 1540 of FIG. 15.

[0238] In operation 1860, the electronic device (200) may transmit layout information and / or position information of the display device (100) to the display device (100). Operation 1860 may correspond to operation 404 of FIG. 4.

[0239] In operation 1870, the source device (300) may generate a control command for the display device (100) based on the layout information and the position information.

[0240] In operation 1880, the source device (300) can control the display device (100) by transmitting a control command for the display device (100) to the display device (100).

[0241] The display device (100) that has received the control command can set the position of each of the plurality of display modules based on layout information and / or position information. The display device (100) can display each partial image of the entire image corresponding to the position of each of the plurality of display modules.

[0242] FIG. 19 is a diagram showing an RGB color space (1910) and an HSV color space (1920) according to one embodiment of the present disclosure.

[0243] Referring to FIG. 19, a processor (210) according to one embodiment can extract a brightness component from a pixel in RGB format within an image through mathematical expression 1.

[0244]

[0245]

[0246]

[0247] According to mathematical expression 1, the processor (210) calculates R', G', and B' values ​​from each of R, G, and B given as values ​​from 0 to 255. The processor (210) sets the largest value among each of the values ​​of R', G', and B' as the maximum value (Cmax), and can obtain the maximum value (Cmax) as brightness data.

[0248] However, the method of extracting the brightness component from the RGB format is only an example and is not limited to the above-described example.

[0249] An electronic device according to one embodiment of the present disclosure includes at least one memory storing one or more instructions, and one or more processors executing the one or more instructions.

[0250] According to one embodiment of the present disclosure, one or more processors acquire a first captured image by capturing a display device in a state of outputting a first image to a screen area by executing one or more instructions.

[0251] One or more processors according to one embodiment of the present disclosure acquire a second captured image of a display device with its screen turned off.

[0252] One or more processors according to one embodiment of the present disclosure obtain a second operation image in which a brightness component is extracted from a first captured image.

[0253] One or more processors according to one embodiment of the present disclosure obtain a third operation image in which a brightness component is extracted from a second captured image.

[0254] One or more processors according to one embodiment of the present disclosure identify a screen area based on a result image obtained by performing a second operation image and a third operation image on the second operation image.

[0255] An electronic device according to one embodiment of the present disclosure can perform image processing on a captured image captured by a display device to obtain a resulting image in which light from a peripheral area of ​​the display device is removed. The electronic device can use the resulting image to identify a screen area of ​​the display device.

[0256] According to one embodiment of the present disclosure, a first captured image may include a luminance component of a screen area and a luminance component of a surrounding area corresponding to the periphery of the screen area. A second captured image may include a luminance component of the surrounding area.

[0257] A display device according to one embodiment of the present disclosure may include a plurality of display modules.

[0258] A first image according to one embodiment of the present disclosure may be an image in which each of a plurality of display modules outputs the same pattern or the same color.

[0259] One or more processors according to one embodiment of the present disclosure can obtain a first operation image from which a brightness component of a screen area has been removed by performing an AND operation on a first captured image and a second captured image by executing one or more instructions.

[0260] According to one embodiment of the present disclosure, one or more processors obtain the second operation image in which a brightness component of a screen area and a brightness component of the surrounding area are extracted from the first captured image by executing one or more instructions, obtain the third operation image in which a brightness component of the surrounding area is extracted from the second captured image, and obtain the result image by performing an XOR operation on the second operation image and the third operation image, wherein the result image may not include a brightness component of the surrounding area.

[0261] One or more processors according to one embodiment of the present disclosure can identify an area in a resulting image that includes a luminance component greater than a predetermined threshold value as the screen area by executing one or more instructions.

[0262] According to one embodiment of the present disclosure, one or more processors obtain brightness data from the result image by executing one or more instructions, wherein the brightness data corresponds to each of one or more units of the result image, each of the one or more units includes a predetermined number of pixels of the result image, the brightness data includes first brightness data and second brightness data, and upon identifying the second brightness data of one of the one or more units, brightness data of a next unit adjacent to the one unit is identified, and when the second brightness data is also identified in the next unit, the location of the last unit that was able to identify the first brightness data can be determined as one end of the screen area.

[0263] One or more processors according to one embodiment of the present disclosure can obtain size information and ratio information of a screen area based on a screen area identified through a result image by executing one or more instructions.

[0264] According to one embodiment of the present disclosure, a first image includes an image output by a plurality of display modules included in the display device, and one or more processors execute one or more instructions to obtain display module information related to the plurality of display modules from the display device including the plurality of display modules, identify layout information of the display device based on ratio information of a screen area and the display module information, and obtain position information of a plurality of areas corresponding to each of the plurality of display modules in the first captured image based on the layout information.

[0265] The plurality of display module information according to one embodiment of the present disclosure may include at least one of information on the number of the plurality of display modules or information on the size of each of the plurality of display modules.

[0266] According to one embodiment of the present disclosure, one or more processors may acquire a plurality of candidate layout information based on information about the number of display modules by executing one or more instructions. The one or more processors may identify one of the plurality of candidate layout information as layout information based on size information of each of the plurality of display modules and ratio information of the display device.

[0267] One or more processors according to one embodiment of the present disclosure may transmit position information and / or layout information to a display device via a communication module by executing one or more instructions.

[0268] An operating method of an electronic device according to one embodiment of the present disclosure includes a step of obtaining a first captured image by capturing a display device in a state of outputting a first image to a screen area, a step of obtaining a second captured image by capturing a display device in a state of having the screen turned off, a step of obtaining a second computed image in which a brightness component is extracted from the first captured image, a step of obtaining a third computed image in which a brightness component is extracted from the second captured image, and a step of identifying a screen area based on a result image obtained by performing a second operation on the second computed image and the third computed image.

[0269] The step of obtaining a first operation image according to one embodiment of the present disclosure may include the step of obtaining a first operation image from which a brightness component of a screen area is removed by performing an AND operation on a first captured image and a second captured image.

[0270] The step of obtaining the second associative image according to one embodiment of the present disclosure may include a step of extracting a brightness component of a screen area and a brightness component of the peripheral area from the first captured image, the step of obtaining the third operation image may include a step of extracting a brightness component of the peripheral area from the second captured image, and the step of identifying the screen area based on the result image may include a step of performing an XOR operation on the second operation image and the third operation image, thereby obtaining the result image, wherein the result image does not include a brightness component of the peripheral area.

[0271] The step of identifying a screen area through a result image according to one embodiment of the present disclosure may include a step of identifying a portion of the result image that includes a brightness component greater than a predetermined threshold value as the screen area.

[0272] An operating method of an electronic device according to one embodiment of the present disclosure may further include a step of reading brightness data in units of a predetermined number of pixels from a result image including first brightness data and second brightness data, a step of identifying brightness data of a next unit adjacent to a unit when second brightness data is identified in a unit, and a step of determining a position of a last unit where first brightness data was able to be identified as one end of a screen area when second brightness data is also identified in the next unit.

[0273] An operating method of an electronic device according to one embodiment of the present disclosure may further include a step of obtaining size information and ratio information of a screen area based on a screen area identified through a result image, a step of identifying layout information of a display device based on the ratio information of the screen area and information on a plurality of display modules included in the display device, and a step of obtaining position information of an area corresponding to each of a plurality of display modules in a first captured image based on the layout information.

[0274] A device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, the term "non-transitory storage medium" simply means a tangible device that does not contain signals (e.g., electromagnetic waves). This term does not distinguish between cases where data is permanently stored in the storage medium and cases where data is temporarily stored. For example, a "non-transitory storage medium" may include a buffer in which data is temporarily stored.

[0275] According to one embodiment, the method according to various embodiments disclosed in the present document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) through an application store or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product (e.g., a downloadable app) may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

Claims

1. In an electronic device (200), At least one memory (220) storing one or more instructions; and comprising one or more processors (210) executing one or more instructions; By executing the one or more instructions by the one or more processors (210), the electronic device (200) Obtain a first captured image (600) by capturing a display device (100) in a state of outputting a first image (610) to a screen area (50), Obtain a second captured image (700) of the display device (100) with the screen turned off, Obtain a second operation image (900) from which a brightness component is extracted from the first shooting image (600), Obtain a third operation image (1000) from which a brightness component is extracted from the second shooting image (700), An electronic device (200) that identifies the screen area (50) based on a result image (1100) obtained by performing a second operation on the second operation image (900) and the third operation image (1000).

2. In paragraph 1, The above first captured image (600) includes a brightness component of the screen area (50) and a brightness component of a surrounding area (60, 70) corresponding to the surrounding of the screen area (50), The second captured image (700) of the electronic device (200) includes a brightness component of the surrounding area (60, 70).

3. In paragraph 1 or 2, The above display device (100) includes a plurality of display modules (10), The above first image (610) is an image in which each of the plurality of display modules (10) outputs the same pattern or the same color, in an electronic device (200).

4. In any one of paragraphs 1 to 3, An electronic device (200) in which the one or more instructions are executed by the one or more processors (210), thereby performing an AND operation on the first captured image (600) and the second captured image (700), thereby obtaining a first operation image (800) in which the brightness component of the screen area (50) is removed.

5. In any one of paragraphs 1 to 4, By executing the one or more instructions by the one or more processors (210), the electronic device (200) Obtain the second operation image (900) from which the brightness component of the screen area (50) and the brightness component of the surrounding area (60, 70) are extracted from the first shooting image (600), Obtain the third operation image (1000) from which the brightness component of the surrounding area (60, 70) is extracted from the second shooting image (700), An electronic device (200) that obtains the result image (1100) by performing an XOR operation on the second operation image (900) and the third operation image (1000), wherein the result image (1100) does not include a brightness component of the surrounding area (60, 70).

6. In any one of paragraphs 1 to 5, An electronic device (200) in which the one or more instructions are executed by the one or more processors (210), thereby identifying an area in the result image (1100) that includes a brightness component greater than a predetermined threshold value as the screen area (50).

7. In any one of paragraphs 1 to 6, By executing the one or more instructions by the one or more processors (210), the electronic device (200) Obtain brightness data from the above result image (1100), wherein the brightness data corresponds to each of one or more units of the above result image (1100), each of the one or more units includes a predetermined number of pixels of the above result image (1100), and the brightness data includes first brightness data and second brightness data. By identifying the second brightness data of one of the one or more units, identifying the brightness data of the next unit adjacent to the one unit, An electronic device (200) that determines the location of the last unit that was able to identify the first brightness data as one end of the screen area (50) when the second brightness data is also identified in the next unit.

8. In any one of paragraphs 1 to 7, An electronic device (200) in which the one or more instructions are executed by the one or more processors (210), thereby obtaining size information and ratio information of the screen area (50) based on the result image (1100).

9. In paragraph 8, The above first image (610) includes an image output by a plurality of display modules included in the display device (100), By executing the one or more instructions by the one or more processors (210), the electronic device (200) Obtaining display module information related to the plurality of display modules (10) from the display device (100) including the plurality of display modules (10), Based on the ratio information of the screen area (50) and the display module information, the layout information of the display device (100) is identified, An electronic device (200) that obtains location information of a plurality of areas corresponding to each of the plurality of display modules (10) in the first captured image (600) based on the above layout information.

10. In paragraph 9, The above display module information includes at least one of information on the number of the plurality of display modules (10) or information on the size of each of the plurality of display modules (10). By executing the one or more instructions by the one or more processors (210), the electronic device (200) obtains a plurality of candidate layout information based on the number information of the plurality of display modules (10), An electronic device (200) that identifies a layout included in the plurality of candidate layout information based on size information of each of the plurality of display modules (10) and ratio information of the display device (100).

11. In paragraph 9 or 10, Further comprising a communication module (230), An electronic device (200) in which the one or more instructions are executed by the one or more processors (210), thereby transmitting the location information and / or the layout information to the display device (100) through the communication module (230).

12. In the operating method of the electronic device (200), A step (1310) of acquiring a first captured image (600) by capturing a display device (100) in a state of outputting a first image (610) to a screen area (50); A step (1320) of acquiring a second captured image (700) of the display device (100) with the screen turned off; A step (1330) of obtaining a second operation image (900) by extracting a brightness component from the first captured image (600); A step (1340) of obtaining a third operation image (1000) by extracting a brightness component from the second captured image (700); and A method comprising a step (1350) of identifying the screen area (50) based on a result image (1100) obtained by performing a second operation on the second operation image (900) and the third operation image (1000).

13. In paragraph 12, The step of obtaining the second association image (900) includes the step of extracting the brightness component of the screen area (50) and the brightness component of the surrounding area (60, 70) from the first captured image (600), The step of obtaining the third operation image (1000) includes the step of extracting the brightness component of the surrounding area (60, 70) from the second captured image (700), A method in which the step of identifying the screen area (50) based on the result image (1100) comprises a step of obtaining the result image (1100) by performing an XOR operation on the second operation image (900) and the third operation image (1000), wherein the result image (1100) does not include a brightness component of the surrounding area (60, 70).

14. In paragraph 12 or 13, The step of identifying the screen area (50) based on the above result image (1100) is as follows: A method comprising a step of identifying a portion of the above result image (1100) that includes a brightness component greater than a predetermined threshold value as the screen area (50).

15. A step (1310) of acquiring a first captured image (600) by capturing a display device (100) in a state of outputting a first image (610) to a screen area (50); A step (1320) of acquiring a second captured image (700) of the display device (100) with the screen turned off; A step (1330) of obtaining a second operation image (900) by extracting a brightness component from the first captured image (600); A step (1340) of obtaining a third operation image (1000) by extracting a brightness component from the second captured image (700); and A computer-readable recording medium having recorded thereon a program for performing an operation method of an electronic device, including a step (1350) of identifying the screen area (50) based on a result image (1100) obtained by performing a second operation on the second operation image (900) and the third operation image (1000).

Images