Interaction system capable of dynamically expanding recognition distance

Abstract

This interaction system comprises: a host device disposed and spaced apart from a display device, and configured to recognize the motion of a user directed at a specific point of the display device; a remote controller operatively coupled to the host device via wireless communication and configured to detect the motion of the user; and a light-emitting module disposed in a bezel area surrounding the display device, one surface of the host device, or a separate light-emitting device spaced apart from the display device, so as to adjust light-emitting areas.

Description

Interaction system with dynamically expandable recognition distance

[0001] The present invention relates to an interaction system that tracks user motion. More specifically, the present invention relates to an interaction system and method capable of dynamically expanding the recognition distance when estimating the motion of a user holding a remote controller.

[0002] In an interaction system that tracks the position and direction of a user's motion, the position and direction of an object in a 3D environment can be acquired based on optical markers and photo detectors. The position and direction of the object acquired in the 3D environment can also be used to control the 3D object within the display.

[0003] Meanwhile, the recognition distance of the remote controller held by the user can be determined by the field of view (FOV) of the camera and the arrangement interval of the light-emitting elements of the light-emitting module placed adjacent to the electronic device. Once the recognition (use) distance of the remote controller held by the user is determined, the FOV of the camera and the number and interval of the light-emitting elements placed within the light-emitting module can be determined.

[0004] In this regard, a problem occurs in which tracking the pose of the remote controller becomes impossible when the size and spacing of the fiducial markers captured by the camera change depending on the distance of the remote controller.

[0005] The purpose of this specification is to provide an interaction system capable of dynamically expanding the recognition distance when estimating the motion of a user holding a remote controller.

[0006] The purpose of this specification is to solve the problem that pose tracking of a remote controller becomes impossible when the size and spacing of fiducial markers captured by a camera change depending on the distance of the remote controller.

[0007] The purpose of this specification is to provide a method for arranging and controlling a light-emitting element of a reference marker that can operate at both close and long distances under a field of view (FOV) environment of a fixed camera.

[0008] An object of this specification is to provide a method for rapid switching between near-field and far-field emission patterns.

[0009] An interaction system for estimating user motion according to the present specification comprises: a host device disposed spaced apart from a display device and configured to recognize a user's motion directed toward a specific point of the display device; a remote controller operably coupled to the host device via wireless communication and configured to detect the user's motion; and a light-emitting module disposed on a bezel area surrounding the display device, on one side of the host device, or on a separate light-emitting device spaced apart from the display device, the light-emitting areas of which are controlled.

[0010] According to an embodiment, the remote controller may control the spacing between adjacent light-emitting areas of the light-emitting module to change based on the distance between the host device and the remote controller, and may control the light-emitting areas whose spacing changes to be photographed through a camera at a regular interval or according to the detected motion.

[0011] According to an embodiment, the remote controller may control the light emitting module to form a first light emitting pattern such that sub-light emitting regions on one side and the other side in the horizontal axis direction of the light emitting region are turned on when the distance is greater than or equal to the threshold distance. Adjacent sub-light emitting regions of the first light emitting pattern may be formed with a first length, and the first light emitting pattern may be formed with a first vertical length on the vertical axis.

[0012] According to an embodiment, when the remote controller determines that the distance is greater than or equal to the threshold distance and the image captured by the camera is small or dark, the remote controller may control the light-emitting module to form a second light-emitting pattern so that the length of the light-emitting region in the horizontal axis direction of the first light-emitting pattern increases. The second horizontal length of each light-emitting region of the second light-emitting pattern may be formed to be twice the first horizontal length of each light-emitting region of the first light-emitting pattern, and the second vertical length on the vertical axis of the second light-emitting pattern may be formed to be the same as the first vertical length of the first light-emitting pattern.

[0013] According to an embodiment, the remote controller may form a third light emitting pattern so that sub-light emitting regions of one side region and the other side region in the horizontal axis direction of the light emitting region are turned off when the distance is less than the threshold distance. A third vertical length on the vertical axis of the third light emitting pattern may be shorter than a first vertical length of the first light emitting pattern, and a third length of an adjacent sub-light emitting region of the third light emitting pattern may be formed to be shorter than a first length of an adjacent sub-light emitting region of the first light emitting pattern.

[0014] According to an embodiment, the light-emitting areas may include M light-emitting areas in the horizontal axis direction and N light-emitting areas in the vertical axis direction. The remote controller may control the light-emitting module to form a first light-emitting pattern such that the intervals between adjacent light-emitting areas spaced apart by the blinking area are spaced apart by a first horizontal interval in the horizontal axis direction and a first vertical interval in the vertical axis direction when the distance is greater than or equal to the threshold distance.

[0015] According to an embodiment, the remote controller may control the light emitting module to form a second light emitting pattern such that the 1x1 pixel structure of the first light emitting pattern becomes a 2x2 pixel structure when the brightness of the image of the first light emitting pattern is below a threshold brightness. Each light emitting area of ​​the first light emitting pattern of the 1x1 pixel structure may have only one LED turned on, and each light emitting area of ​​the second light emitting pattern of the 2x2 pixel structure may have two LEDs that are adjacent in the horizontal axis and the vertical axis turned on simultaneously.

[0016] In an embodiment, the remote controller may control the light emitting module to form a third light emitting pattern of a 1x1 pixel structure such that the adjacent light emitting areas are spaced apart by a second horizontal distance in the horizontal axis direction and a second vertical distance in the vertical axis direction when the distance is less than the threshold distance. A third vertical length on the vertical axis of the third light emitting pattern may be shorter than a first vertical length of the first light emitting pattern, the second horizontal distance may be shorter than the first horizontal distance, and the second vertical distance may be formed to be shorter than the first vertical distance.

[0017] According to an embodiment, the remote controller may control the light emitting module to emit the third light emitting pattern, which is a close-range light emitting pattern, in a first time period, and control the light emitting module to emit the first light emitting pattern, which is a long-range light emitting pattern, in a second time period following the first time period, and control the light emitting module to repeatedly form the close-range light emitting pattern and the long-range light emitting pattern.

[0018] According to an embodiment, the remote controller controls the light emitting module to emit the third light emitting pattern, which is a close-range light emitting pattern, and if the light emitting areas photographed through the camera are not recognized as the third light emitting pattern, transmits a first message associated with a failure to recognize the close-range light emitting pattern to the host device, and controls the light emitting module to emit the first light emitting pattern, which is a long-range light emitting pattern, and if the light emitting areas photographed through the camera are recognized as the first light emitting pattern, controls the light emitting module to emit the first light emitting pattern.

[0019] According to an embodiment, if the light-emitting areas captured by the camera are not recognized as the first light-emitting pattern, the remote controller can transmit a second message associated with a failure to recognize the long-distance light-emitting pattern to the host device and control the light-emitting module to emit the third light-emitting pattern, which is the short-distance light-emitting pattern.

[0020] According to an embodiment, the remote controller may control the light emitting module to emit the second light emitting pattern if the light emitting areas photographed through the camera are not recognized as the first light emitting pattern, and may transmit a second message related to a failure to recognize the long-distance light emitting pattern to the host device if the light emitting areas photographed through the camera are not recognized as the second light emitting pattern, and may control the light emitting module to emit the third light emitting pattern, which is the short-distance light emitting pattern.

[0021] According to an embodiment, the remote controller may control the spacing between adjacent light-emitting areas of the bezel area to change based on the distance between the host device and the remote controller, and may control the light-emitting areas of which the spacing changes to be photographed through a camera at a regular interval or according to the detected motion. The bezel area may include a first bezel area and a second bezel area formed at an upper portion and a lower portion of the display device, and a third bezel area and a fourth bezel area formed at one side and the other side of the display device.

[0022] According to an embodiment, M light-emitting elements may be arranged in a horizontal axis direction in each of the first bezel area and the second bezel area, and N light-emitting elements less than M may be arranged in a vertical axis direction in each of the third bezel area and the fourth bezel area. The remote controller may control the light-emitting elements in the bezel area so that adjacent light-emitting elements in the bezel area form a first light-emitting pattern spaced apart by a first interval when the distance is greater than or equal to the threshold distance. The number of light-emitting elements emitting light in the first bezel area and the second bezel area recognized within the field of view of the camera is characterized in that less than k.

[0023] According to an embodiment, the remote controller may control the bezel area so that adjacent light-emitting elements within the bezel area are spaced apart by a second interval when the distance is less than the threshold distance, thereby forming a third light-emitting pattern. The second interval of the third light-emitting pattern may be formed narrower than the first interval of the first light-emitting pattern, and third positions of the light-emitting elements that emit light in the third light-emitting pattern may be different from the first positions of the light-emitting elements that emit light in the first light-emitting pattern. The number of light-emitting elements that emit light in the third bezel area and the fourth bezel area recognized within the field of view of the camera may be k or more.

[0024] According to an embodiment, the remote controller may control the light emitting elements of the bezel area so that the third light emitting pattern, which is a close-range light emitting pattern, is emitted in a first time period, control the light emitting elements of the bezel area so that the first light emitting pattern, which is a far-range light emitting pattern, is emitted in a second time period following the first time period, and control the light emitting elements of the bezel area so that the close-range light emitting pattern and the far-range light emitting pattern are repeatedly formed.

[0025] According to an embodiment, the remote controller controls the light emitting elements of the bezel area to emit the third light emitting pattern, which is a close-range light emitting pattern, and if the light emitting areas photographed through the camera are not recognized as the third light emitting pattern, transmits a first message associated with a failure to recognize the close-range light emitting pattern to the host device, and controls the light emitting elements of the bezel area to emit the first light emitting pattern, which is a long-range light emitting pattern, and if the light emitting areas photographed through the camera are recognized as the first light emitting pattern, controls the light emitting elements of the bezel area to emit the first light emitting pattern.

[0026] According to an embodiment, the remote controller may control the light emitting elements of the bezel area so that the light emitting elements of the first group within the bezel area emit a first light emitting pattern spaced apart by a first interval, and the light emitting elements of the second group emit a third light emitting pattern spaced apart by a second interval. The first light emitting pattern may be formed when the light emitting elements of the first group emit a first optical signal of a first wavelength band, and the third light emitting pattern may be formed when the light emitting elements of the second group emit a second optical signal of a second wavelength band different from the first wavelength band. Third positions of the light emitting elements emitting light in the third light emitting pattern may be different from first positions of the light emitting elements emitting light in the first light emitting pattern.

[0027] According to an embodiment, when the light-emitting areas captured by the camera are recognized as the first light-emitting pattern, the remote controller can control the light-emitting elements of the bezel area to emit the first light-emitting pattern, which is a long-distance light-emitting pattern. When the light-emitting areas captured by the camera are recognized as the third light-emitting pattern, the remote controller can control the light-emitting elements of the bezel area to emit the third light-emitting pattern, which is a short-distance light-emitting pattern. When the light-emitting areas captured by the camera are not recognized as both the first light-emitting pattern and the third light-emitting pattern, the remote controller can transmit a third message associated with a recognition failure to the host device. The remote controller can control the light-emitting elements of the bezel area to emit a second light-emitting pattern, which has an increased light-emitting area than the first light-emitting pattern.

[0028] The technical effects of the interaction system capable of dynamically expanding the recognition distance according to this specification can be summarized as follows, but are not limited thereto.

[0029] According to this specification, an interaction system capable of dynamically expanding a recognition distance when estimating the motion of a user holding a remote controller is provided.

[0030] According to this specification, even if the size and spacing of fiducial markers captured by a camera change depending on the distance of the remote controller, it is possible to track the pose of the remote controller by changing the marker pattern depending on the close / far distance.

[0031] According to the present specification, a method for arranging and controlling a light-emitting element of a reference marker capable of operating at both a close range and a long range under a field of view (FOV) environment of a fixed camera can be provided by changing the marker pattern according to the close range / long range.

[0032] According to this specification, marker patterns can be dynamically changed based on distance, providing a user experience that dynamically adapts to changes in the distance between the user and the display device. This enables the provision of diverse user experiences in metaverse or digital twin products.

[0033] According to the present specification, rapid switching between a near-field emission pattern and a far-field emission pattern is possible by using a predetermined emission pattern or emission pattern control of a sub-array structure in which a non-emission region and an emission region are distinguished.

[0034] Further scope of the applicability of the present invention will become apparent from the detailed description below. However, since various modifications and variations within the spirit and scope of the present invention will become apparent to those skilled in the art, it should be understood that the detailed description and specific examples, such as preferred embodiments of the present invention, are given by way of example only.

[0035] Figure 1 illustrates a user interaction system capable of dynamically expanding recognition distance according to the present specification.

[0036] Fig. 2 shows a detailed configuration of a remote controller for estimating a user's motion in an interaction system according to the present specification.

[0037] Figure 3 shows a detailed configuration of a host device and a light-emitting device that control a light-emitting module in an interaction system according to the present specification.

[0038] FIG. 4 illustrates a conceptual diagram of a single size, minimum spacing, and arrangement spacing of light-emitting patterns in an interaction system capable of distance-aware dynamic expansion according to the present specification.

[0039] FIG. 5 is a drawing showing luminous patterns recognizable at a distance and a close distance in an interaction system according to the present specification.

[0040] FIG. 6 is a drawing showing light-emitting patterns implemented in sub-array units in an interaction system according to the present specification.

[0041] FIG. 7 is a drawing showing light emitting patterns recognizable from a distance in relation to light emitting patterns implemented in the sub-array units of FIG. 6.

[0042] FIG. 8 is a drawing showing luminescence patterns recognizable at close range in relation to luminescence patterns implemented in the sub-array units of FIG. 6.

[0043] FIG. 9 is a drawing showing light-emitting areas composed of a total array unit of a plurality of light-emitting elements in the interaction system of the present specification.

[0044] FIG. 10 is a drawing showing light emitting patterns recognizable from a distance in relation to light emitting patterns implemented as the entire array unit of FIG. 9.

[0045] FIG. 11 is a drawing showing luminescent patterns recognizable at close range in relation to luminescent patterns implemented as the entire array unit of FIG. 9.

[0046] Figure 12 shows a light emission control method for extending the recognition distance in a time-division manner by using different light emission patterns for each time interval.

[0047] Fig. 13 shows a light control method for extending the recognition distance by using different light emitting patterns in relation to whether the light emitting pattern is recognized by the remote controller.

[0048] Figure 14 shows a long-distance light emission pattern and a short-distance light emission pattern formed in the bezel area of ​​the display device.

[0049] Figure 15 shows an example in which a long-distance light-emitting pattern and a short-distance light-emitting pattern formed in a bezel area of ​​a display device are simultaneously emitted.

[0050] The technology disclosed herein is applicable to a user interaction system that tracks a user's motion. However, the technology disclosed herein is not limited to this system and can be applied to any user interaction system, method, or device to which the technical principles of the technology can be applied.

[0051] It should be noted that the technical terms used in this specification are used merely to describe specific embodiments and are not intended to limit the present invention. Furthermore, unless specifically defined otherwise herein, the technical terms used herein should be interpreted as having a meaning generally understood by those skilled in the art to which the present invention pertains, and should not be interpreted in an excessively broad or narrow sense. Furthermore, if a technical term used herein is incorrect and does not accurately express the spirit of the present invention, it should be replaced with a technical term that can be correctly understood by those skilled in the art. Furthermore, general terms used herein should be interpreted according to their dictionary definitions or according to the context, and should not be interpreted in an excessively narrow sense.

[0052] Additionally, the singular expressions used herein include plural expressions unless the context clearly dictates otherwise. In this application, terms such as "consist of" or "comprises" should not be construed to necessarily include all of the various components or various steps described in the specification, and should be construed to mean that some of the components or some of the steps may not be included, or that additional components or steps may be included.

[0053] In addition, the suffixes "module" and "part" used in this specification for components are given or used interchangeably only for the convenience of writing the specification, and do not have distinct meanings or roles in themselves.

[0054] Additionally, terms including ordinal numbers, such as "first," "second," etc., used herein may be used to describe various components, but these components should not be limited by these terms. These terms are used solely to distinguish one component from another. For example, without departing from the scope of the present invention, a first component could be referred to as a "second component," and similarly, a second component could also be referred to as a "first component."

[0055] Hereinafter, a preferred embodiment of the present invention will be described in detail with reference to the attached drawings. Regardless of the drawing numbers, identical or similar components are given the same reference numbers and redundant descriptions thereof will be omitted.

[0056] Furthermore, when describing the present invention, detailed descriptions of related known technologies will be omitted if they are deemed to obscure the gist of the present invention. Furthermore, it should be noted that the attached drawings are intended solely to facilitate understanding of the spirit of the present invention and should not be construed as limiting the spirit of the present invention.

[0057] In this regard, Fig. 1 illustrates a user interaction system capable of dynamically expanding the recognition distance according to the present specification. In this regard, an object of the present specification is to provide an interaction system capable of dynamically expanding the recognition distance when estimating the motion of a user holding a remote controller. An object of the present specification is to solve a problem in which pose tracking of a remote controller becomes impossible when the size and spacing of fiducial markers captured by a camera change according to the distance of the remote controller. An object of the present specification is to provide a method for arranging and controlling a light-emitting element of a fiducial marker that can operate at both a close range and a long range under a fixed camera field of view (FOV) environment. An object of the present specification is to provide a method for quickly switching between a close range light-emitting pattern and a long range light-emitting pattern.

[0058] Referring to FIG. 1, a user interaction system may be configured to include a remote controller (100) and a host device (200). The user interaction system may further be configured to include a display device (300), a light-emitting module (400), and a light-emitting device (500). The host device (200) may be configured to be included in the display device (300). As another example, the host device (200) may be configured to be placed outside the display device (300).

[0059] A light-emitting module (400) may be displayed on one side of the host device (200), for example, the front surface. The remote controller (100) may include an IR camera (105) for recognizing the light-emitting module (400). The remote controller (100) is equipped with necessary additional input devices, for example, a key, a joystick, a trackball, etc. In addition, a wireless device such as Bluetooth (BT / BLE) for communicating with the host device (200) and a 6-axis motion sensor for sensor fusion during 6-DoF tracking are included. The 6-axis motion sensor may include an acceleration sensor and a gyro sensor.

[0060] The host device (200) can display a 3D virtual environment on the display device (300). In addition, the host device (200) can display 6-DoF information or coordinate information acquired from the remote controller (100) within the 3D virtual environment. The host device (200) may also be included within the display device (300).

[0061] The light-emitting module (400) is information that serves as a reference for the remote controller (100) to calculate the 6-DoF value through the IR camera (105). The light-emitting module (400) may be placed on one side of the host device (200) or in the bezel area of ​​the display device (300). The light-emitting module (400) may be configured to include a plurality of markers. The plurality of markers of the light-emitting module (400) may be referred to as IR markers because they can be recognized by the IR camera (105) of the remote controller (100).

[0062] Based on the distance / pattern / number, etc. between the markers of the light-emitting module (400), the overall size and recognizable distance for estimating user motion can be determined. The light-emitting module (400) can be included in the host device (200) or can be included in the outer area of ​​the display device (300) or in a separate structure.

[0063] When the host device (200) is included in the display device (300), the light emitting module (400) may be placed in the bezel area of ​​the display device (300) or on a separate stand. The shape of the light emitting module (400) may be circular or polygonal, but is not limited thereto and may be any shape depending on the application. The arrangement structure of the light emitting module (400) may be arranged in a predetermined cycle on the horizontal axis and the vertical axis or may be a structure shifted by a predetermined interval.

[0064] The display device (300) is a device that creates and displays 3D virtual space and objects. The IR camera (105) can be mounted on the remote controller (100). By ignoring external visible light through the optical filter and the IR camera (105), only the IR marker image can be extracted and acquired, thereby minimizing the amount of computation when tracking the user's motion.

[0065] In this regard, the user interaction system according to the present specification may be configured to include a remote controller (100), a host device (200), and a light-emitting module (400). The IR camera (105) of the remote controller (100) may capture an IR LED pattern, which is a fiducial marker, as a real-time image, calculate a pose (e.g., 6-DoF) of the remote controller (100), and then wirelessly transmit the 6-DoF coordinates to the host device (200).

[0066] Meanwhile, a remote controller (100) that wirelessly exchanges information with a host device (200) and tracks a user's motion may be configured to include a plurality of components. Fig. 2 illustrates a detailed configuration of a remote controller that estimates a user's motion in an interaction system according to the present specification. Meanwhile, Fig. 3 illustrates a detailed configuration of a host device and a light-emitting device that control a light-emitting module in an interaction system according to the present specification.

[0067] Referring to FIG. 2, the remote controller (100) may include an IR camera (105), a wireless communication unit (110), a sensor module (120), a haptic driver (130), and a processor (140). The remote controller (100) may further include a power management integrated chip (PMIC) (150) and a memory (160).

[0068] The IR camera (105) may be configured to capture the light emitting pattern of the light emitting module (400). The wireless communication unit (110) may be configured to transmit and receive wireless signals to and from the host device (200). The wireless communication unit (110) may be implemented as a wireless communication chipset and may include an RFIC for transmitting, receiving, and processing wireless signals and a modem for processing frequency-converted signals. The sensor module (120) may be configured to estimate the motion of the user and may be configured to include an acceleration sensor, a gyro sensor, and an inertial sensor. The haptic driver (130) is configured to provide a feedback output, such as vibration, to a user holding the remote controller (100) according to a user input or control signal.

[0069] The processor (140) is configured to be operatively coupled with the IR camera (105), the wireless communication unit (110), the sensor module (120), and the haptic driver (130) and to control their operations. The power management integrated chip (PMIC) (150) is configured to provide power to each component of the remote controller (100) and control the power consumed. The memory (160) is configured to store preset information and information received from the host device (200) and the display device (300).

[0070] The remote controller (100) is equipped with a processor (140) and a memory (160) for calculating the posture of the remote controller (100) in real time through data (IR LED pattern) and a sensor module (120) using an image captured by an IR camera (105). The sensor module (120) may further include a gyro sensor and an acceleration sensor as well as an inertial sensor (IMU). The remote controller (100) may be equipped with buttons for operating functions, a multi-directional joystick (e.g., a 4-way joystick), a trackball, etc.

[0071] Referring to FIG. 3, a host device (200) may include a wireless communication unit (210), a display driving circuit (220), a power management integrated chip (PMIC) (230), a processor (240), and a memory (250). A light-emitting device (500) in which a light-emitting module (400) including a plurality of light-emitting elements is arranged may include a light-emitting element driving control unit (510) and a power management integrated chip (PMIC) (520).

[0072] The wireless communication unit (210) may be configured to transmit and receive wireless signals to and from the remote controller (100). The wireless communication unit (210) may be implemented as a wireless communication chipset and may include an RFIC for transmitting, receiving, and processing wireless signals and a modem for processing frequency-converted signals. The display driving circuit (220) is configured to control the display device (300) or the light-emitting element driving control unit (510) of the light-emitting device (500). The power management integrated chip (PMIC) (230) is configured to provide power to each component of the host device (200) and control the power consumed.

[0073] The processor (240) is configured to be operatively coupled with the IR camera (105), the wireless communication unit (110), the sensor module (120), and the haptic driver (130) and to control their operations. The memory (250) is configured to store preset information and information received from the remote controller (100), the display device (300), and the light-emitting device (500).

[0074] Referring to FIGS. 1 to 3, an interaction system (1000) capable of dynamically expanding distance recognition according to the present specification is described. In this regard, it is necessary to capture a known light-emitting pattern arranged on a light-emitting device (500) with an IR camera (105) to calculate the pose of the IR camera (105) in real time. The light-emitting pattern may be a pattern emitted by an IR LED, but is not limited thereto and may be changed according to the application. The pose of the IR camera (105) can be expressed as 6-DoF pose information including the 3-axis position and 3-axis orientation of the remote controller (100). In order to calculate the 6-DoF pose information of the remote controller (100) in real time, the arrangement of the LEDs viewed by the IR camera (105) from various angles or distances must satisfy the following conditions.

[0075] Meanwhile, in the interaction system capable of dynamically expanding distance recognition according to the present specification, the single size, minimum spacing, and arrangement spacing of the light-emitting pattern need to be configured to be recognizable by a camera at both close and long distances. In this regard, Fig. 4 illustrates a conceptual diagram of the single size, minimum spacing, and arrangement spacing of the light-emitting pattern in the interaction system capable of dynamically expanding distance recognition according to the present specification.

[0076] Referring to Fig. 4(a), the size of a single blob (Binary Large Object) must be large enough to be captured at least 4 pixels wide and 4 pixels high. In this regard, as the distance between the remote controller (100) and the host device (200) or the light-emitting device (500) increases, the brightness of the light-emitting element recognized by the IR camera (105) decreases. In this regard, the minimum size becomes smaller as the blob size decreases.

[0077] Referring to Fig. 4(b), the minimum spacing between the light-emitting elements of the light-emitting module (400) needs to be arranged so that they do not overlap each other by several pixels or more when the IR camera (105) is tilted. Here, the posture of the IR camera (105) corresponds to the posture of the remote controller (100).

[0078] Referring to Fig. 4(c), it is advantageous for the performance of posture calculation to design the spacing between the light-emitting elements of the light-emitting module (400) to be different rather than uniform. It is advantageous for the arrangement of the light-emitting elements of the light-emitting module (400) to be arranged in the second row rather than the first row so that different patterns can be seen depending on the angle at which the camera captures them. It is advantageous for posture calculation to have four or more light-emitting elements captured by the IR camera at a time.

[0079] Referring to FIGS. 1 to 4, an interaction system capable of distance recognition dynamic expansion according to the present specification will be described. A host device (200) may be disposed spaced apart from a display device (300). The host device (200) may be configured to recognize a user's motion directed toward a specific point of the display device (300). A remote controller (100) may be operatively coupled to the host device (200) via wireless communication. The remote controller (100) may be carried by a user and disposed in a front area of ​​the display device (300). The remote controller (100) may be configured to detect a user's motion.

[0080] The light-emitting module (400) may be arranged on a bezel area surrounding the display device (300), on one side of a host device (200), or on a separate light-emitting device (500) spaced apart from the display device (300) so that the light-emitting areas can be adjusted. The separate light-emitting device (500) spaced apart from the display device (300) may be arranged on the upper part of the display device (300). The light-emitting device (500) may be referred to as an LED light-emitting module since IR-markers recognized by the IR camera (105) are implemented by LED elements.

[0081] The host device (200) and the light emitting device (500) can be connected by wires to enable wired communication via a cable. The remote controller (100) and the host device (200) can be connected to enable wireless communication by transmitting and receiving wireless signals.

[0082] The remote controller (100) can control the spacing between adjacent light-emitting areas of the light-emitting module (400) to change based on the distance between the host device (200) and the remote controller (100). In this regard, FIG. 5 is a drawing showing light-emitting patterns recognizable at a long distance and a short distance in the interaction system according to the present specification.

[0083] Referring to FIG. 5, the light-emitting module (400) can be implemented with a structure in which light-emitting elements such as LEDs are arranged on a PCB (Printed Circuit Board) or an FPCB (Flexible Printed Circuit Board). The light-emitting elements of the light-emitting module (400) can be implemented to emit light in two patterns, a long-distance pattern and a short-distance pattern, determined according to the recognition distance of the remote controller. In this regard, the first length (L1) and the third length (L3), which are the intervals between the light-emitting elements in the first light-emitting pattern (LP1) and the third light-emitting pattern (LP3) of the long-distance pattern and the short-distance pattern, can be formed differently.

[0084] The light emitting patterns can be adjusted according to the distance between the remote controller and the light emitting module (400). When the distance between the light emitting patterns of the light emitting module (400) and the remote controller is far, the light emitting elements at the corresponding positions can be turned on so that a first light emitting pattern (LP1) arranged at a wide interval is formed. When the distance between the light emitting patterns of the light emitting module (400) and the remote controller is close, the light emitting elements at the corresponding positions can be turned on so that a third light emitting pattern (LP3) arranged at a narrow interval is formed.

[0085] Meanwhile, Fig. 6 is a drawing showing light emitting patterns implemented in sub-array units in the interaction system according to the present specification. Referring to Fig. 6, the light emitting module (400) can be implemented in a structure in which light emitting elements such as LEDs are arranged in a sub-array structure on a PCB or FPCB. By configuring the array of light emitting elements in a specific number (e.g., 3x3 array), the on / off operation of the light emitting elements can be individually controlled within a set array structure. There may be restrictions on the shape of the light emitting pattern of the light emitting module (400) implemented with LEDs. However, since the implementation of the near / far light emitting pattern is relatively simple, the dynamic expansion of the recognition distance can be quickly achieved according to the user's motion.

[0086] FIG. 7 is a drawing showing light emitting patterns recognizable from a distance in relation to light emitting patterns implemented in the sub-array units of FIG. 6. Referring to FIG. 7, the spacing and pattern of light emitting elements that emit light can be adjusted according to the distance between the remote controller and the light emitting module (400). When the distance between the remote controller and the light emitting module (400) is far, the spacing between the light emitting elements can be widened to a first length (L1) so that individual light emitting elements can be turned on. In this regard, when the image of the first light emitting pattern (LP1) is small and dark in the image captured by the camera of the remote control, a second light emitting pattern (L2) can be formed so that a plurality of light emitting elements are turned on to improve brightness and increase image size.

[0087] Fig. 8 is a drawing showing light emitting patterns recognizable at close range in relation to light emitting patterns implemented in the sub-array units of Fig. 6. Referring to Fig. 8, when the distance between the remote controller and the light emitting module (400) is close, the interval between the light emitting elements can be narrowed to a third length (L3) so that the individual light emitting elements are turned on.

[0088] Although each of the sub-array units in FIG. 6 is configured as a 3x3 LED array, this is not limited to the configuration and may be changed depending on the application. For example, as shown in FIGS. 7 and 8, each of the sub-array units may be configured as a 4x4 LED array.

[0089] Referring to FIGS. 5 to 8, the light emitting module (400) can be operably coupled to the processor (240) via the display driving circuit (220). The processor (240) can control the light emitting module (400) to form a close-range light emitting pattern or a long-range light emitting pattern in the light emitting module (400). The processor (240) can be implemented as a MUC (Micron Control Unit) or an AP (Application Processor). The long-range light emitting pattern can be formed as a first light emitting pattern (LP1) or a second light emitting pattern (LP2). The close-range light emitting pattern can be formed as a third light emitting pattern (LP3).

[0090]

[0091] Referring to FIGS. 1 to 8, an interaction system capable of distance recognition dynamic expansion is described. A remote controller (100) can control adjacent light-emitting areas with changing spacing between light-emitting modules (400) to be photographed through a camera (105) at a regular interval or according to a user's detected motion.

[0092] The remote controller (100) determines whether the distance between the host device (200) and the remote controller (100) is greater than or less than a threshold distance. If the distance between the host device (200) and the remote controller (100) is greater than or equal to the threshold distance, the distance may be defined as a long distance. If the distance between the host device (200) and the remote controller (100) is less than the threshold distance, the distance may be defined as a short distance.

[0093] The remote controller (100) can control the light emitting module (400) to form a first light emitting pattern when the distance between the host device (200) and the remote controller (100) is greater than a threshold distance. Through the first light emitting pattern, the sub-light emitting regions of one side region (400a) and the other side region (400b) in the horizontal axis direction of the light emitting region of the light emitting module (400) can be turned on.

[0094] The adjacent sub-emission regions of the first emission pattern are formed with a first length (L1). The adjacent sub-emission regions of the first emission pattern are formed with the first length (L1) in one side region (400a), the other side region (400b), and the central region (400c). The first emission pattern is formed with a first vertical length (VL1) on the vertical axis.

[0095] If the distance between the host device (200) and the remote controller (100) is greater than a threshold distance and the image captured by the IR camera (105) is determined to be small or dark, the remote controller (100) can control to form a second light emitting pattern. The second horizontal length (HL2) of each light emitting region of the second light emitting pattern can be formed to be twice the first horizontal length (HL1) of each light emitting region of the first light emitting pattern. Accordingly, in a light emitting module (400) structure having a limited vertical length, the recognition success rate of the light emitting pattern of the light emitting module (400) can be improved even from a long distance. The second vertical length (VL2) on the vertical axis of the second light emitting pattern can be formed to be the same as the first vertical length (VL1) of the first light emitting pattern.

[0096] The remote controller (100) can control the light emitting module (400) to form a third light emitting pattern when the distance between the host device (200) and the remote controller (100) is less than a threshold distance. Through the third light emitting pattern, the sub light emitting regions of one side region (400a) and the other side region (400b) in the horizontal axis direction of the light emitting region of the light emitting module (400) can be turned off. A third vertical length (VL3) on the vertical axis of the third light emitting pattern can be formed shorter than a first vertical length (VL1) of the first light emitting pattern. A third length (L3) of an adjacent sub light emitting region of the third light emitting pattern can be formed shorter than a first length (L1) of an adjacent sub light emitting region of the first light emitting pattern. Therefore, when a user is present at a close range, the same number of LED patterns can be recognized within a limited area of ​​the light emitting module (400). Accordingly, the posture of the remote controller (100) can be tracked at both a long range and a close range depending on the movement of the user.

[0097] Meanwhile, the LED pattern recognition method of the interaction system according to the present specification can be applied not only to a sub-array structure but also to a full array structure. In this regard, FIG. 9 is a drawing showing light-emitting areas formed as a full array unit of a plurality of light-emitting elements in the interaction system according to the present specification. Referring to FIG. 9, a light-emitting module (400) implemented with an IR LED can be arranged in a checkerboard pattern such that each light-emitting area by each light-emitting element is sequentially arranged. The light-emitting module (400) formed with an IR LED can be operably coupled to a display driving circuit (220) so that the on / off and light-emitting brightness can be individually controlled. The degree of freedom of light-emitting patterns emitted through the light-emitting module (400) formed in a full-array structure is high. Meanwhile, when the brightness of a single LED, which is a unit element of the light-emitting module (400), is insufficient, the brightness can be increased by turning on multi-LEDs. Accordingly, the size of the light-emitting pattern captured by the camera in which multi-LEDs are turned on can be increased. The required number of LEDs (M x N) and the overall LED module size can be defined to suit the recognition distance range of the remote controller.

[0098] Meanwhile, Fig. 10 is a drawing showing light emitting patterns recognizable at a long distance in relation to light emitting patterns implemented in the entire array unit of Fig. 9. Fig. 11 is a drawing showing light emitting patterns recognizable at a close distance in relation to light emitting patterns implemented in the entire array unit of Fig. 9.

[0099] Referring to FIGS. 10 and 11, instead of installing a few light-emitting elements such as IR LEDs as point light sources, a plurality of such elements can be configured as an array matrix. In this regard, the LED arrangement can be implemented to be suitable for only two modes of operation, that is, near-field and far-field. Meanwhile, all light-emitting elements as wide (M) x long (N) can be installed. For example, a total of 300 light-emitting elements, 30 wide x 10 high, can be arranged in a light-emitting module (400). The full-array structure can flexibly form a desired LED light-emitting pattern compared to a sub-array structure in which only a specific area is illuminated.

[0100] Regarding the control method, the spacing, number, and pattern of the emitting LEDs can be adjusted depending on the distance of the remote controller. If the brightness is low or the LEDs appear small from a distance, the LED unit elements can be grouped to expand the emitting area and increase brightness.

[0101] An interaction system capable of distance recognition dynamic expansion will be described with reference to FIGS. 1 to 4 and 9 to 11. The light-emitting areas of the light-emitting module (400) may include M light-emitting areas in the horizontal axis direction and N light-emitting areas in the vertical axis direction. The remote controller (100) may control the host device (200) to form a first light-emitting pattern when the distance between the host device (200) and the remote controller (100) is greater than or equal to a threshold distance. Through the first light-emitting pattern, the intervals between adjacent light-emitting areas spaced apart by the blinking area may be spaced apart by a first horizontal interval (HG1) in the horizontal axis direction and a first vertical interval (VG1) in the vertical axis direction.

[0102] If the brightness of the image of the first light-emitting pattern is lower than or equal to a threshold brightness, the remote controller (100) can control the host device (200) to form a second light-emitting pattern so that the 1x1 pixel structure of the first light-emitting pattern becomes a 2x2 pixel structure. Each light-emitting area of ​​the first light-emitting pattern of the 1x1 pixel structure can have only one LED turned on. Meanwhile, each light-emitting area of ​​the second light-emitting pattern of the 2x2 pixel structure can have two LEDs that are adjacent in the horizontal and vertical axes turned on simultaneously.

[0103] The remote controller (100) can control the light emitting module (400) to form a third light emitting pattern when the distance between the host device (200) and the remote controller (100) is less than a threshold distance. The third light emitting pattern of a 1x1 pixel structure can be formed so that adjacent light emitting areas are spaced apart from each other by a second horizontal distance (HG2) in the horizontal axis direction and a second vertical distance (VG2) in the vertical axis direction. A third vertical length (VL3) on the vertical axis of the third light emitting pattern can be formed to be shorter than a first vertical length (VL1) of the first light emitting pattern.

[0104] The second horizontal interval (HG2) of the third light emitting pattern may be formed shorter than the first horizontal interval (HG1) of the first light emitting pattern. The second vertical interval (VG2) of the third light emitting pattern may be formed shorter than the first vertical interval (VG1) of the first light emitting pattern. Therefore, when a user is present at a close range, the same number of LED patterns can be recognized within the limited area of ​​the light emitting module (400). Accordingly, tracking of the posture of the remote controller (100) is possible both at a long distance and at a close range depending on the movement of the user.

[0105] Meanwhile, in the interaction system according to the present specification, the near-field light-emitting pattern and the far-field light-emitting pattern can be adaptively formed according to a regular cycle or the motion of the user. In this regard, the first light-emitting pattern and the second light-emitting pattern can be defined as far-field light-emitting patterns. The third light-emitting pattern can be defined as a far-field light-emitting pattern. Fig. 12 shows a light-emitting control method for extending the recognition distance in a time-division manner by using different light-emitting patterns for each time interval. Referring to Fig. 12, the near-field light-emitting pattern and the far-field light-emitting pattern can be controlled to alternately light-emitting periodically in a time-division manner. Based on the recognition rate of the light-emitting pattern captured by the camera, both near-field / far-field pose tracking of the remote controller is possible. The near-field / far-field light-emitting patterns can be repeated for both near-field / far-field pose tracking of the remote controller.

[0106] Meanwhile, Fig. 13 illustrates a light emission control method for extending the recognition distance by using different light emission patterns in relation to whether the light emission pattern is recognized by the remote controller. Referring to Fig. 13, if the camera in the remote controller (100) fails to recognize the light emission pattern, information related to the inability to recognize the light emission pattern can be transmitted to the host device (200). The host device (200) changes to a long-distance light emission pattern and emits light, and if the light emission pattern is recognized by the remote controller (100), information that the light emission pattern can be recognized can be transmitted to the host device (200) again. In this regard, the light emission pattern can be controlled in the same manner even when the user's location changes from a close distance to a long distance or from a long distance to a close distance. On-demand light emission control can be performed to change the light emission patterns depending on whether the light emission pattern is recognized successfully or not.

[0107] Referring to FIGS. 1 to 13, an interaction system for dynamic determination of distance recognition according to the present specification will be described. A remote controller (200) can control a light-emitting module (400) to emit a third light-emitting pattern, which is a close-range light-emitting pattern, in a first time interval. The remote controller (200) can control the light-emitting module (400) to emit a first light-emitting pattern, which is a far-range light-emitting pattern, in a second time interval following the first time interval. The first time interval can be defined as a time interval between a first time (t1) and a second time (t2). The second time interval can be defined as a time interval between a second time (t3) and a third time (t3). The remote controller (200) can control the light-emitting module (400) to repeatedly form a close-range light-emitting pattern and a far-range light-emitting pattern.

[0108] The remote controller (200) can control the light emitting module (400) to change the light emitting pattern based on the image captured by the IR camera (105) in relation to the user's motion. The remote controller (200) can control the light emitting module (400) to emit a third light emitting pattern, which is a close-range light emitting pattern. If the light emitting areas captured by the IR camera (105) are not recognized as the third light emitting pattern, the remote controller (200) can transmit a first message related to the failure to recognize the close-range light emitting pattern to the host device (200).

[0109] The remote controller (200) can control the light emitting module (400) so that the first light emitting pattern, which is a long-distance light emitting pattern, is emitted. When the light emitting areas captured by the IR camera (105) are recognized as the first light emitting pattern, the remote controller (200) can control the light emitting module (400) so that the first light emitting pattern is emitted. In this regard, a short distance can be defined as a distance from the light emitting module (400) to the remote controller (100) less than a threshold distance (e.g., 2.5 m). A long distance can be defined as a distance from the light emitting module (400) to the remote controller (100) from a threshold distance (e.g., 2.5 m) or more to a detectable distance (e.g., 4 m).

[0110] If the light-emitting areas captured by the IR camera (105) are not recognized as the first light-emitting pattern, the remote controller (200) can transmit a second message related to the failure to recognize the long-distance light-emitting pattern to the host device (200). The remote controller (200) can control the light-emitting module (400) to emit the third light-emitting pattern, which is a short-distance light-emitting pattern.

[0111] Meanwhile, when the recognition of the first light-emitting pattern fails, a recognition attempt may be made first by increasing the light-emitting area with the second light-emitting pattern. If the light-emitting areas photographed through the IR camera (105) are not recognized as the first light-emitting pattern, the remote controller (200) may control the light-emitting module (400) to emit the second light-emitting pattern. If the light-emitting areas photographed through the IR camera (105) are not recognized as the second light-emitting pattern, the remote controller (200) may transmit a second message related to the failure in recognition of the long-distance light-emitting pattern to the host device (200). Thereafter, the remote controller (200) may control the light-emitting module (400) to emit the third light-emitting pattern, which is a short-distance light-emitting pattern.

[0112] Meanwhile, in the interaction system according to the present specification, light-emitting patterns may be formed in the bezel area of ​​the display device (300). In this regard, FIG. 14 illustrates a long-distance light-emitting pattern and a short-distance light-emitting pattern formed in the bezel area of ​​the display device. FIG. 14(a) illustrates a first light-emitting pattern (LP1), which is a long-distance light-emitting pattern formed in the bezel area (400) of the display device (300). FIG. 14(b) illustrates a third light-emitting pattern (LP3), which is a short-distance light-emitting pattern formed in the bezel area (400) of the display device (300).

[0113] Figure 15 shows an example in which a long-distance light-emitting pattern and a short-distance light-emitting pattern formed in a bezel area of ​​a display device are simultaneously emitted.

[0114] Referring to FIGS. 1 to 4 and 14, an interaction system according to the present specification that utilizes light-emitting patterns formed in a bezel area of ​​a display device will be described. A remote controller (100) can control the spacing between adjacent light-emitting areas of a bezel area (410 to 440) to change based on a distance between a host device (200) and the remote controller (100). The remote controller (100) can control the light-emitting areas of the bezel area (410 to 440) in which the spacing between adjacent light-emitting areas is changed to be photographed through an IR camera (105) at a regular interval or according to a motion detected by a user.

[0115] The bezel area may include a first bezel area (410) and a second bezel area (420) formed at the upper and lower portions of the display device (300). The bezel area may include a third bezel area (430) and a fourth bezel area (440) formed at one side and the other side of the display device (300). M light-emitting elements may be arranged in the horizontal axis direction in each of the first bezel area (410) and the second bezel area (420). N light-emitting elements, which are less than M, may be arranged in the vertical axis direction in each of the third bezel area (430) and the fourth bezel area (440). The light-emitting elements of the bezel area (400) may be LED elements, but are not limited thereto and may be implemented as arbitrary light-emitting elements depending on the application.

[0116] The remote controller (100) can control the light emitting elements of the bezel area (400) so that adjacent light emitting elements within the bezel area (400) form a first light emitting pattern spaced apart by a first distance (G1) when the distance between the host device (200) and the remote controller (100) is greater than a threshold distance. The number of light emitting elements emitting light in the first bezel area (410) and the second bezel area (420) recognized within the field of view of the IR camera (105) can be set to be less than k.

[0117] If the distance between the host device (200) and the remote controller (100) is less than a threshold distance, the remote controller (100) can control the light emitting elements in the bezel area (400) so that adjacent light emitting elements in the bezel area (400) form a third light emitting pattern spaced apart by a second distance (G2).

[0118] The second interval (G2) of the third light emitting pattern may be formed narrower than the first interval (G1) of the first light emitting pattern. The second interval (G2) of the third light emitting pattern may be formed narrower than 0.5 times the first interval (G1) of the first light emitting pattern. Third positions of the light emitting elements that emit light in the third light emitting pattern may be set differently from the first positions of the light emitting elements that emit light in the first light emitting pattern. The number of light emitting elements that emit light in the third bezel area (430) and the fourth bezel area (440) recognized in the field of view of the IR camera (105) may be set to k or more. For example, the number of light emitting elements that emit light in the third bezel area (430) and the fourth bezel area (440) may be set to 4 or more, but is not limited thereto and may be changed depending on the field of view (FOV) and application of the IR camera (105).

[0119] The near-range light emitting pattern and the far-range light emitting pattern by the light emitting elements of the bezel area (410 to 440) can be changed adaptively at regular intervals or according to changes in the user's motion. The remote controller (100) can control the light emitting elements of the bezel area (410 to 440) so that a third light emitting pattern, which is a near-range light emitting pattern, is emitted in a first time period. The remote controller (100) can control the light emitting elements of the bezel area (410 to 440) so that a first light emitting pattern, which is a far-range light emitting pattern, is emitted in a second time period following the first time period. The remote controller (100) can control the light emitting elements of the bezel area (410 to 440) so that the near-range light emitting pattern and the far-range light emitting pattern are repeatedly formed.

[0120] The remote controller (100) can control the light emitting elements of the bezel area (410 to 440) to emit a third light emitting pattern, which is a close-range light emitting pattern. If the light emitting areas photographed by the IR camera (105) are not recognized as the third light emitting pattern, the remote controller (100) can transmit a first message related to the failure to recognize the close-range light emitting pattern to the host device (200). The remote controller (100) can control the light emitting elements of the bezel area (410 to 440) to emit a first light emitting pattern, which is a long-range light emitting pattern. If the light emitting areas photographed by the IR camera (105) are recognized as the first light emitting pattern, the remote controller (100) can control the light emitting elements of the bezel area (410 to 440) to emit the first light emitting pattern.

[0121] Meanwhile, the light emitting patterns of the light emitting elements in the bezel area (410 to 440) can emit light simultaneously in different ways. In this regard, Fig. 15 illustrates a structure in which a close-range light emitting pattern and a long-range light emitting pattern emit light simultaneously.

[0122] A structure in which a near-field light emitting pattern and a far-field light emitting pattern are simultaneously emitted will be described with reference to FIGS. 1 to 4, 14, and 15. The remote controller (100) can control the light emitting elements of the bezel area (400) so that a first light emitting pattern (LP1), which is a far-field light emitting pattern, is emitted within the bezel area (400) while a third light emitting pattern (LP3), which is a near-field light emitting pattern, is emitted. The remote controller (100) can control the light emitting elements of the bezel area (400) so that the first light emitting pattern (LP1) is emitted while the light emitting elements of the first group are spaced apart by a first interval (G1), while the third light emitting pattern (LP3) is emitted while the light emitting elements of the second group are spaced apart by a second interval.

[0123] The first light emitting pattern (LP1) can be formed by the light emitting elements of the first group emitting a first optical signal of the first wavelength band. The third light emitting pattern (LP3) can be formed by the light emitting elements of the second group emitting a second optical signal of the second wavelength band different from the first wavelength band. The first optical signal of the first wavelength band and the second optical signal of the second wavelength band may be expressed in different colors because they use different wavelength bands, but are not limited thereto. In this regard, the camera (105) can be configured to recognize both the first optical signal of the first wavelength band and the second optical signal of the second wavelength band depending on the distance between the camera (105) and the light emitting device (500). Third positions of the light emitting elements emitting light in the third light emitting pattern (LP3) may be different from the first positions of the light emitting elements emitting light in the first light emitting pattern (LP1).

[0124] When the light-emitting areas photographed through the camera (105) are recognized as the first light-emitting pattern (LP1), the remote controller (100) can control the light-emitting elements of the bezel area (400) to emit the first light-emitting pattern (LP1), which is a long-distance light-emitting pattern. When the light-emitting areas photographed through the camera (105) are recognized as the third light-emitting pattern (LP3), the remote controller (100) can control the light-emitting elements of the bezel area (400) to emit the third light-emitting pattern (LP3), which is a short-distance light-emitting pattern.

[0125] If the light-emitting areas captured by the camera (105) are not recognized by both the first light-emitting pattern (LP1) and the third light-emitting pattern (LP3), the remote controller (100) can generate a third message associated with a recognition failure. The remote controller (100) can transmit the third message associated with the recognition failure to the host device (200). Thereafter, the remote controller (100) can control the light-emitting elements of the bezel area (400) so that a second light-emitting pattern having an increased light-emitting area than the first light-emitting pattern (LP1) is emitted.

[0126] As described above, the light emitting elements of the light emitting module (400) may be arranged on one side of the light emitting device (500) or in the bezel area (410 to 440). In this regard, when the light emitting module (400) is configured as a separate external module, the light emitting device (500), as shown in FIGS. 1, 2, and 5 to 11, it has the following technical characteristics.

[0127] A light-emitting module (400) including light-emitting elements can be implemented as a sub-array module. This enables the arrangement and switching of two patterns, a first light-emitting pattern (LP1), which is a long-distance light-emitting pattern, and a third light-emitting pattern (LP3), which is a short-distance light-emitting pattern. In addition, the second light-emitting pattern (LP2) can be formed so that the unit light-emitting area increases to improve the recognition success rate at a long distance. In the sub-array module, the arrangement of light-emitting elements can be implemented only in some areas, and a non-luminous empty space can be formed. Accordingly, the first light-emitting pattern (LP1) to the third light-emitting pattern (LP3) can be fixed to a preset pattern, so that the degree of freedom in pattern setting can be reduced. However, since rapid switching between patterns is relatively easy, rapid long-distance / short-distance switching is possible according to the user's movement.

[0128] A light-emitting module (400) including light-emitting elements can be implemented as a full-array module. In this regard, the light-emitting elements of the light-emitting module (400) can be arranged in an M x N LED array. The degree of freedom in setting the light-emitting patterns of the first light-emitting pattern (LP1) to the third light-emitting pattern (LP3) is the highest. By causing a plurality of light-emitting elements to emit light, it is possible to increase brightness and light-emitting size, etc.

[0129] With respect to a method for controlling the operation of light-emitting elements such as LEDs, each configuration can be controlled to emit a dedicated pattern for each distance / short distance. When a light-emitting module (400) including light-emitting elements is implemented as a full array module, the overall area size of the fiducial marker arrangement can be changed through selective LED operation through individual LED control. In order to increase the size of the LED captured by a camera from a distance, it is possible to control the LEDs so that multiple LEDs can be recognized as a single fiducial marker.

[0130] The pose of the remote controller (100) can be calculated by the remote controller (100) through the camera (105) and the processor (120) and transmitted wirelessly to the host device (200). Accordingly, the host device (200) transmits only the content to the display device (300), thereby preventing data transmission conflicts between the content transmission and the transmission of control actions based on the user's recognized motion.

[0131] Meanwhile, the light-emitting module may be arranged in the bezel areas (410 to 440) of the display device (300). In this regard, it may be implemented so that light-emitting elements such as IR LEDs are arranged in the bezel areas (410 to 440) of the display device (300). The light-emitting elements such as LEDs may be selectively driven to change the overall area size of the reference marker arrangement. The pose of the remote controller (100) may be calculated by the remote controller (100) through the camera (105) and the processor (120) and transmitted wirelessly to the host device (200). Accordingly, the host device (200) may transmit only the content to the display device (300), thereby preventing data transmission conflict between the content transmission and the transmission of the control operation for the user's recognized motion.

[0132] The above describes an interaction system capable of dynamically expanding the recognition distance. The technical effects of the interaction system capable of dynamically expanding the recognition distance according to this specification can be summarized as follows, but are not limited thereto.

[0133] According to this specification, an interaction system capable of dynamically expanding a recognition distance when estimating the motion of a user holding a remote controller is provided.

[0134] According to this specification, even if the size and spacing of fiducial markers captured by a camera change depending on the distance of the remote controller, it is possible to track the pose of the remote controller by changing the marker pattern depending on the close / far distance.

[0135] According to the present specification, a method for arranging and controlling a light-emitting element of a reference marker capable of operating at both a close range and a long range under a field of view (FOV) environment of a fixed camera can be provided by changing the marker pattern according to the close range / long range.

[0136] According to this specification, marker patterns can be dynamically changed based on distance, providing a user experience that dynamically adapts to changes in the distance between the user and the display device. This enables the provision of diverse user experiences in metaverse or digital twin products.

[0137] According to the present specification, rapid switching between a near-field emission pattern and a far-field emission pattern is possible by using a predetermined emission pattern or emission pattern control of a sub-array structure in which a non-emission region and an emission region are distinguished.

[0138] Further scope of the applicability of the present invention will become apparent from the detailed description below. However, since various modifications and variations within the spirit and scope of the present invention will become apparent to those skilled in the art, it should be understood that the detailed description and specific examples, such as preferred embodiments of the present invention, are given by way of example only.

[0139] The configuration for controlling an interaction system capable of dynamically expanding the recognition distance can be implemented as a computer-readable code on a program-recorded medium. The computer-readable medium includes all types of recording devices that store data that can be read by a computer system. Examples of computer-readable media include hard disk drives (HDDs), solid state disks (SSDs), silicon disk drives (SDDs), ROMs, RAMs, CD-ROMs, magnetic tapes, floppy disks, optical data storage devices, etc., and also includes media implemented in the form of carrier waves (e.g., transmission via the Internet). In addition, the computer may include a control unit, i.e., a processor, of a terminal or vehicle. Therefore, the above detailed description should not be construed as limiting in all respects, but should be considered as illustrative. The scope of the present invention should be determined by a reasonable interpretation of the appended claims, and all changes within the equivalent scope of the present invention are intended to be included in the scope of the present invention.

Claims

1. In electronic devices, A host device disposed spaced apart from a display device and configured to recognize a user's gesture directed toward a specific point of the display device; A remote controller operatively coupled to the host device via wireless communication and configured to detect motion of the user; and A light emitting module is disposed in a bezel area surrounding the display device, on one side of the host device, or in a separate light emitting device spaced from the display device, and the light emitting areas are controlled. The above remote controller, Controlling the spacing between adjacent light-emitting areas of the light-emitting module to change based on the distance between the host device and the remote controller, An interaction system that controls the light-emitting areas whose intervals change to be photographed by a camera at a certain cycle or according to the detected motion.

2. In paragraph 1, The above remote controller, If the distance is greater than or equal to the threshold distance, the light emitting module is controlled to form a first light emitting pattern so that one side region and the other side region in the horizontal axis direction of the light emitting region are turned on as sub light emitting regions, The adjacent sub-emitting regions of the first emitting pattern are formed with a first length, An interaction system, wherein the first light-emitting pattern is formed in a first vertical length on a vertical axis.

3. In paragraph 2, The above remote controller, If the distance is greater than or equal to the threshold distance and the image captured by the camera is determined to be small or dark, the light emitting module is controlled to form a second light emitting pattern so that the length of the light emitting area in the horizontal axis direction of the first light emitting pattern increases, The second horizontal length of each light-emitting region of the second light-emitting pattern is formed to be twice the first horizontal length of each light-emitting region of the first light-emitting pattern, An interaction system, wherein a second vertical length on the vertical axis of the second light-emitting pattern is formed to be identical to the first vertical length of the first light-emitting pattern.

4. In paragraph 2, The above remote controller, If the above distance is less than the above threshold distance, a third light emitting pattern is formed so that one side area and the other side area in the horizontal axis direction of the light emitting area are in an off state as sub light emitting areas, The third vertical length on the vertical axis of the third light emitting pattern is shorter than the first vertical length of the first light emitting pattern, An interaction system, wherein a third length of an adjacent sub-emitting region of the third light-emitting pattern is shorter than a first length of an adjacent sub-emitting region of the first light-emitting pattern.

5. In paragraph 2, The above light-emitting regions include M light-emitting regions in the horizontal axis direction and N light-emitting regions in the vertical axis direction, The above remote controller, An interaction system that controls the light emitting module to form a first light emitting pattern such that the intervals between adjacent light emitting areas spaced apart by the blinking area are spaced apart by a first horizontal interval in the horizontal axis direction and a first vertical interval in the vertical axis direction when the distance is greater than or equal to the threshold distance.

6. In paragraph 5, The above remote controller, If the brightness of the image of the first light emitting pattern is lower than or equal to a threshold brightness, the light emitting module is controlled to form a second light emitting pattern such that the 1x1 pixel structure of the first light emitting pattern becomes a 2x2 pixel structure. In each light-emitting area of ​​the first light-emitting pattern of the above 1x1 pixel structure, only one LED is turned on, An interaction system, wherein each light-emitting area of ​​the second light-emitting pattern of the above 2x2 pixel structure is in a state where two LEDs adjacent to each other in the horizontal and vertical axes are turned on simultaneously.

7. In paragraph 5, The above remote controller, If the distance is less than the threshold distance, the light emitting module is controlled to form a third light emitting pattern of a 1x1 pixel structure such that the adjacent light emitting areas are spaced apart by a second horizontal interval in the horizontal axis direction and a second vertical interval in the vertical axis direction; The third vertical length on the vertical axis of the third light emitting pattern is shorter than the first vertical length of the first light emitting pattern, An interaction system, wherein the second horizontal interval is shorter than the first horizontal interval, and the second vertical interval is shorter than the first vertical interval.

8. In paragraph 4, The above remote controller, Controlling the light emitting module so that the third light emitting pattern, which is a short-range light emitting pattern, is emitted in the first time period; Controlling the light emitting module so that the first light emitting pattern, which is a long-distance light emitting pattern, is radiated in a second time period following the first time period; An interaction system that controls the light-emitting module so that the close-range light-emitting pattern and the long-range light-emitting pattern are repeatedly formed.

9. In paragraph 8, The above remote controller, Controlling the light emitting module so that the third light emitting pattern, which is a close-range light emitting pattern, is emitted; If the light-emitting areas captured by the above camera are not recognized as the third light-emitting pattern, a first message associated with a failure to recognize the close-range light-emitting pattern is transmitted to the host device, Controlling the light emitting module so that the first light emitting pattern, which is a long-distance light emitting pattern, is radiated; An interaction system that controls the light-emitting module so that the first light-emitting pattern is emitted when the light-emitting areas captured by the camera are recognized as the first light-emitting pattern.

10. In paragraph 9, The above remote controller, If the light-emitting areas captured by the above camera are not recognized as the first light-emitting pattern, a second message associated with the failure to recognize the remote light-emitting pattern is transmitted to the host device, An interaction system that controls the light-emitting module so that the third light-emitting pattern, which is the short-range light-emitting pattern, is emitted.

11. In paragraph 9, If the light-emitting areas captured by the above camera are not recognized as the first light-emitting pattern, the light-emitting module is controlled so that the second light-emitting pattern is emitted, If the light-emitting areas captured by the above camera are not recognized as the second light-emitting pattern, a second message associated with the failure to recognize the remote light-emitting pattern is transmitted to the host device, An interaction system that controls the light-emitting module so that the third light-emitting pattern, which is the short-range light-emitting pattern, is emitted.

12. In paragraph 3, The above remote controller, Controlling the spacing between adjacent light-emitting areas of the bezel area to change based on the distance between the host device and the remote controller, Controlling the light-emitting areas whose intervals change so that they are photographed through the camera at a certain cycle or according to the detected motion, An interaction system, wherein the bezel area includes a first bezel area and a second bezel area formed at the upper and lower portions of the display device, and a third bezel area and a fourth bezel area formed at one side and the other side of the display device.

13. In paragraph 12, M light-emitting elements are arranged in the horizontal axis direction in each of the first bezel area and the second bezel area, In each of the third bezel area and the fourth bezel area, N light-emitting elements, which are less than M, are arranged in the vertical axis direction, The above remote controller, If the distance is greater than or equal to the threshold distance, the light emitting elements in the bezel area are controlled so that adjacent light emitting elements in the bezel area are spaced apart by a first interval to form a first light emitting pattern, An interaction system, characterized in that the number of light-emitting elements emitting light in the first bezel area and the second bezel area recognized within the field of view of the camera is less than k.

14. In paragraph 13, The above remote controller, If the distance is less than the threshold distance, the bezel area is controlled so that adjacent light-emitting elements within the bezel area are spaced apart by a second distance to form a third light-emitting pattern; The second interval of the third light-emitting pattern is formed narrower than the first interval of the first light-emitting pattern, The third positions of the light-emitting elements emitting light in the third light-emitting pattern are different from the first positions of the light-emitting elements emitting light in the first light-emitting pattern, An interaction system, characterized in that the number of light-emitting elements emitting light in the third bezel area and the fourth bezel area recognized within the field of view of the camera is k or more.

15. In paragraph 14, The above remote controller, Controlling the light-emitting elements of the bezel area so that the third light-emitting pattern, which is a close-range light-emitting pattern, is emitted in the first time period, Controlling the light-emitting elements of the bezel area so that the first light-emitting pattern, which is a long-distance light-emitting pattern, is emitted in a second time period following the first time period; An interaction system that controls light-emitting elements in the bezel area so that the close-range light-emitting pattern and the long-range light-emitting pattern are repeatedly formed.

16. In paragraph 14, The above remote controller, Controlling the light-emitting elements in the bezel area so that the third light-emitting pattern, which is a close-range light-emitting pattern, is emitted; If the light-emitting areas captured by the above camera are not recognized as the third light-emitting pattern, a first message associated with a failure to recognize the close-range light-emitting pattern is transmitted to the host device, Controlling the light-emitting elements in the bezel area so that the first light-emitting pattern, which is a long-distance light-emitting pattern, is radiated; An interaction system that controls light-emitting elements in the bezel area so that the first light-emitting pattern is emitted when light-emitting areas captured by the camera are recognized as the first light-emitting pattern.

17. In paragraph 12, The above remote controller, Controlling the light emitting elements of the bezel area so that the first light emitting pattern is emitted while the first group of light emitting elements within the bezel area are spaced apart by a first interval, and the third light emitting pattern is emitted while the second group of light emitting elements are spaced apart by a second interval, The first light emitting pattern is formed when the light emitting elements of the first group emit a first light signal of a first wavelength band, and the third light emitting pattern is formed when the light emitting elements of the second group emit a second light signal of a second wavelength band different from the first wavelength band. An interaction system, wherein the third positions of the light-emitting elements emitting light in the third light-emitting pattern are different from the first positions of the light-emitting elements emitting light in the first light-emitting pattern.

18. In paragraph 17, The above remote controller, When the light-emitting areas captured by the above camera are recognized as the first light-emitting pattern, the light-emitting elements of the bezel area are controlled so that the first light-emitting pattern, which is a long-distance light-emitting pattern, is emitted. When the light-emitting areas captured by the above camera are recognized as the third light-emitting pattern, the light-emitting elements of the bezel area are controlled so that the third light-emitting pattern, which is a close-range light-emitting pattern, is emitted. If the light-emitting areas captured by the above camera are not recognized by both the first light-emitting pattern and the third light-emitting pattern, a third message associated with a recognition failure is transmitted to the host device, An interaction system that controls light-emitting elements in the bezel area so that a second light-emitting pattern having an increased light-emitting area than the first light-emitting pattern is emitted.

Images