Display device and control method thereof

Abstract

A control method of a display device, according to an embodiment of the present invention, comprises the steps of: recognizing a specific voice; starting capturing an image by a camera, on the basis of the recognized specific voice; displaying image data corresponding to a virtual remote controller, on the basis of distance information of a first object; and performing control to execute a specific function in the image data, on the basis of a motion of a second object.

Description

Display device and control method thereof

[0001] Embodiments of the present invention relate to display devices. More specifically, they can be applied to televisions (TVs) and set-top boxes (STBs) controlled by remote controls, etc. Of course, the present invention is not necessarily limited to these devices.

[0002] When a TV and STB are connected, the user uses the STB's remote control to change channels, etc., and the TV's remote control can only use limited functions.

[0003] To solve this problem, integrated remote controls have been discussed recently, but there is still the inconvenience of requiring users to have an actual remote control.

[0004] There is a growing demand from users for new technologies that can replace conventional remote controls while simultaneously controlling display devices (e.g., TVs) with a new concept.

[0005] According to one embodiment of the present invention, it is an object to provide a technology that can control a display device such as a TV using only the user's voice and motion.

[0006] According to one embodiment of the present invention, it is an object to provide a technique for increasing the accuracy of controlling a display device regardless of the distance, position, etc. between a user and the display device.

[0007] A method for controlling a display device according to one embodiment of the present invention includes a step of recognizing a specific voice, a step of starting a camera to shoot based on the recognized specific voice, a step of displaying image data corresponding to a virtual remote control based on distance information of a first object, and a step of controlling the image data to execute a specific function based on motion of a second object.

[0008] Furthermore, for example, it is characterized in that the value of the size of the image data is proportional to the value of the distance information of the first object.

[0009] In addition, a display device according to another embodiment of the present invention is designed to display additional image data that can move according to the motion of the second object.

[0010] Meanwhile, for example, as the value of the distance information of the first object increases, the change in the position of the additional image data moving according to the motion of the second object increases.

[0011] The first object and the second object may be the same or different. The first object corresponds to, for example, the user's eye, and the second object corresponds to, for example, the user's finger.

[0012] And, a display device according to one embodiment of the present invention includes a microphone that receives a specific voice, a camera that starts shooting based on the received specific voice, a display that displays image data corresponding to a virtual remote control based on distance information of a first object, and a controller that controls the execution of a specific function of the image data based on the motion of a second object.

[0013] It is also within the scope of the present invention for a third party to implement a computer-readable medium (e.g., an application, memory, software, etc.) recording a program for performing any of the above-described methods and various embodiments described in the specification.

[0014] According to one of the embodiments of the present invention, there is a technical effect of providing a technology that can control a display device such as a TV using only the user's voice and motion.

[0015] According to one embodiment of the present invention, a technique is provided for increasing the accuracy of controlling a display device regardless of the distance, position, etc. between a user and the display device.

[0016] In addition to the technical effects described above, it is self-evident that effects that can be inferred by a person skilled in the art from the entire specification should also be taken into consideration.

[0017] FIG. 1 is a drawing schematically illustrating the appearance of a display device according to one of the embodiments of the present invention.

[0018] FIG. 2 illustrates at least one object captured by a camera of a display device according to one embodiment of the present invention.

[0019] FIG. 3 illustrates image data corresponding to a virtual remote control generated by a display device according to one of the embodiments of the present invention.

[0020] FIG. 4 illustrates a process in which a display device according to one embodiment of the present invention outputs image data corresponding to a virtual remote control under specific conditions.

[0021] FIG. 5 illustrates a process in which a display device according to one embodiment of the present invention changes the size of image data corresponding to a virtual remote control according to the distance from the object.

[0022] FIG. 6 illustrates a process in which a display device according to one embodiment of the present invention changes the position of image data corresponding to a virtual remote control according to the shape of an object.

[0023] FIG. 7 is a flowchart illustrating a method for controlling a display device according to one of the embodiments of the present invention.

[0024] And, FIG. 8 is a block diagram showing the main components of a display device according to one of the embodiments of the present invention.

[0025] Hereinafter, embodiments disclosed in this specification will be described in detail with reference to the attached drawings. Regardless of the drawing numbers, identical or similar components will be assigned the same reference numbers, and redundant descriptions thereof will be omitted. The suffixes "module" and "part" used for components in the following description are assigned or used interchangeably solely for the convenience of writing the specification, and do not in themselves have distinct meanings or roles.

[0026] In addition, when describing the embodiments disclosed in this specification, if it is determined that a detailed description of a related known technology may obscure the gist of the embodiments disclosed in this specification, the detailed description thereof will be omitted. In addition, the attached drawings are only intended to facilitate easy understanding of the embodiments disclosed in this specification, and the technical ideas disclosed in this specification are not limited by the attached drawings, and should be understood to include all modifications, equivalents, and substitutes included within the spirit and technical scope of the present invention.

[0027] Terms that include ordinal numbers, such as "first," "second," etc., may be used to describe various components, but these components are not limited by these terms. These terms are used solely to distinguish one component from another. When a component is referred to as being "connected" or "connected" to another component, it should be understood that while it may be directly connected or connected to that other component, there may also be other components present in between.

[0028] On the other hand, when it is said that a component is "directly connected" or "directly connected" to another component, it should be understood that there are no other components in between.

[0029] Singular expressions include plural expressions unless the context clearly indicates otherwise.

[0030] In this application, terms such as “include” or “have” are intended to specify the presence of a feature, number, step, operation, component, part or combination thereof described in the specification, but should be understood not to exclude in advance the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts or combinations thereof.

[0031] FIG. 1 is a drawing schematically illustrating the appearance of a display device according to one of the embodiments of the present invention.

[0032] As illustrated in FIG. 1, a display device (100) (e.g., TV, etc.) according to one embodiment of the present invention enables all operations of a remote control to be performed without a remote control.

[0033] To implement this, the display device (100) is equipped with a camera (110), and after recognizing the user's hand with the camera (110), displays a graphic image related to the remote control.

[0034] However, since keeping the camera (110) always on can have a negative impact on the user's privacy, it is important to design the camera (110) to operate only under certain trigger conditions (e.g., receiving a specific voice command, etc.).

[0035] To implement this, the display device (100) additionally includes a microphone (not shown).

[0036] Meanwhile, a method for recognizing a specific gesture of a user using the camera (110) illustrated in FIG. 1 will be described below with reference to FIG. 2.

[0037] FIG. 2 illustrates at least one object captured by a camera of a display device according to one embodiment of the present invention.

[0038] In an image captured by a camera directly or indirectly installed in a display device according to one embodiment of the present invention, there are a number of objects, as shown in FIG. 2.

[0039] However, if the display device recognizes that only one finger (211) is extended from the user's hand (200), a virtual matrix (210) of, for example, 3X3, 3X4, 4X3, or 4X4 is generated around the finger (211). Of course, the above-mentioned numbers are only examples, and it is within the scope of the present invention to arbitrarily adjust them according to the number of remote control functions, etc.

[0040] Furthermore, a display device according to one embodiment of the present invention regards the gesture shown in FIG. 2 as a user's intention to use a remote control function, and generates a remote control-related graphic image.

[0041] FIG. 3 illustrates image data corresponding to a virtual remote control generated by a display device according to one of the embodiments of the present invention.

[0042] A display device according to one embodiment of the present invention generates image data (300) corresponding to a virtual remote control as shown in FIG. 3 based on the virtual matrix (210) shown in FIG. 2.

[0043] However, as mentioned above, always keeping the camera on to recognize the user's gesture as shown in Fig. 2 has privacy issues, so a solution to address this will be described below with reference to Fig. 4.

[0044] FIG. 4 illustrates a process in which a display device according to one embodiment of the present invention outputs image data corresponding to a virtual remote control under specific conditions.

[0045] A display device (410) according to one embodiment of the present invention recognizes a specific voice (e.g., “gesture mode” or “execute gesture mode”).

[0046] And, the display device (410) starts shooting with the camera based on the recognized specific voice.

[0047] Furthermore, the display device (410) displays image data (411) corresponding to the virtual remote control based on distance information of the first object (e.g., the user's eyes, etc.).

[0048] Additionally, the display device (410) controls the execution of a specific function of the image data (411) based on the motion of the second object (e.g., the user's finger, etc.).

[0049] Meanwhile, the display device (410) is designed so that the size value of the image data (411) is proportional to the value of the distance information of the first object. A more specific embodiment related to this will be described later with reference to FIG. 5.

[0050] Furthermore, the display device (410) displays additional image data that can move according to the motion of the second object. The additional image data here refers to a pointer or a highlighted box portion within the image data (411) illustrated in FIG. 4.

[0051] In addition, it is a feature of the present invention to design the position change of additional image data moving according to the motion of the second object to increase as the value of the distance information of the first object increases.

[0052] When a user, an example of the first object, is recognized from a distance from the display device (410), and the size of the finger, an example of the second object, is small, it is necessary to design the pointer to move more even if the finger moves only a little.

[0053] That is, according to one embodiment of the present invention, a TV, which is an example of a display device, recognizes a user's hand using a camera or the like and displays a gesture UI (No. 411 shown in FIG. 4).

[0054] When the finger moves in the desired direction from the recognized point, the corresponding button in the image data (411) of the virtual remote control is highlighted.

[0055] In the highlighted state, when the display device (410) detects a bending motion of a finger (i.e., a clicking motion) using a camera or the like, the function corresponding to the highlighted button is executed.

[0056] It is also possible to design the image data (411) corresponding to the virtual remote control to enable hot key matching, such as home, back, and settings, for the remaining blocks, excluding the directional keys.

[0057] As described above, since the finger recognition start and end process is performed only when a specific voice such as “start gesture mode” or “end gesture mode” is recognized through the microphone of the display device (410), there is no need to always keep the camera on, and privacy issues are also expected to be improved.

[0058] Furthermore, the display device (410) according to one embodiment of the present invention is designed to execute different functions depending on whether the hand recognized using a camera or the like is the right or left hand.

[0059] For example, when the display device (410) recognizes the right hand, image data (411) corresponding to the virtual remote control is displayed on the right side of the screen of the display device (410) (as shown in FIG. 4).

[0060] On the other hand, if the display device (410) recognizes the left hand, image data (411) corresponding to the virtual remote control is displayed on the left side of the screen of the display device (410). A more specific embodiment related to this will be described in more detail later in FIG. 6.

[0061] Furthermore, the display device (410) recognizes the distance from the user and adjusts the size of the image data (411) corresponding to the virtual remote control according to the distance. A more specific embodiment related to this will be described in more detail later in FIG. 5.

[0062] In addition, if a user's motion of extending and bending a finger (“clicking motion”) is recognized for a preset period of time (e.g., 3 seconds) or longer through a camera or the like of the display device (410), the display device (410) is designed to regard it as a long click on a highlighted block (button) in the image data (411) corresponding to the virtual remote control.

[0063] Furthermore, when a user's motion of extending and bending a finger (“clicking motion”) is repeatedly recognized through a camera or the like of the display device (410), the display device (410) is designed to regard it as a repeated click on a highlighted block (button) in the image data (411) corresponding to the virtual remote control.

[0064] And, when the user performs a zoom-in / out motion with two fingers (for example, a motion of spreading and narrowing two fingers) through the camera of the display device (410), the size of the image data (411) corresponding to the virtual remote control is enlarged or reduced.

[0065] FIG. 5 illustrates a process in which a display device according to one embodiment of the present invention changes the size of image data corresponding to a virtual remote control according to the distance from the object.

[0066] A technology is needed to change the size of image data corresponding to a virtual remote control depending on the distance between the display device and the user.

[0067] For example, as shown in (a) of FIG. 5, when the distance between a user (500) who has uttered a specific voice (e.g., gesture mode or gesture mode execution) and the display device (510) is within 3 m, image data (520) corresponding to the initially set virtual remote control is displayed.

[0068] On the other hand, as shown in (b) of FIG. 5, when the distance between the user (501) who has uttered a specific voice (e.g., gesture mode or gesture mode execution) and the display device (511) is 6 m or more, an image (521) whose size is twice that of the image data corresponding to the initially set virtual remote control is displayed.

[0069] FIG. 6 illustrates a process in which a display device according to one embodiment of the present invention changes the position of image data corresponding to a virtual remote control according to the shape of an object.

[0070] First, (a) of Fig. 6 assumes that the display device (600) recognizes the left hand (not shown). In this case, image data (610) corresponding to the virtual remote control is designed to be displayed on the left side of the screen of the display device (600).

[0071] On the other hand, (b) of Fig. 6 assumes that the display device (601) recognizes the right hand (not shown). In this case, image data (611) corresponding to the virtual remote control is designed to be displayed on the right side of the screen of the display device (601).

[0072] FIG. 7 is a flowchart illustrating a method for controlling a display device according to one of the embodiments of the present invention.

[0073] A display device (e.g., TV, etc.) according to one embodiment of the present invention recognizes a specific voice (e.g., “gesture mode”, etc.) (S710), and starts shooting with a camera based on the recognized specific voice (S720).

[0074] Furthermore, the display device displays image data corresponding to the virtual remote control based on distance information of the first object (e.g., the user's eyes, etc.) (S730).

[0075] And, the display device is designed to execute a specific function of the image data based on the motion of a second object (e.g., a user's finger) (S740).

[0076] Additionally, the size of the image data is designed to be proportional to the distance information value of the first object. Therefore, there is a technical effect that allows a user to clearly identify and control the image of the virtual remote control from a distance.

[0077] In addition, additional image data that can move according to the motion of the second object is displayed. In particular, as the distance information of the first object increases, the positional change of the additional image data that moves according to the motion of the second object increases. Therefore, the technical effect of quickly changing the highlighted portion of the virtual remote control is expected even with a slight movement of the user's finger from a distance.

[0078] And, Fig. 8 is a block diagram illustrating the main components of a display device according to one of the embodiments of the present invention. Fig. 8 can be supplemented with reference to the previously described embodiments.

[0079] First, as illustrated in FIG. 8, a display device (800) according to one embodiment of the present invention includes a microphone (810), a camera (820), a display (830), and a controller (840).

[0080] The microphone (810) receives a specific voice, and the camera (820) starts shooting based on the received specific voice.

[0081] The display (830) displays image data corresponding to the virtual remote control based on distance information of the first object.

[0082] And, the controller (840) executes a specific function of the image data based on the motion of the second object.

[0083] The present invention described above can be implemented as computer-readable code on a medium having a program recorded thereon. Computer-readable media include all types of recording devices that store data that can be read by a computer system. Examples of computer-readable media include applications, hard disk drives (HDDs), solid-state disks (SSDs), silicon disk drives (SDDs), ROMs, RAMs, CD-ROMs, magnetic tapes, floppy disks, optical data storage devices, and the like, and also include media implemented in the form of carrier waves (e.g., transmission via the Internet).

[0084] Although the data processing device and method according to the embodiments of the present invention have been described in specific embodiments, these are merely examples, and the present invention is not limited thereto, and should be construed to have the broadest scope in accordance with the basic idea disclosed in this specification. Those skilled in the art may implement embodiments not specified by combining or replacing the disclosed embodiments, but this also does not exceed the scope of the present invention. In addition, those skilled in the art may easily modify or alter the disclosed embodiments based on this specification, and it is clear that such modifications or alterations also fall within the scope of the present invention.

[0085] As for various embodiments for implementing the present invention, those skilled in the art can make modifications as fully described in the previous table of contents.

[0086] The present invention is applicable to various display devices such as TVs, and thus its industrial applicability is recognized.

Claims

1. In a method for controlling a display device, Step of recognizing a specific voice; A step of starting the camera's shooting based on the above-mentioned recognized specific voice; A step of displaying image data corresponding to a virtual remote control based on distance information of a first object; and A step of controlling the execution of a specific function of the image data based on the motion of the second object. A method for controlling a display device, characterized in that it includes:

2. In paragraph 1, A control method for a display device, characterized in that the size value of the image data is proportional to the value of the distance information of the first object.

3. In paragraph 1, A step of displaying additional image data that can move according to the motion of the second object. A method for controlling a display device, characterized in that it further includes:

4. In paragraph 3, A control method for a display device, characterized in that as the value of the distance information of the first object increases, the change in the position of additional image data moving according to the motion of the second object increases.

5. In paragraph 1, A control method for a display device, characterized in that the first object corresponds to the user's eye and the second object corresponds to the user's finger.

6. In the display device, A microphone that receives specific voices; A camera that starts shooting based on the specific voice received above; A display that displays image data corresponding to a virtual remote control based on distance information of a first object; and A controller that controls the execution of a specific function of the image data based on the motion of the second object. A display device characterized by including:

7. In paragraph 6, A display device characterized in that the size value of the image data is proportional to the value of the distance information of the first object.

8. In paragraph 6, The above display is, A display device characterized in that it displays additional image data that can move according to the motion of the second object.

9. In paragraph 8, A display device characterized in that as the value of the distance information of the first object increases, the change in the position of the additional image data moving according to the motion of the second object increases.

10. In paragraph 6, A display device, characterized in that the first object corresponds to the user's eye and the second object corresponds to the user's finger.

Images