REMOTE CONTROL SYSTEM.

Abstract

The present invention relates to a remote control system and, more specifically, to a remote control system capable of executing or controlling operations or functions by means of gestures. The gesture-operated remote control system comprises: an input unit and an output unit; a storage unit for storing gesture information that must match an input gesture; and a control unit electrically connected to at least one of the input units, the output unit, and the storage unit to recognize the gesture and control the system, wherein the control unit recognizes a gesture input from the input unit, matches the recognized gesture with the gesture information stored in the storage unit, executes an operation corresponding to the matching gesture information, and sends it to the output unit.

Description

The present invention relates to a remote control system and, more particularly, to a remote control system capable of executing or controlling an operation or function through a gesture. BACKGROUND OF THE INVENTION Unless otherwise stated herein, the material described in this section is not prior art to the claims of this application, and inclusion in this section is not an admission that it is prior art. With the recent development of technology, a vast number of electronic devices with diverse functions are emerging in a wide range of fields, too numerous to list. These include smart TVs, smartphones, AR and VR devices, interactive game consoles, smart home appliances, smart home systems, and intelligent and electrified automobiles. Furthermore, these devices offer a wide variety of functions. However, the remote control methods for conveniently operating these functions remotely have not developed significantly compared to the sheer number of electronic devices and the considerable increase in their capabilities. The remote control method that is still most commonly used is one in which the remote control has a button corresponding to each function, and when the button for the desired function is pressed, the electronic device receives the corresponding signal and the control unit recognizes it and executes the corresponding function. The button-type remote control method provides the best convenience when the remote control has as many buttons as the number of functions provided, and the user can easily select and press the corresponding function button. However, this was possible long ago when it only had a small number of functions, but with the rapid pace of technological development, a button-type remote control now requires many buttons, making it difficult to carry as it grows large and heavy. Furthermore, it becomes difficult for the user to find and execute a desired function among the numerous buttons, resulting in a loss of convenience. Judging by the current situation, the karaoke machine remote control seems to be the remote control with the most buttons that the user understands and can commercially use. So, to overcome this, what was introduced from the method of using a button-type remote control is to introduce four-way buttons and an OK button, and to display the functions to be executed on the screen, and select and execute them. For example, in the case of a current smart TV, there is no button that directly corresponds to selecting and running a specific application from among several applications on the screen, playing or stopping a specific video, or making a purchase on the home shopping TV; use the button to find the function displayed on the screen, select it, and press the confirmation button to run it. For example, on a modern smart TV, when a user selects and launches a specific app from among several on the screen, plays or pauses a video, or makes a purchase in the home TV store, there isn't a dedicated button for that function. Instead, they have to find and select the function displayed on the screen using the four-way buttons and then press the confirmation button to execute it. However, user convenience is already reduced by repeatedly pressing the four-way button and finally selecting the confirmation button. Furthermore, as more functions are added in the future, users will increasingly have to press the four-way button and the confirmation button to find the function on the screen, thus further diminishing user convenience. Furthermore, recently, instead of a button-type remote control, methods such as voice recognition or a 3D spatial mouse have been tested. However, in the case of voice recognition, the recognition rate and responsiveness are still not ideal, and it is also inconvenient for the user to command and execute it by voice. And, in the case of the 3D spatial mouse, it has not been widely commercialized because it does not exhibit the responsiveness of a PC mouse in terms of quick and precise selection and execution. Therefore, there is a need for a new remote control system with greater user convenience, a fast response capability, and a convenient and easy remote control of many functions that can be performed through technological development in the future and in the present. Furthermore, the present invention is not limited to the technical problems described above, and it is obvious that another technical problem may arise from the following description. nnza Ln / zznz / E / YiAi In this regard, Korean patent publication No. 10-2010-0131213 discloses a gesture-based remote control system. The gesture-based remote control system includes a camera module 300, an image recognition module 100, a wireless transmitter 400, and a main electronic control device 200. The image recognition module 100 is electrically connected to the camera module 300. The wireless transmitter 400 is electrically connected to the image recognition module 100. The main control electronics 200 is detachably connected to the image recognition module 100. The main control electronics 200 includes a monitor 202. The motion control command is obtained by recognizing the user's image in the camera module 300, which contains the image recognition module 100. The key control command is obtained from the motion control command and key code information by the image recognition module 100. The key control command is transmitted to the wireless transmitter 400 by the image recognition module 100.The key control command is obtained from the motion control command and key code information by the image recognition module 100. The key control command is transmitted to a target electronic device (nnza Ln / zznz / E / YiAi) by the wireless transmitter to control the target electronic device. Prior art patent document. Publication of Korean patent No. 10-2010-0131213 (2010.12.15.) . OBJECTIVE OF THE INVENTION An object of the present invention is to provide a new remote control system with greater user convenience and rapid response capability, while conveniently and easily executing numerous functions of electronic devices remotely using gestures. BRIEF DESCRIPTION OF THE INVENTION The invention relates to a gesture-operated remote control system according to one embodiment of the present invention, comprising: an input unit and an output unit; a storage unit for storing gesture information that must match an input gesture; and a control unit electrically connected to at least one of the input units, the output unit, and the storage unit to recognize the gesture and control the system, wherein the control unit recognizes a gesture input from the input unit, matches the recognized gesture with the gesture information stored in the storage unit, executes an operation corresponding to the matching gesture information, and outputs to the output unit. According to a preferred feature of the present invention, the control unit emits information about the gestures that the user must input to the output unit. According to a preferred feature of the present invention, the control unit designates each gesture graphic for output of graphic objects for execution through the output unit, and stores information about the designated graphic objects and gesture graphics as gesture information to match the gesture input by the input unit to the storage unit; wherein the graphic objects include each menu item, an application launch icon, a specified thing or person in a streaming video, a string of characters, a portion of a slider bar, and a specified thing or person in an image or photo output through the output unit, and addresses all output objects through the output unit; and with respect to the gesture graphic, a gesture graphic consists of only one or more gestures or a combination of one or more numbers and one or more gestures. According to a preferred feature of the present invention, the control unit designates each gesture graphic for execution functions that are not designated using the output of graphic objects for execution purposes through the output unit, and stores information about the designated execution functions and gesture graphics as gesture information to match the gesture input from the input unit to the storage unit. According to a preferred feature of the present invention, when a specific gesture is entered, the control unit outputs all gesture graphics designated for execution purpose graphic objects to the output unit. According to a preferred feature of the present invention, characterized in that when a specific gesture is introduced, among the gesture graphics designated for execution functions that are not designated using graphic objects, the control unit generates a mapping list for the currently available execution functions and the gesture graphics designated for them to the output unit. According to a preferred feature of the present invention, the control unit outputs all designated gesture graphics for graphic objects with execution purposes to the output unit along with the graphic objects to the output unit. According to a preferred feature of the present invention, the control unit installs, deletes, modifies, and updates gesture information in the output of graphic objects for execution purposes through the output unit and the gesture graphics designated therefor, and gesture information in non-designated execution functions using the graphic objects and the gesture graphics designated therefor in real time through a server and a network, and stores them in the storage unit. According to a preferred feature of the present invention, the control unit emits a trajectory of each input gesture operation to the output unit. According to a preferred feature of the present invention, the gesture information stored in the storage unit includes an application gesture library that each application has and that is used to match recognized gestures when an application function is executed, and a system gesture library that the system has and that is used to match recognized gestures when a system function is executed, and wherein each of the gesture libraries includes a visible gesture library unit that stores information about the output of execution-purpose graphical objects through the output unit and the gesture graphics designated therefor as the gesture information,and a hidden gesture library unit that stores information about execution functions that are not designated using the output of execution-purpose graphical objects through the output unit and gesture graphics designated for it as gesture information. According to a preferred feature of the present invention, when an application is installed, the control unit registers the output of the application's run icon through the output unit and the gesture graphic designated therefor in the system's gesture library, wherein the gesture graphic designated for the application's run icon uses one designated by the application producer, one recommended by the control unit, or one desired by the user.According to a preferred feature of the present invention, the control unit uses the selected application's gesture library to match recognized gestures when an application is currently selected from one or more running applications, and uses the system's gesture library to match recognized gestures when none of the applications are running, or none of the applications are selected even though one or more applications are running. According to a preferred feature of the present invention, the control unit installs, deletes, modifies, updates the application gesture library and the system gesture library in real time via a server and a network, and stores them on the storage unit. According to a preferred feature of the present invention, when a specific gesture is introduced into the input unit and it is determined that the gesture is a multi-gesture input signal, the control unit emits a signal indicating the multi-gesture input status and a signal waiting for gesture input to the output unit, and then repeats a process of receiving input gestures one by one and continuously storing them in the storage unit, wherein when it is determined that the input gesture is a multi-gesture input termination signal, the control unit matches a plurality of gestures stored so far in the storage unit with the stored gesture information and executes an operation. According to a preferred feature of the present invention, when a gesture chart is designated with multiple gestures having two or more times a specific gesture repeated, when the specific gesture is sent to the output unit, the control unit repeatedly emits as many gestures as the gesture number to the output unit, or generates only one gesture and generates the repeated gesture number as a subscript. According to a preferred feature of the present invention, comprising a transmitter located in the input unit and transmitting power-on and power-off signals to the control unit, wherein the control unit sends all gesture graphics designated for execution-purpose graphic objects in the output unit to the output unit when the transmitter in the input unit transmits a power-on signal without a gesture, and maintains the state in which the gesture graphics are sent to the output unit when the transmitter continuously maintains a power-on signal without a gesture, and causes the output gesture graphics to disappear from the output unit when the transmitter transmits a power-off signal. According to a preferred feature of the present invention, comprising a transmitter located in the input unit and transmitting on and off signals to the control unit, characterized in that when the transmitter of the input unit transmits an on and off signal twice within a predetermined time without a gesture, the control unit emits a signal indicating the input status of multiple gestures and a signal waiting for input gestures to the output unit, and then repeats a process of receiving input gestures one by one and storing them in the storage unit continuously, wherein when the transmitter of the input unit transmits an on and off signal two more times within a predetermined time without a gesture, the control unit matches the plurality of gestures stored so far in the storage unit with the stored gesture information, and executes an operation. According to a preferred feature of the present invention, comprising a transmitter located in the input unit and transmitting on and off signals to the control unit, characterized in that when a graphic gesture is designated with a Multi-(on)On (off)Off gesture having a specific gesture transmitted together with on and off signals two or more times within a predetermined time, the control unit outputs the number of times that the input of on and off signals within a predetermined time prior to the specific gesture as a number when sent to the output unit. nnza Ln / zznz / E / YiAi According to a preferred feature of the present invention, comprising a transmitter located in the input unit and transmitting on and off signals to the control unit, wherein when a gesture is input to the input unit with on and off signals two or more times within a predetermined time, the control unit matches the gesture and the number of times the input on and off signals occur within a predetermined time with the gesture information and executes an operation. nnza Ln / zznz / E / YiAi ADVANTAGEOUS EFFECTS According to one embodiment of the present invention, unlike a conventional remote control system, in order to allow for the intuitive, easy, and rapid selection and execution of a desired function by a user, the present invention assigns gestures to graphic objects such as application launch icons to execute a function on the screen, playback buttons on a video player, and the purchase button on a home shopping service, and the like. When the user performs a specific gesture, the gestures assigned to the graphic objects as exemplified above are displayed on the screen. In this way, the user can identify which gesture corresponds to the graphic object on the screen to be executed. And when the user performs a gesture among these, the function assigned to the corresponding graphic object is executed.Through this, the present invention has the advantage of being able to select and execute a desired function on the screen in a more intuitive, easy, and faster way than a PC mouse. Furthermore, since the installation, removal, addition, updating, etc. of the designated gestures through the server and the network by the application producer, content provider, system producer, etc. is possible in real time, the present invention has the advantage that a new gesture can be assigned to a new function at any time, unlike the existing remote control system where functions are fixed once manufactured and new functions cannot be added. Furthermore, the present invention has the advantage that user comfort is not lost even with the minimum number of buttons because numerous gestures can be used flexibly in every situation when a transmitter is configured with buttons. The effects of the present invention are not limited to the above effects, and should be understood to include all effects that can be inferred from the configuration of the invention described in the detailed description or claims of the present invention. nnzo Ln / zznz / E / YiAi BRIEF DESCRIPTION OF THE FIGURES FIG. 1 is a schematic configuration diagram of a remote control system according to an embodiment of the present invention. FIG. 2 is a schematic configuration diagram showing that the input unit is composed of a transmitter and a receiver in the remote control system according to an embodiment of the present invention. FIG. 3 is a diagram showing information about a gesture to be entered by a user that is displayed on a screen in the remote control system according to an embodiment of the present invention. FIG. 4 is a diagram showing that gesture graphics are designated and displayed on graphic objects shown on a screen in the remote control system according to an embodiment of the present invention. FIG. 5 is a mapping list of gesture graphics designated for execution functions that are not designated using graphic objects on the screen in the remote control system according to an embodiment of the present invention. FIG. 6 is a schematic configuration diagram for installing, deleting, modifying, and updating gesture information through a server and network in a remote control system according to an embodiment of the present invention. FIG. 7 is a schematic configuration diagram showing that the application and system each have a gesture library, and each gesture library is composed of a visible library and a hidden library in a remote control system according to an embodiment of the present invention. FIG. 8 is a schematic configuration diagram showing the situation in which each application gesture library and system gesture library are installed, removed, modified, and updated through each server and network in a remote control system according to an embodiment of the present invention. nnzo Ln / zznz / E / YiAi DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT OF THE INVENTION The configuration, operation, and effect of the remote control system will now be described according to a preferred modality, with reference to the accompanying drawings. For reference, in the following drawings, each component is either omitted or illustrated schematically for convenience and clarity, and the size of each component does not reflect its actual size. Furthermore, the same part numbers refer to the same components throughout this specification, and part numbers for the same components will be omitted in individual drawings. FIG. 1 is a conceptual configuration diagram of a gesture-operated remote control system 100 according to a modality of the present invention. The remote control system includes an input unit 110 and an output unit 120; a storage unit 130 for storing gesture information that must match an input gesture; and a control unit 140 electrically connected to at least one of the input units 110, the output unit 120, and the storage unit 130 for recognizing the gesture and controlling the system, wherein the control unit 140 recognizes a gesture input by the input unit 110, compares the recognized gesture with the gesture information stored in the storage unit 130, executes an operation corresponding to the matching gesture information, and sends it to the output unit 120. The input unit 110 can be configured as a camera to receive gesture input from the camera-based remote control system 100 according to one embodiment of the present invention. Here, when a person makes a gesture using a hand movement, the camera captures the hand gesture and receives the gesture input. In the remote control system 100 according to another modality, as shown in FIG. 2, the input unit nnza Ln / zznz / E / YiAi 110 may include a transmitter 111 to transmit a gesture and a receiver 112 to receive a transmitted signal. The 120 output unit is an output device after performing an operation corresponding to the matching gesture information that will be described later, and any configuration is possible as long as it includes a device capable of outputting an image or video. The 130 storage unit is a device for storing gesture information that must match an input gesture. And, control unit 140 serves to control the general process that recognizes a gesture input to input unit 110, matches the recognized gesture with the gesture information stored in storage unit 130, executes the corresponding operation and outputs to output unit 120. According to a preferred feature of the present invention, the control unit 140 emits gesture information that must be entered by the user into the output unit 120. The biggest problem with the existing gesture-based remote control system is that the user must memorize all the gestures to execute the desired functions. In the existing gesture-based remote control system, as the number of functions increases, the number of gestures that must be remembered inevitably increases. Although the use of gestures is intended for user convenience, this convenience decreases as the number of functions increases. Therefore, the existing gesture-based remote control system has only been used to the extent that a gesture can be assigned and used for only a small number of specific functions in an auxiliary capacity. In response to this problem, the present invention proposes a solution by displaying information on the screen about gestures to execute the desired functions without the user having to memorize the gestures. Referring to the example in FIG. 3, there are many applications on the screen, and 1, 2, 3, k, B, and C are designated and displayed by a separate Ln / zznz / E / YiAi for each application. These 1, 2, 3, c, A, B and C are examples of different gestures designated to run each application. The system of the present invention provides gestures to execute a function desired by the user on the screen, and when the user makes one of the gestures, the function corresponding to the gesture is executed. For example, to run app2 310 in FIG. 3, if the user makes gesture2 320, app2 310 is executed. According to a preferred feature of the present invention, the control unit 140 designates each gesture graphic for output of graphic objects for execution purposes through the output unit 120, and stores information about the designated graphic objects and gesture graphics as gesture information to be matched with gesture input by the input unit 110 to the storage unit 130; wherein the graphic objects include each menu item, an application launch icon, a specific thing or person in a streaming video, a string of characters, a portion of a slide bar, and a specific thing or person in an image or photo output through the output unit 120, and addresses all output objects through the output unit 120; and with respect to the gesture graphic, a gesture graphic consists of only one or more gestures or a combination of one or more numbers and one or more gestures. As in the example in FIG. 4, numerous graphic objects such as images, application launch icons, text, slide bars, directories, and people or things in a video are displayed on the screen. The execution functions corresponding to specific graphic objects are determined among these graphic objects. In the present invention, system producers, application producers, or content providers can designate a desired gesture for each of the graphic objects that executes these functions. In Figure 4, numbers, Korean letters, English letters, symbols, or a combination thereof are shown as examples of their respective gestures next to each graphic object. The user can execute a function corresponding to the graphic object by performing the corresponding gesture. Another major problem with existing remote control systems is the difficulty in selecting and executing a desired function from the numerous functions displayed on the screen, as illustrated in Figure 4. As mentioned earlier, the most common type of button-based remote control is slow and inconvenient for users because it requires navigating through a four-way button and a confirmation button to find and execute the desired function. Furthermore, naming and calling each of the numerous functions on the screen is difficult, especially with voice recognition, and the accuracy and responsiveness are still unsatisfactory, making speech cumbersome. Additionally, executing a function on the screen by moving the pointer, similar to a 3D mouse, also lacks the responsiveness users expect. In this respect, in the present invention, as described above, when a user intuitively sees a gesture designated for a graphic object displayed on the screen and performs the gesture, the function is executed immediately. This is faster and more convenient than the method of moving the mouse pointer to the desired graphic object on the PC mouse system and double-clicking to execute it. Even if the user uses it frequently, the gesture for a specific graphic object is automatically remembered. In this case, since the user executes the function by making the memorized gesture without having to look at the screen to find the gesture designated for the graphic object on the screen, the desired function can be executed more quickly and conveniently, and the user's convenience can be greatly increased. On the other hand, here, as in the example in FIG. 4, the gestures designated for each graphic object are not simply designated as gesture 410 and 420 per graphic object. The multiple gestures 430 and 450 can be designated as required. And, in the case of having a transmitter that transmits on / off signals to the control unit 140 to the input unit 110, which will be described later, a Multi-on / off gesture 440 can be designated in which a gesture and more than double the on / off signal can be designated within a predetermined time nnza Ln / zznz / E / YiAi. If there are many graphical objects on the screen, a large number of gestures will be required. As a result, there will be a limit to the number of simple, easy gestures, necessitating many complex ones. However, the gesture-based remote control system is designed to allow the user to easily and conveniently execute a function with a single gesture. Due to the complexity of these gestures, it may be difficult for the user to perform a gesture, or a situation may arise where the gesture recognition rate of the control unit 140 decreases. In this regard, the present invention provides a combination of a plurality of simple and easy gestures or a combination of Multi-on / Off signals and gestures using the multiple gestures and the Multi-on / Off gesture as described above, so that many functions can be covered with a small number of gestures. And, as shown in FIG. 4, the gesture chart to graphically display a designated gesture on the screen is shown as either one or more gestures 410, 420 and 430 or a combination of one or more numbers and one or more gestures 440 and 450, so that multiple gestures and Multi-on / off gestures can be displayed. According to a preferred feature of the present invention, the control unit 140 designates each gesture graphic for execution functions not designated using the execution-purpose graphic object output through the output unit 120, and stores information about the designated execution functions and gesture graphics as gesture information to be matched with gesture input by the input unit to the storage unit 130. As described earlier, it's possible to launch an application, select a menu, and move the slider up or down using the designated gesture graphic on the graphical object displayed on the screen. However, operating the system using only the graphical objects displayed on the screen and the designated gesture graphic for all execution functions can be cumbersome, requiring more graphical objects than necessary on the screen, and may even reduce user convenience. In this sense, the present invention designates each gesture graphic for the necessary execution functions, which do not use the graphic objects on the screen, and stores information about the designated execution functions and gesture graphics as gesture information in storage unit 130. As in the example in FIG. 5, the present invention can directly designate the gesture graphics 530 and 540 without a graphic object for function 510 for raising the sound or function 520 for raising the TV channel. According to a preferred feature of the present invention, when a specific gesture is entered, the control unit 140 outputs all designated gesture graphics for execution purposes to the output unit 120. The user cannot memorize all the gesture graphics assigned to every execution-purpose graphic object displayed on the screen. However, if gesture graphics are continuously displayed simply because the user cannot remember them, the screen will become cluttered with gesture graphics and graphic objects, making it appear complicated. Eventually, the user's concentration and ease of use will decrease. In this respect, the present invention displays all the designated gesture graphics for the execution-purpose graphic objects on the current screen only when the user performs a specific gesture. That is, user convenience is improved by allowing the user to see the designated gesture graphics for the graphic objects on the screen only when desired. According to a preferred feature of the present invention, when a specific gesture is introduced, among the gesture graphics designated for execution functions that are not designated using graphic objects, the control unit generates a mapping list for the currently available execution functions and the gesture graphics designated therefor to the output unit 120. Even if the user searches for graphical objects to execute the function currently displayed on the screen, the desired function might not be among them. For example, the sound control function is a necessary function, but for this function, the application developer might not want to place a graphical object on the screen and instead designate the gesture graphic to execute it. In this case, when the user performs a specific gesture, as in the example in FIG. 5, the control unit 140 displays a mapping list on the screen for all currently available execution functions and gesture graphics. This list includes all gesture graphics designated for execution functions that are not designated using graphical objects. For example, the user can view the mapping list, verify the desired function and gesture, and then perform the right gesture 530 to increase the volume. According to a preferred feature of the present invention, the control unit 140 outputs all gesture graphics designated for execution purpose graphic objects to the output unit 120 together with the graphic objects to the output unit 120. As display technology develops, many electronic devices are manufactured using the largest possible screen. In this situation, even if all the graphical objects are displayed on a relatively large screen in a specific device that doesn't have many execution functions, there can still be ample screen space available. In this case, it doesn't seem complicated even if all the graphical objects for executing functions and the gesture graphics designated for them are displayed together on the screen. This function displays gesture graphics on the screen even if the user does not perform a gesture. According to a preferred feature of the present invention, the control unit 140 installs, deletes, modifies, and updates gesture information in the execution-purpose graphical output objects via the output unit 120 and the gesture graphics designated therefor, and gesture information in non-designated execution functions using the graphical objects and the gesture graphics designated therefor in real time via a server and a network, and stores them in the storage unit 130. One of the main drawbacks of the existing remote control system is that products that have already been manufactured cannot be modified or updated. Therefore, when an existing remote control system is produced and released from the factory, its function must be used as is. For example, in the case of a button-type remote control, since the button is already physically fixed and manufactured, it is, of course, impossible to modify it. However, in the case of a smart TV, it can be used continuously. The reason is that the smart TV displays the operating functions on the screen. The user can find and select a desired function using the four directional buttons and execute the function by pressing the confirmation button, allowing them to continue using it even after adding more functions. However, this method significantly reduces user convenience when searching for a desired function on the screen because the user has to press the confirmation button again even after finding the desired function by repeatedly pressing the four directional buttons. In this sense, the present invention allows the installation, deletion, modification and updating of gesture information in real time through the server 150 and the network 160 as shown in the example in FIG. 6, so that, if a new function is added to the system, the present invention designates a new graphic gesture accordingly, and stores the information about it in the storage unit 130. According to a preferred feature of the present invention, the control unit 140 emits a path of each input gesture operation to the output unit 120. When a user initially inputs a gesture into input unit 110, the user doesn't know the required level of sensitivity for the gesture's speed or magnitude. Therefore, as a preferred example, control unit 140 can display the path of the gesture input from input unit 110 to output unit 120 as an overlay. Users can view this path visualization and adjust the gesture by estimating the sensitivity of the gesture operation. According to a preferred feature of the present invention, the gesture information stored in the storage unit 130 includes an application gesture library 710 that each application has and that is used to match recognized gestures when an application function is executed, and a system gesture library 720 that the system has and that is used to match recognized gestures when a system function is executed, and wherein each of the gesture libraries 710 and 720 includes a visible gesture library unit 711 and 721 that stores information about the output of execution-purpose graphic objects through the output unit 120 and the gesture graphics designated therefor as gesture information, and a hidden gesture library unit 712 and 722 that stores information about the execution,functions that are not designated using the output of graphical objects for execution purposes through output unit 120 and gesture graphics designated for this purpose as gesture information. To enable application developers and system developers to store and manage designated gesture graphics as gesture information within their own ecosystem, as shown in the example in FIG. 7, gesture information includes an application gesture library 710 that each application has and a system gesture library 720 that the system has. Each gesture library 710 and 720 includes visible gesture libraries 711 and 721 based on the graphical objects displayed on the screen and hidden gesture library units 712 and 722 based on the execution functions not used by the graphical objects displayed on the screen. According to a preferred feature of the present invention, an application is installed, the control unit 140 records the output of the application's run icon through the output unit and the gesture graphic designated therefor in the system gesture library 720, wherein the gesture graphic designated for the application's run icon uses one designated by the application producer, one recommended by the control unit 140, or one desired by the user. The installed application itself is a system application, and when the application is installed, the application's launch icon displayed on the system screen is merely a graphical object for display purposes. Therefore, when the application is installed, the graphical object and its accompanying gesture graphic are registered in the system's 720 gesture library. If the application's developer is a well-known brand or broadcaster, the designated gesture graphic might use the application's own gestures. For example, CNN, a famous broadcaster, might create a news application and make its gesture graphic resemble CNN's English-language interface with its own set of gestures. However, in a specific case, several gestures that resemble CNN's interface might already be registered in the user's 720 system gesture library.In this way, when a new gesture chart is installed and the designation 'b' is impossible because there is a predefined gesture chart, the 140 control unit can recommend another gesture chart. Furthermore, the user can designate and use the desired gesture chart. According to a preferred feature of the present invention, the control unit 140 uses the selected application gesture library 710 to match recognized gestures when an application is currently selected from one or more running applications, and uses the system gesture library 720 to match recognized gestures when none of the applications are running, or none of the applications are selected even though one or more applications are running. As described above, the present invention includes an application gesture library 710 produced, implemented, and managed by each application producer, and a system gesture library 720 produced, implemented, and managed by a system producer for gesture information management. The present invention enhances user convenience by displaying and allowing selection of numerous gestures on the screen, enabling quick, easy, and convenient execution of functions. However, the major problem arising from the use of so many gestures is when gesture conflicts occur. Each of the application gesture libraries (710) and the system gesture library (720) will be managed to address gesture conflicts that may arise during the development of their respective software programs. If a gesture conflict occurs within the respective gesture libraries (710 and 720), it is discovered through the same compilation steps used during software development, and since it is a bug, it can be corrected. However, if a gesture exists in both the application gesture library (710) and the system gesture library (720), this can cause problems. Furthermore, since convenient and easy-to-use gestures are frequently used in both the 710 and 720 gesture libraries, the likelihood of a gesture conflict is higher. To solve this problem, the present invention divides the system into two cases: one where each application runs and another where the system runs. By matching input gestures, gesture conflicts are avoided by using its own 710 and 720 gesture library only when its own function is running, regardless of whether an application or the system is running. nnzo Ln / zznz / E / YiAi According to a preferred feature of the present invention, the control unit 140 installs, deletes, modifies, updates the application gesture library 710 and the system gesture library 720 in real time through a server 150 and a network 160, and stores them in the storage unit 130. As in the example in FIG. 8, each application producer and system producer installs, deletes, modifies, and updates their gesture libraries 710 and 720 in real time using their respective server 150 and network 160, and stores them in storage unit 130. In the existing remote control system, when an application is run, only the functions within the range established by the system producer can be used. However, according to the present invention, an application producer can independently operate an application with its own functions using its own gesture library 710. For example, a home shopping company that is an app producer can designate and send gesture graphics by updating the 710 app gesture library in real time so that there is a special surprise discount on a product that is displayed on the screen during the live stream. In this case, if the user simply performs a specific gesture (nnza Ln / zznz / E / YiAi) as described above while watching the live stream, all currently available gesture charts will be displayed on the screen. If the user then performs a gesture from the designated gesture chart for the product shown on the screen, the information provided by the home shopping company will be visible. Furthermore, when a news organization conducts a real-time poll during a live broadcast, the present invention updates the 710 application's gesture library in real time to display various selectable options for the poll on the screen, along with corresponding gesture graphics. The user can then select a desired option by selecting a gesture graphic on the screen. Moreover, even during a live exchange with the user, it is possible to easily and conveniently interact with them using these various gesture graphics. In addition to this, if the respective producers can install, remove, modify, update the system gesture application and library 710 and 720 in real time and store it on storage unit 130, the fields that can be applied when using it are unlimited. According to a preferred feature of the present invention, when a specific gesture is introduced into the input unit 110 and the gesture is determined to be a multi-gesture input signal, the control unit 140 emits a signal indicating the multi-gesture input status and a signal waiting for input gestures to the output unit 120, and then repeats a process of receiving input gestures one by one and storing them in the storage unit continuously, wherein when the input gesture is determined to be a multi-gesture input termination signal, the control unit 140 matches a plurality of gestures so far stored in the storage unit 130 with the stored gesture information, and executes an operation. As described above, unlike the previous gesture-based remote control system, the present invention can receive not just one gesture, but multiple gestures. When a specific gesture graphic is composed of multiple gestures and designates a specific graphic object, the user must input multiple gestures into the input unit 110 to execute a function corresponding to the graphic object, and the control unit 140 must be able to receive and recognize multiple gestures and determine their mutual matching with the stored gesture information. nnza Ln / zznz / E / YiAi To receive multiple gestures, the control unit 140 does not immediately execute a gesture upon receiving one, as in a normal situation, but must wait for more incoming gestures. When the user performs a specific gesture, the control unit 140 can receive multiple gesture inputs, display them step by step on the screen, and wait until each of the multiple gestures desired by the user has finished. When the user performs a specific gesture and reports that there is no further gesture input, the control unit 140 summarizes all gestures stored up to that point to match the stored gesture information. According to a preferred feature of the present invention, when a gesture chart is designated with multiple gestures having two or more times a specific gesture repeated, when the specific gesture is sent to output unit 120, the control unit repeatedly emits as many gestures as the gesture number to output unit 120, or generates only one gesture and generates the repeated gesture number as a subscript. When the gesture graphic is displayed on the screen, if all gestures are shown one by one to illustrate the multiple gestures, the display of the gesture graphic becomes lengthy and may appear to overlap with a nearby graphic object. In this respect, the present invention, as in example 450 shown in FIG. 4, when a specific gesture is repeated two or more times and displayed on the screen, if the gesture is marked with a subscript, such as exponential notation in mathematics, the graphical display of the gesture can be shown concisely, which can be a useful feature. According to a preferred feature of the present invention, comprising: a transmitter located in the input unit 110 and transmitting on and off signals to the control unit 140, characterized in that the control unit 140 outputs all gesture graphics designated for execution-purpose graphic objects in the output unit 120 to the output unit when the transmitter of the input unit 110 transmits an On signal without a gesture and maintains the state in which the gesture graphics are sent to the output unit 120 when the transmitter continuously maintains an On signal without a gesture, and causes the output gesture graphics to disappear from the output unit 120 when the transmitter transmits an Off signal. Any device capable of generating on / off signals, such as a button, a switch, an optical type button, a magnetic field type button, a touch button, a touch panel, a joystick, a touch panel, and a touch screen, can be used as a transmitter located at nnza Ln / zznz / E / YiAi the input unit 110 and transmitting on / off signals to the control unit 140. As a preferred mode, as in the example in FIG. 2, the input unit 110 can be configured with the transmitter 111, including a button, and the receiver 112 to receive the transmitted content. For users who find gestures bothersome, when the button is pressed once (On) without a gesture, all gesture graphics designated for all graphic objects for execution purposes in the current screen state are displayed. And, while the button is held down (On), all gesture graphics are continuously displayed on the screen, and when the button is released (Off), all gesture graphics on the screen disappear. From the user's perspective, the function of displaying the graphic object on the screen and the corresponding gesture control will be frequently used. Therefore, user convenience can be maximized by assigning a single button press, the simplest operation, to this function. This will be more convenient and faster than voice recognition using speech. According to a preferred feature of the present invention, comprising a transmitter located in the input unit 110 and transmitting power-on and power-off signals to the control unit 140, characterized in that when the transmitter of the input unit 110 transmits a power-on and power-off signal twice within a predetermined time without a gesture, the control unit 140 emits a signal indicating the multi-gesture input status and a signal waiting for input gestures to the output unit 120, and then repeats a process of receiving input gestures one by one and storing them in the storage unit 130 continuously, wherein when the transmitter of the input unit 110 transmits a power-on and power-off signal two more times within a predetermined time without a gesture,Control unit 140 matches the plurality of gestures stored so far in storage unit 130 with the stored gesture information, and executes an operation. As a preferred configuration, as in the example in FIG. 2, the input unit 110 can consist of the transmitter 111, which includes a button, and the receiver 112 for receiving the transmitted content. From the user's perspective, gestures can also be cumbersome. The start and end of multiple gesture inputs are signaled to the control unit 140 by double-clicking. While the ability to designate multiple gestures is a significant advantage for gesture graphics, it will be a very convenient feature for users. nnza Ln / zznz / E / YiAi According to a preferred feature of the present invention, comprising a transmitter located in the input unit 110 and transmitting on and off signals to the control unit 140, characterized in that when a graphic gesture is designated with a MultiOn Off gesture having a specific gesture transmitted together with on and off signals two or more times within a predetermined time, the control unit 140 outputs the number of times that the input of on and off signals within a predetermined time prior to the specific gesture as a number when sent to the output unit 120. As a preferred modality, as in the illustration of the present invention in FIG. 4, when a Multi-On Off gesture using two or more On and Off signals is displayed on the screen within a predetermined time, the control unit 140 displays (440) a number in front of the gesture. According to a preferred feature of the present invention, comprising a transmitter located in the input unit 110 and transmitting on and off signals to the control unit 140, characterized in that when a gesture is entered into the input unit 110 with on and off signals two or more times within a predetermined time, the control unit 140 matches the gesture and the number of on and off signal inputs within a predetermined time with the stored gesture information and executes an operation. When a specific gesture graphic is designated as a Multi-On / Off gesture, the user must input the Multi-On / Off gesture into input unit 110. Control unit 140 must then be able to receive and recognize the Multi-On / Off gesture and match it to stored gesture information. The control unit matches the gesture and the number of times the on / off signals are input within a predetermined time with the stored gesture information and executes an operation. Although preferred embodiments of the present invention have been described with reference to the accompanying drawings, the embodiments described in this specification and the configurations shown in the drawings are only the most preferred embodiments of the present invention and do not represent all the technical ideas of the present invention. Therefore, it should be understood that the present invention may have various equivalents and modifications that may be substituted for it at the time of filing this application. Therefore, the embodiments described above should be understood as illustrative and not restrictive in all respects, and the scope of the present invention is indicated by the following claims instead of the detailed description, and the meaning and scope of the claims and all changes or modifications thereto arising from the concept of equivalents should be interpreted as being included within the scope of the present invention. [Explanation of the symbols] 100: Remote control system 110: Input Unit 111: Transmitter 112: receiver 120: Output unit 130: Storage unit 140: Control Unit 150: server 160: network 310: Application launch icon 320: User gesture 410, 420: single gesture 430: multiple gestures 440: Multi-On / Off gesture 450: multiple gestures 510: Execution function (volume up) not designated by graphic object 520: Undesignated execution function (up channel) nnzo Ln / zznz / E / YiAi via graphic object 530: gesture graphic designated for increasing volume 540: gesture graphic designated for upward channeling 710: Gesture Library application 711: Visible gesture library of the application's gesture library 712: Hidden gesture library of the application's gesture library 720: System gesture library 721: Visible gestures library of the system gesture library 722: Hidden gesture library of the system gesture library

Claims

1. A gesture-operated remote control system comprising: an input unit and an output unit; a storage unit for storing gesture information to be combined with an input gesture; and a control unit electrically connected to at least one of the input units, the output unit, and the storage unit to recognize the gesture and control the system, wherein the control unit recognizes a gesture input from the input unit, matches the recognized gesture with the gesture information stored in the storage unit, executes an operation corresponding to the matching gesture information, and outputs to the output unit.

2. The remote control system of claim 1, characterized in that the control unit emits information about the gestures that the user must input to the output unit.

3. The remote control system of claim 2, characterized in that the control unit designates each gesture graphic for execution purposes, outputs the graphic objects through the output unit, and stores information about the designated graphic objects and gesture graphics as gesture information to match the gesture input from the input unit to the storage unit; characterized in that the graphic objects include each menu item, an application launch icon, a specific thing or person in a streaming video, a string of characters, a portion of a slider bar, and a specific thing or person in an image or photo output through the output unit, and all output objects are directed through the output unit; and with respect to the gesture graphic, a gesture graphic consists of only one or more gestures or a combination of one or more numbers and one or more gestures.

4. The remote control system of claim 3, characterized in that the control unit designates each gesture graphic for the execution functions that are not designated using the execution purpose graphic objects that come out through the output unit, and stores information about the designated execution functions and gesture graphics as gesture information to match the gesture input from the input unit to the storage unit.

5. The remote control system of claim 3, characterized in that when a specific gesture is entered, the control unit outputs all the designated gesture graphics for the graphic objects for execution to the output unit. nnza Ln / zznz / E / YiAi 6. The remote control system of claim 4, characterized in that when a specific gesture is entered, among the gesture graphics designated for the execution functions that are not designated using the graphic objects, the control unit issues a mapping list for the currently available execution functions and the gesture graphics designated for them to the output unit.

7. The remote control system of claim 3, characterized in that the control unit emits all the designated gestural graphics for the output of graphic objects for execution purposes in the output unit together with the graphic objects to the output unit.

8. The remote control system of claim 3 or 4, characterized in that the control unit installs, deletes, modifies and updates gesture information in execution-purpose graphic objects generated through the output unit and the gesture graphics designated therefor, and gesture information in non-designated execution functions using the graphic objects and the gesture graphics designated therefor in real time through a server and a network, and stores them in the storage unit.

9. The remote control system of claims 3 or 4, characterized in that the control unit emits a trajectory of each input gesture operation to the output unit.

10. The remote control system of claim 3 or 4, characterized in that the gesture information stored in the storage unit includes an application gesture library that each application has and that is used to match recognized gestures when an application function is executed, and a system gesture library that the system has and that is used to match recognized gestures when a system function is executed, and wherein each of the gesture libraries includes a visible gesture library unit that stores information about the output of graphic objects for the purpose of execution through the output unit and the gesture graphics designated therefor as gesture information,and a hidden gesture library unit that stores information about execution functions not designated using the output of graphical objects for execution purposes through the output unit and the gesture graphics designated for this purpose as gesture information.

11. The remote control system of claim 10, characterized in that when an application is installed, the control unit registers the output of the application's execution icon through the output unit and the gesture graphic designated therefor in the system's gesture library, wherein the gesture graphic designated for the application's execution icon uses one designated by the application producer, one recommended by the control unit, or one desired by the user.

12. The remote control system of claim 10, characterized in that the control unit uses the gesture library of the selected application to match recognized gestures when an application is currently selected from one or more running applications, and uses the system gesture library to match recognized gestures when none of the applications are running, or none of the applications are selected even though one or more applications are running.

13. The remote control system of claim 10, wherein the control unit installs, deletes, modifies, updates the application gesture library and the system gesture library in real time via a server and a network, and stores them on the storage unit.

14. The remote control system of claim 3 or 4, characterized in that when a specific gesture is entered into the input unit and it is determined that the gesture is a multi-gesture input signal, the control unit emits a signal indicating the multi-gesture input status and a signal waiting for gesture input to the output unit, and then repeats a process of receiving input gestures one by one and continuously storing them in the storage unit, wherein when it is determined that the input gesture is a multi-gesture input termination signal, the control unit matches a plurality of gestures so far stored in the storage unit with the stored gesture information, and executes an operation.

15. The remote control system of claim 3 or 4, characterized in that when a gesture graphic is designated with multiple gestures having two or more times a specific gesture repeated, when the specific gesture is sent to the output unit, the control unit repeatedly emits as many gestures as the gesture number to the output unit, or generates only one gesture and generates the repeated gesture number as a subscript.

16. The remote control system of claim 3 or 4, comprising a transmitter located in the input unit and transmitting on and off signals to the control unit, characterized in that the control unit emits all gesture graphics designated for execution-purpose graphic objects in the output unit to the output unit when the transmitter of the input unit transmits an on signal without a gesture, and maintains the state in which the gesture graphics are sent to the output unit when the transmitter continuously maintains an on signal without a gesture, and causes the output gesture graphics to disappear from the output unit when the transmitter transmits an off signal.

17. The remote control system of claim 3 or 4, comprising a transmitter located in the input unit and transmitting on and off signals to the control unit, characterized in that when the transmitter of the input unit transmits an on and off signal twice within a predetermined time without a gesture, the control unit emits a signal indicating the multi-gesture input status and a signal waiting for gesture input to the output unit, and then repeats a process of receiving input gestures one by one and storing them in the storage unit continuously, wherein when the transmitter of the input unit transmits an on and off signal two more times within a predetermined time without a gesture, the control unit matches the plurality of gestures so far stored in the storage unit with the stored gesture information, and executes an operation.

18. The remote control system of claim 3 or 4, comprising a transmitter located in the input unit and transmitting on and off signals to the control unit, characterized in that when a gesture graphic is designated with a Multi-On Off gesture having a specific gesture transmitted together with on and off signals two or more times within a predetermined time, the control unit outputs the number of times that the input of on and off signals within a predetermined time prior to the specific gesture as a number when sent to the output unit.

19. The remote control system of claim 3 or 4, comprising a transmitter located in the input unit and transmitting on and off signals to the control unit, characterized in that when a gesture is input to the input unit with on and off signals two or more times within a predetermined time, the control unit matches the gesture and the number of times the on and off signals are input within a predetermined time with stored gesture information and executes an operation.

Images