Application and system for dual control of games, application and system for displaying virtual buttons, system for determining a value of at least one parameter of a user's finger, and system for determining and presenting the position of a user's finger on a display

Abstract

The present invention relates to an application and system for dual control of a game, an application and system for displaying virtual buttons, a system for determining the value of at least one parameter of a user's finger, and a system for determining and presenting the position of a user's finger on a display.

Description

[Technical Field]

[0001] The present invention relates to an application and system for dual control of a game, an application and system for displaying virtual buttons, a system for determining the value of at least one parameter of a user's finger, and a system for determining and presenting the position of a user's finger on a display. The system, including software installed on a smartphone, allows a user to hover over the screen and then accurately display the finger on the screen beneath the fingertip, allowing the user to control the game by means of the specified parameter of the finger. This is achieved by providing feedback information regarding the exact position of the finger relative to the phone screen and the type of movement made by the finger. [Background technology]

[0002] In the prior art, there are many known wearable devices intended to monitor the position of a user's hands or fingers. Many of them are based on the recognition of gestures to control computer programs or VR environments. This allows commands to be more intuitive, without the need to learn the rules that control complex input systems.

[0003] An example of such a device is disclosed in document WO2019165139. The invention discloses a wearable device that can determine the position and gestures of a user's fingers based on the flexion of their joints using only magnetic fields. The device can display the outline of the hand on the screen, but no trace of the device's position relative to the screen is disclosed. The method for locating elements with magnetic sensors can also prove useful in tablet technology. Another document, WO2019005586, discloses a device that can be worn on one of the fingers. It has optical, acceleration, and pressure sensors and can be wirelessly connected to other devices. The device can track the movement of the finger in a VR environment. This publication does not disclose any use of the invention in relation to a touchscreen or in relation to determining the position of the finger before touching the screen.

[0004] Another example of a device for determining the position of a user's hand is disclosed in US Patent No. US10642356. This invention discloses a device in the form of a flexible material containing an acceleration sensor and a gyroscope. It is intended to determine the position and movement of the hand in space in order to transfer the hand into a VR environment. A method for monitoring the user's movements involves tracking the relative movements of at least two fingers by means of this device. Tracing the movements relative to an external device such as a touchscreen is not disclosed.

[0005] Document US10579099B2 discloses a system including a ring device. The ring device may have a housing adapted to be worn on a user's finger. Sensors that can be used to collect input data from the user include force sensors, ultrasonic sensors, inertial measurement units, optical sensors, touch sensors, and other elements. A control circuit can wirelessly transmit information collected from the sensors and other input devices to an associated electronic device. This information can be used to control the operation of the electronic device. The housing of the ring device can have an annular body and an extendable portion connected to the body. The extendable portion can include a hinged flap, a rotatable housing element, a housing with an internal adjustable frame and lid, an extendable housing made of an extendable tube connected between the first and second ring devices, and other extendable structures. The presented solution does not define the type of input achieved after touching and / or pressing the screen. Summary of the Invention [Problem to be solved by the invention]

[0006] The inventions disclosed in the above documents focus on applications related to VR environments, detecting gestures based on hand movements. Furthermore, users do not receive feedback information regarding the position of their fingers. In the case of devices with touchscreens, users are forced to input commands by touching the touchscreen and then modify the input position according to their intentions. The prior art does not provide a solution that enables the determination of the exact position of a user's finger on a touchscreen and displays the determined position on the screen. Therefore, one objective of the present invention is to enable a user to stream a phone screen to an external display device and perform precise actions on the display without looking at it, but instead simply looking at the external display device. High accountability and precision of gestures input into applications displayed on the external display can then be maintained without the need to check the position of the fingers on the control device. For example, in embodiments including buttons located under the fingertips, it is even possible to transfer input gestures without the need to use a touchscreen, which allows the control device to be replaced with any flat surface, allowing the values ​​of finger parameters to be checked, and based on these, actions in a video game corresponding to the gesture to be performed. [Means for solving the problem]

[0007] The object of the present invention is an application for controlling a video game, configured to receive information from a device including a touch layer and data from an additional device for inputting data, and to control the progress of the game depending on this information and these data.

[0008] Preferably, the application is adapted to transfer data, in particular video data, to a display or a device having a display, where the display is in the form of a monitor, a smart TV, a projector or a VR or AR device, a smartphone screen or a tablet screen, in particular to a device for inputting data, in particular when the device for inputting data is a game controller, a gamepad, a joystick or a device that can be worn on a user's finger, to said display and / or to an external device.

[0009] The invention also relates to a system for controlling a video game, comprising: a first control device comprising a processor, a memory and an application for controlling a video game according to the invention; a device comprising a touch layer configured to determine the position of a user's finger on said touch layer and to transmit information about this position to said first control device; an additional device for inputting said data, configured to transmit data to the first control device; and a display, wherein the first control device is configured to receive said information from the device comprising the touch layer and data from the additional device for inputting data, and is configured to control the progress of the game depending on this information and these data, and is also configured to transmit said data, in particular video data, to the display, The display is characterized in that it is adapted to receive data, in particular video data, from the first control device and to display a video image based thereon.

[0010] Preferably the first control device, the device with the touch layer and the display are a single integrated device, in particular a smartphone or tablet.

[0011] Preferably, the device including the touch layer is a smartphone or tablet, and the touch layer is the screen of the smartphone or tablet.

[0012] Preferably, the first control device is a smartphone, a tablet, a smart TV, or a video game console.

[0013] Preferably, the additional device for inputting data is a game controller, a gamepad, a joystick, or a device that can be worn on the user's finger.

[0014] Preferably, the device with the touch layer is configured to transmit information regarding the user's finger position and / or input and / or data and / or triggered actions input by means of the finger on the touch layer directly to the first control device.

[0015] Preferably, the device with the touch layer is configured to transmit information regarding the position of the user's finger and / or input and / or data and / or trigger actions input by means of the finger on the touch layer to the first control device via an additional device for inputting data.

[0016] Preferably, the additional device for inputting data is configured to transmit the data directly to the first control device.

[0017] Preferably, the additional device for inputting data is configured to transmit data to the first control device via the device with the touch layer.

[0018] Preferably, the display is configured to receive additional data, in particular video data, from additional external sources, such as a video game console, a smartphone, a tablet, or to receive video streaming from the Internet, and to display video images based thereon.

[0019] Preferably, the display has the form of a monitor, a smart TV, a projector or a VR or AR device, a smartphone screen or a tablet screen.

[0020] Preferably, the device comprising the touch layer and / or the additional device for inputting data and / or the first control device and / or the display is configured to communicate wirelessly via Bluetooth and / or Wi-Fi and / or near field communication NFC and / or Li-Fi.

[0021] An object of the invention is also an application for generating virtual buttons, adapted to display the virtual buttons on an application or game.

[0022] Preferably, the virtual button has the form of a context menu and is configured to receive information from the device including the touch layer and possibly data from additional devices and to generate data, in particular video data, in the form of a context menu that depends on this information and these data.

[0023] It is added that preferably the application is configured to allow virtual buttons to be assigned to selected actions of a game or physical buttons of a game controller, gamepad, mouse buttons or keyboard keys.

[0024] It is added that preferably, said application is configured to send information about activated buttons to said display or an additional device for inputting data.

[0025] It is added that preferably, said application is adapted to transmit data, in particular video data, to the display.

[0026] An object of the present invention is also a system for displaying virtual buttons, comprising: a second control device comprising a processor, a memory and an application for generating virtual buttons according to the invention; optionally a system for controlling a video game according to the invention; information from a device comprising a touch layer and adapted to determine the position of a user's finger on said touch layer and to transmit information about this position to the second control device, and possibly from an additional device adapted to transmit data to the second control device; and a display, wherein said second control device is adapted to receive data from the device comprising the touch layer and possibly from said additional device, to generate data, in particular video data, in the form of a context menu depending on this information and these data, and to transmit these data, in particular video data, to the display, The display is configured to receive data, particularly video data, from the second controller and to display a video image based thereon.

[0027] Preferably, the context menu includes any number of buttons distributed in a circle around a location corresponding to the position of the user's finger as determined by the device including the touch layer.

[0028] Preferably, the button is configured to transmit data, after activation, to an external device, in particular an additional device for inputting data, and these data are transmitted to a second control device, in particular a tablet, phone or television, as information about pressing a physical button corresponding to a selected button of a game controller or a keyboard and / or mouse.

[0029] Preferably, the device including the touch layer is a smartphone or a tablet, and the touch layer is the screen of the smartphone or tablet.

[0030] Preferably, the second control device is a smartphone, a tablet, a smart TV, or a video game console.

[0031] Preferably, the additional device for inputting said data is a game controller, a gamepad, a joystick, or a device wearable on the user's finger.

[0032] Preferably, the display is configured to receive additional data, in particular video data, from additional external sources, such as a video game console, a smartphone, a tablet, or to receive video streaming from the Internet, and to display video images based thereon.

[0033] Preferably, the display has the form of a monitor, a smart TV, a projector or a VR or AR device, a smartphone screen or a tablet screen.

[0034] Preferably, the device comprising the touch layer and / or the additional device for inputting said data and / or the second control device and / or the display is configured to communicate wirelessly via Bluetooth and / or Wi-Fi and / or Near Field Communication NFC and / or Li-Fi.

[0035] The invention also relates to a system for determining the value of at least one parameter of a user's finger, comprising: a device wearable on a user's finger; at least one sensor wearable on the user's finger; a processor; a memory; a third control application; optionally a system for controlling a video game according to the present invention; and optionally a system for displaying virtual buttons according to the present invention, wherein the at least one sensor is configured to send a signal to the third control application; further comprising a configuration for determining a value of at least one parameter of the user's finger, the configuration including at least one sensor and configured to determine a value of the at least one parameter of the user's finger and to transmit information regarding the determined value of the at least one parameter of the user's finger to the third control device; The third control device is configured to receive information regarding the determined value of at least one parameter of the user's finger.

[0036] Preferably, the sensor is in the form of a motion sensor and / or a pressure sensor and / or an optical sensor and / or a proximity sensor.

[0037] Preferably, the system for determining the value of at least one parameter of a user's finger further comprises a navigation surface, in particular a touchscreen of a smartphone or tablet, or a navigation surface in the form of a touchpad or trackpad.

[0038] Preferably, said navigation surface is configured to determine a value of at least one parameter of a user's finger, and the navigation surface is configured to transmit the value of the at least one parameter of the user's finger to a third control device.

[0039] Preferably, the system for determining the value of at least one parameter of a user's finger is configured to determine the value of a parameter of a user's finger (meaning the angle between the distal phalanges and the proximal phalanges).

[0040] Preferably, said system for determining the value of at least one parameter of a user's finger is configured to determine the value of a parameter of a user's finger (meaning the pressure applied by the finger to a sensor).

[0041] Preferably, said system for determining a value of at least one parameter of a user's finger is configured to determine a value of a parameter of a user's finger (meaning the pressure exerted by the finger on the navigation surface).

[0042] Preferably, the system for determining the value of at least one parameter of a user's finger further comprises a display.

[0043] Preferably, the display is a projector, a monitor, a smart TV, a VR or AR device, a smartphone screen, or a tablet screen.

[0044] Preferably, the display is configured to receive data from a third control device and display an image based thereon.

[0045] Preferably, the display is configured to receive additional data, in particular video data, from additional external sources, for example a video game console, a smartphone, a tablet, or to receive video streaming from the Internet, and to display video images including images on the basis thereof.

[0046] Preferably, said third control device is configured to perform an action based on the value of a parameter of a user's finger.

[0047] Preferably, the third control device is a tablet or a smartphone.

[0048] Preferably, the third control device is integrated with a display, in particular the third control device is a smart TV.

[0049] Preferably, the third control device is a video game console.

[0050] Preferably, the user's finger wearable device has the form of a clip intended to be placed on the user's finger from above, or from the side, or in the form of an open or closed ring with a sensor.

[0051] Preferably, the device wearable on a user's finger comprises a sensor adapted to measure the angle between the distal and proximal phalanges of the user's finger.

[0052] Preferably, the device wearable on the user's finger comprises a motion sensor adapted to measure the pressure applied by the finger to the navigation surface, and to measure the positional and angular differences between the finger and the navigation surface.

[0053] Preferably, said device wearable on a user's finger comprises a pressure sensor and / or an optical sensor adapted to measure the pressure applied by the finger on the navigation surface.

[0054] Preferably, the device wearable on a user's finger has an additional portion extending over the distal phalanx of the finger, the additional portion having a movement sensor located thereon.

[0055] Preferably, the device wearable on the user's finger comprises a button under the fingertip, in particular a pressure sensor configured to generate and send a signal to a third control device when pressed.

[0056] Preferably, said third control device and / or device wearable on a user's finger and / or the arrangement for determining a value of at least one parameter of the user's finger and / or the navigation surface and / or the display is configured to communicate wirelessly via Bluetooth and / or Wi-Fi and / or near field communication NFC and / or Li-Fi.

[0057] Preferably, the system for determining the value of at least one parameter of a user's finger is configured such that the user defines a portion of the surface of a touch layer in the form of a touchscreen of a smartphone or tablet constituting a navigation surface.

[0058] The present invention also provides a system for determining and presenting on a display the position of a user's finger, comprising: a fourth control device comprising a processor, a memory, a fourth control application, the fourth control device being configured to determine a position of a finger on a navigation surface, the navigation surface being configured to transmit information about the position to the fourth control device, a display, and optionally a system for controlling a video game according to the present invention, a system for displaying virtual buttons according to the present invention, and a system for determining at least one parameter of a finger according to the present invention; further comprising an arrangement for determining a user's finger position on the navigation surface, the arrangement having at least one sensor and configured to determine the user's finger position and to transmit the determined information to the fourth control device; the fourth control device receives the information from the arrangement for determining a user's finger position and transmits data, in particular video data, to the display, the data, in particular video data, including a finger position indicator corresponding to the user's finger position on the navigation surface; The display is configured to receive data, in particular video data, from the fourth control device and to display a video image based thereon.

[0059] Preferably, the fourth control device is a tablet or a smartphone or a video game console.

[0060] Preferably, the fourth control device is integrated with a display, in particular the fourth control device is a smart TV.

[0061] Preferably, the display has the form of a monitor, a smart TV, a projector or a VR or AR device, a smartphone screen or a tablet screen.

[0062] Preferably, the display is configured to receive additional data, in particular video data, from additional external sources, for example a video game console, a smartphone, a tablet, or to receive video streaming from the Internet, and to display video images including images thereon.

[0063] Preferably, the navigation surface is a touch layer, in particular the surface of a touchscreen of a tablet or smartphone.

[0064] Preferably, the system for determining and presenting the position of a user's finger on a display is in the form of a touchscreen of a smartphone or tablet, where the user constitutes the navigation surface, and is configured to constitute the surface of the touch layer.

[0065] Preferably, the arrangement for determining the position of a user's finger on the navigation surface is adapted to be placed on at least one of the user's fingers, the arrangement comprising an accelerometer and / or a gyroscope and configured to use data from said accelerometer and / or gyroscope to determine the position of the user's finger on the navigation surface in real time.

[0066] Preferably, the arrangement for determining the position of the user's finger on the navigation surface further comprises an optical sensor configured to assist in determining the position of the user's finger on the navigation surface by determining the finger flexion level by determining the angle between the distal and proximal phalanges.

[0067] Preferably, the fourth control device and / or the navigation surface and / or the arrangement for determining the position of a user's finger on the navigation surface and / or display are configured to communicate wirelessly via Bluetooth and / or Wi-Fi and / or near field communication NFC and / or Li-Fi.

[0068] Preferably, the system for determining and presenting the position of a user's fingers on the display is configured to recognize gestures performed by the user's fingers through and / or on the navigation surface and / or in the form of a particular configuration of one or more of the user's fingers.

[0069] Preferably, the first controller and / or the second controller and / or the third controller and / or the fourth controller are the same device. [Effects of the Invention]

[0070] The system according to the present invention combines touch navigation, known from mobile devices such as phones or tablets, with additional external input and feedback from one or more external devices worn on the fingers to form a controller for games, particularly, but not exclusively. That is, it combines the positive properties of touch navigation known from mobile games, such as the precision of the touch interface and control, with an additional input that allows the execution of multiple actions simultaneously, thanks to feedback functionality, without the need to use several fingers, making it possible to control the system, application, and game without looking at the fingers. Combined with virtual buttons displayed on a transparent layer above the game, the system is configured to transmit, after activation, information corresponding to the pressed physical button corresponding to the selected button on the game controller or keyboard and / or mouse, allowing players to use touch navigation to play any game streamed from the cloud or on any device, even if these games do not officially support touch navigation. [Brief explanation of the drawings]

[0071] The present invention will now be presented in more detail in preferred embodiments with reference to the accompanying drawings, in which: [Figures 1 to 3] User's finger parameter values ​​(meaning the angle between the distal and proximal phalanges) 1 shows a system for determining [Figures 4 to 6] Determining the parameter values ​​of the user's finger (pressure applied to the sensor by the finger) This shows a system for doing so. [FIGS. 7-11] A display configured to allow a user to define a portion of the surface of the touch layer. 1 illustrates a system for determining and presenting the position of a user's finger on a ray. [Fig. 12-Fig. 13] A ring with a sensor, which can be accessed from above, from the side, or open, or indicates a device that can be worn on a user's finger, in the form of a clip, meaning that it is positioned on the user's finger from a closed ring. [Figures 14-15] Adapted to measure the angle between the distal and proximal phalanges of a user's finger 1 shows a device wearable on a user's finger with a sensor configured as a [FIGS. 17-19] A device wearable on a user's finger, comprising an additional finger extending over the distal phalanges of the finger. 1 shows a device having a portion with a motion sensor located in an additional portion. DETAILED DESCRIPTION OF THE INVENTION

[0072] In one preferred embodiment, the present invention takes the form of a system for controlling a video game. The system includes a first control device, the first control device including a processor, a memory, and an application for controlling the video game. The application for controlling the video game is configured to receive information from a device with a touch layer and data from an additional device for inputting data, and is configured to control the progress of the game in response to the information and data. Preferably, the application for controlling the video game is configured to transmit data, in particular video data, to a display or device with a display and / or to an external device, where the display is in the form of a monitor, a smart TV, a projector, a VR or AR device, a smartphone screen, or a tablet screen. In particular, the device for inputting data, in particular when the device for inputting data is a game controller, gamepad, joystick, or device wearable on a user's finger. The system includes a device with a touch layer, configured to determine the position of a user's finger thereon and transmit information about this position to a first control device. The system also includes an additional device for inputting data and configured to transmit data to the first control device, and a display. The first control device is adapted to receive said information from the device with the touch layer and data from the additional device for inputting data, and is configured to control the progress of the game according to this information and these data and to send data, in particular video data, to the display, which in turn is configured to receive the data, in particular the video data, from the first control device and display a video image based thereon.

[0073] Preferably the device comprising the touch layer is a smartphone or tablet, and the touch layer is the screen of the smartphone or tablet.

[0074] The first control device may be a smartphone, tablet, smart TV, or video game console. The additional device for inputting data may be a game controller, gamepad, joystick, or a device wearable on the user's finger.

[0075] The device including the touch layer is configured to transmit information about the position of the user's finger and / or input and / or data, and / or information about triggered actions, that are input directly to the first control device by means of a finger on the touch layer.

[0076] Preferably, the device with the touch layer is configured to transmit information about the position of the user's finger on the touch layer to the first control device via an additional device for inputting data, the additional device for inputting data being configured to transmit data directly to the first control device.

[0077] According to another preferred embodiment, the additional device for inputting data is configured to transmit data to the first control device via the device comprising the touch layer.

[0078] Preferably, the display is configured to receive additional data, in particular video data, from additional external sources, such as a video game console, a smartphone, a tablet, or to receive video streaming from the Internet, and to display video images based thereon.

[0079] Preferably, the display may have the form of a monitor, a smart TV, a projector or a VR or AR device, a smartphone screen or a tablet screen.

[0080] The device with the touch layer and / or additional devices for inputting data and / or the first control device and / or the display is preferably configured to communicate wirelessly via Bluetooth and / or Wi-Fi and / or near field communication NFC and / or Li-Fi.

[0081] According to a further embodiment, the invention comprises a second control device in the form of a system for displaying virtual buttons in the form of a context menu, the second control device comprising a processor, a memory and an application for generating the virtual buttons, and the application for generating data, in particular video data, in the form of a context menu, is configured to display them on an application or game. The virtual buttons are in the form of a context menu and are configured to receive information from a device comprising a touch layer and possibly data from an additional device, and to generate data, in particular video data, in the form of a context menu that depends on this information and on these data. The application is additionally configured so that virtual buttons can be assigned to selected actions in the game or physical buttons on a game controller, gamepad, mouse button, or keyboard key. The application can also be configured to send information about the activated button to a display or an additional device for inputting data and to send data, particularly video data, to the display. The system further includes a device with a touch layer configured to determine the position of a user's finger on the touch layer and send information about this position to a second control device. The system can also include an additional device for inputting data configured to send data to the second control device. The system also includes a display on which virtual buttons are displayed, for example in the form of a context menu. The second control device is configured to receive the information from the device with the touch layer and possibly data from the additional device, generate data, particularly video data, in the form of a context menu depending on this information and data, and send this data, particularly video data, to the display. The display is configured to receive data, particularly video data, from the second control device and display a video image based on it.

[0082] Furthermore, the system may be a system including a first control device for controlling a video game, the system including a processor, a memory, and an application for controlling the video game. The application for controlling the video game is configured to receive information from the device with a touch layer and data from the additional device for inputting data, and to control the progress of the game according to the information and data. Preferably, the application for controlling the video game is further configured to transmit data, in particular video data, to a display. Thus, the system further includes a device with a touch layer configured to determine the position of a user's finger on the touch layer and transmit information about the position to the first control device. The system also includes an additional device for inputting data, configured to transmit data to the first control device, and a display. The first control device is configured to receive the information from the device with a touch layer and data from the additional device for inputting data, control the progress of the game according to the information and data, and transmit data, in particular video data, to the display. The display is then configured to receive data, in particular video data, from the first control device and display a video image based thereon.

[0083] Preferably, the context menu includes any number of buttons, preferably four buttons, distributed in a circle around a location corresponding to the user's finger position as determined by the device including the touch layer. This allows the user to select an appropriate command from the context menu depending on the need or game situation. Preferably, the buttons correspond in appearance and function to physical buttons on a standard game controller, which is typically a pad or joystick. After activation, the buttons are configured to transmit data to an external device, particularly an additional device for inputting data, and these data are transmitted to a second control device, particularly a tablet, phone, or television, as information regarding the pressing of a physical button corresponding to a selected button on the game controller or keyboard and / or mouse.

[0084] The device including the touch layer is a smartphone or tablet, and the touch layer is the screen of the smartphone or tablet. The second control device can be a smartphone, tablet, smart TV, or video game console.

[0085] Preferably, the additional device for inputting data is a game controller, a gamepad, a joystick, or a device wearable on a user's finger.

[0086] Preferably, the display is configured to receive additional data, in particular video data, from additional external sources, such as a video game console, a smartphone, a tablet, or to receive video streaming from the Internet, and to display video images based thereon.

[0087] Preferably, the display has the form of a monitor, a smart TV, a projector or a VR or AR device, a smartphone screen or a tablet screen.

[0088] Preferably, the device comprising the touch layer and / or the additional device for inputting data and / or the second control device and / or the display is configured to communicate wirelessly, preferably via Bluetooth and / or Wi-Fi and / or Near Field Communication NFC and / or Li-Fi.

[0089] By using a system for displaying virtual buttons, the virtual buttons may appear anywhere in the interface, or in the form of a menu that appears within the touch range of one of the user's fingers on the screen, or on a transparent layer above a game or application, particularly a game streaming application. Upon activation, the virtual buttons are not in the form of performing an action, but rather the game or application performs an action assigned to a physical button on the game controller, but rather to a button corresponding to any button on a conventional controller, or to a conventional navigation, e.g., keyboard, action assigned to the game or application.

[0090] The purpose of a context menu is to add more additional inputs to perform side actions, and this context menu can have the form of a menu that is displayed on the interface at the location of the finger, whether the finger is positioned where it contacts the navigation surface or by a feedback function.

[0091] Launching the menu: a. The menu may be activated using one of the buttons on a device wearable on the user's finger b. The menu may be activated by pressing a button on the game or application interface. b. The menu may be activated by pressing a button on the game or application interface. c. The menu may be activated by an external mechanical button that depresses the game or application interface. Preferably, these may be triggers that, when pressed by a finger after a call is made, mechanically contact the screen where the button is on the interface. d. The menu may be activated using a gesture.

[0092] The context menu may be displayed only on the screen of the navigation space and / or on an external screen. For example, the menu may be displayed within one of the fingers surrounding a given finger (where it touches the navigation space or where it is located in the space). This may be the same finger that activates the menu, or a different finger. For example, one of the buttons on the tip of the left thumb activates the menu surrounding the right finger.

[0093] The context menu may have one or more buttons that correspond to actions assigned to buttons on a conventional controller, where buttons on the conventional controller are assigned to the menu and the game / application performs the action previously assigned in the conventional controller configuration. The context menu may also have one or more buttons that correspond to actions assigned in the game / application.

[0094] The application displaying the virtual buttons in the context menu communicates back with the controller, which, after receiving information about the selected position from the context menu, transmits information about the input, i.e., the pressed button, to the device. For example, the main and side actions from the context menu can be assigned to specific buttons on a conventional controller supported by the game or application. In such a case, upon receiving information about the selection of an action in the input menu, the controller returns information about the input corresponding to, for example, pressing button A on the controller. An alternative solution envisions using a motion sensor that recognizes the direction of movement toward the button / action displayed in the menu being approached by the finger. In such a situation, data about the type of input transmitted, e.g., button A or B, is transmitted directly by the controller without first receiving data from the application. A context menu displayed in an application or on an external screen is merely one form of visual feedback information when making a selection.

[0095] Activation of the context menu proceeds by touching / pressing the selected button or by swiping a finger and releasing it over the button.

[0096] The context menu can take any form that places action icons within a short distance of the user's finger from which the user can select an action. The simplest example is a circle divided into any number of sections. Where multiple levels of context menus are possible, selecting one of the buttons in the context menu displays the next level of the context menu with additional actions.

[0097] An interface with virtual buttons on a transparent layer over a game or application, where the virtual buttons are configured to correspond to actions in the game or to cause input corresponding to any button on a popular game controller, gamepad, mouse button, or keyboard key.

[0098] This solution is particularly intended to use game-supported navigation using a game controller, keyboard, and mouse, and to generate virtual navigation based thereon, providing the ability to play games streamed from the cloud that do not have native touch navigation on devices with touch screens.

[0099] The use of the system with the context menus makes it possible to play games streamed from the cloud on devices with a touch layer that do not have native touch navigation, but only traditional navigation that supports game controllers, keyboards, and mice.

[0100] The system for displaying data, in particular video data, in the form of said context menu is within the range of one of the user's fingers touching the touch layer, which makes it possible to select any / each action of this menu only by moving the finger, without any significant hand movement.

[0101] Preferably, a system for displaying data in the form of a context menu, in particular video data, is arranged around one of the fingers that contacts the touch layer.

[0102] According to another preferred embodiment of the present invention, selection of an action from a context menu occurs by swiping and releasing a finger on the selected button, without withdrawing the finger from the moment said menu appears on the touch layer.

[0103] The context menu may be launched in another application, in particular an external application launched via a game or game streaming application. The context menu may also be launched in a game streaming application on a layer with streamed images of the game. It may also be displayed on the graphics layer of the game or application / game or application.

[0104] Preferably, activation of said context menu virtual button is assigned in the application code to trigger an action indirectly by a game controller or keyboard and / or mouse button assigned thereto.

[0105] Preferably, information regarding the input from the context menu received by the external device is further transmitted to another device, e.g. a smart TV, or returned to the device from which it was received, e.g. a phone.

[0106] Preferably, information about the button selected from said context menu is sent to an external device, in particular an additional device for inputting data, and returned to the control device, in particular a tablet, phone or television, as information about pressing a physical button / physical input corresponding to a button on a game controller or keyboard and / or mouse.

[0107] The purpose of adding external inputs is to allow multiple actions to be performed simultaneously while gaming without the need to use additional fingers. This solution improves the ergonomics, efficiency, comfort, and experience of playing more complex mobile titles, as well as enabling the playing of console and PC titles with touch navigation via cloud gaming, including but not limited to:

[0108] Yet another aspect of the present invention provides a system for determining a value of at least one parameter of a user's finger, the system comprising: a device wearable on the user's finger; at least one sensor disposed within or on the device wearable on the user's finger; and a third control device comprising a processor, a memory, and a third control application. The at least one sensor is adapted to transmit a signal to the third control device. The system further comprises an arrangement for determining the value of the at least one parameter of the user's finger, the arrangement comprising the at least one sensor, configured to determine the value of the at least one parameter of the user's finger, and to transmit information regarding the determined value of the at least one parameter of the user's finger to the third control device. The third control device is configured to receive information regarding the determined value of the at least one parameter of the user's finger.

[0109] Additionally, the system for determining the value of at least one parameter of a user's finger comprises a first control device for controlling a video game, the device comprising a processor, a memory, and an application for controlling the video game. The application for controlling the video game is configured to receive information from a device with a touch layer and data from a device for inputting data, and to control the progress of the game according to the information and the data. Preferably, the application for controlling the video game is further configured to transmit data, in particular video data, to a display. Accordingly, the system further comprises a device with a touch layer, configured to determine the position of the user's finger on the touch layer and transmit information about the position to the first control device. The system also comprises an additional device for inputting data, configured to transmit data to the first control device, and a display. The first control device is configured to receive the information from the device with a touch layer and data from the additional device for inputting data, control the progress of the game according to the information and the data, and transmit data, in particular video data, to the display. The display is then configured to receive data, in particular video data, from the first control device and display a video image based thereon.

[0110] Additionally, a system for determining the value of at least one parameter of a user's finger may include a second control device consisting of a processor, a memory, and an application for a context menu; and an application for generating a virtual button, the second control device including a processor, a memory, and an application for the virtual button. The application for generating data, particularly video data, in the form of a context menu is configured to display them on an application or game. The virtual button has the form of a context menu and is configured to receive information from a device including a touch layer and possibly data from an additional device, and generate data, particularly video data, in the form of a context menu depending on the information and data. The application is additionally configured so that the virtual button can be assigned a selected action of a physical button in a game or on a game controller, gamepad, mouse button, or keyboard key. The application may also be configured to send information about the activated button to a display or an additional device for inputting data and to send data, particularly video data, to the display. The system further includes a device including a touch layer, configured to determine the position of the user's finger on the touch layer and send information about the position to the second control device. The system may comprise an additional device for inputting data, configured to transmit the data to a second control device. The system also comprises a display for displaying, for example, a context menu. The second control device is configured to receive the information from the device with the touch layer and possibly data from the additional device, generate data in the form of a context menu, in particular video data, depending on the information and data, and transmit the data, in particular video data, to a display. The display is configured to receive the data, in particular video data, from the second control device and display a video image based thereon.

[0111] At least one sensor may contact the surface of the finger.

[0112] Preferably, the sensor is in the form of a camera and / or a proximity sensor and / or a motion sensor and / or a pressure sensor and / or an optical sensor.

[0113] The system for determining the value of at least one parameter of a user's finger may further comprise a navigation surface, in particular in the form of a touch layer, in particular in the form of a touchscreen or touchpad or trackpad of a smartphone or tablet, configured to determine the value of the at least one parameter of the user's finger, and the navigation surface configured to transmit the value of the at least one parameter of the user's finger to a third control device.

[0114] Preferably, the system for determining the value of at least one parameter of the user's finger is configured to determine the value of the parameter of the user's finger, meaning the angle between the distal phalanges and the proximal phalanges, which is shown in Figures 1-3.

[0115] Preferably, the system for determining the value of at least one parameter of a user's finger is configured to determine the value of a parameter of a user's finger, meaning the pressure applied by the finger to the sensor, as presented in Figures 4-6.

[0116] Preferably, the system for determining the value of at least one parameter of a user's finger is configured to determine the value of the parameter of the user's finger, meaning the pressure applied by the finger on the navigation surface and / or the angle between the distal and proximal phalanges, where information about the parameter is collected when the finger touches the navigation surface, preferably through the value of the pressure applied by the finger to the navigation surface.

[0117] The system for determining the value of at least one parameter of a user's finger also includes a display, which may be a projector, a monitor, a smart TV, a VR or AR device, a smartphone screen, or a tablet screen, configured to receive data from the third control device and display an image based thereon.

[0118] Preferably, the display is configured to receive additional data, in particular video data, from an additional external source, for example a video game console, a smartphone, a tablet, or to receive video streaming from the Internet, and to display video images based thereon from a third control device, including images obtained based on the received data.

[0119] Preferably, the third control device is configured to perform an action, a movement based on the value of the parameter of the user's finger.

[0120] The third control device may be a tablet or a smartphone. Furthermore, the third control device may be integrated with a display, particularly a smart TV. Preferably, the third control device is a video game console.

[0121] Preferably, the device wearable on a user's finger has the form of a clip that can be placed on or on the side of the user's finger, or an open or closed ring with sensors as shown in FIGS. 12 and 13. The ring only has a sensor (e.g., but not limited to, EMG) that detects changes in muscle or tissue tension in the ring finger. It can detect finger pressure, and information about the tension change can be used to read the input, and information about the finger tension can be used to identify the finger flexion angle, which signifies the type of input. The ring can be supplemented with additional sensors, such as a camera, proximity sensor, or infrared sensor, to help evaluate finger flexion.

[0122] Preferably, the device wearable on the user's finger comprises a sensor adapted to measure the angle between the distal and proximal phalanges of the user's finger, as presented in Figures 14 and 15. Preferably, an additional element extends from the top and is made of a flexible material, for example rubber, so as to always press against the top of the distal phalange of the finger. There are many more fastening methods possible; it could also extend from the side or bottom. The important feature is that the motion sensor should abut against the distal phalange and move with it during flexion.

[0123] Preferably, the device wearable on a user's finger has an additional portion extending over the distal phalanges of the finger, with the motion sensor located in the additional portion shown in Figures 16-19. Although the structure has a completely different visual appearance, this solution can be derived from the solutions shown in Figures 14 and 15. This solution has the form of a clip or open ring, which in this case is placed laterally. The motion sensor is still located on the distal phalanges of the finger and performs the same function. A "fingertip button" element below the fingertip is an additional portion that generates an input when pressed.

[0124] Preferably, the device wearable on the user's finger may comprise a button under the fingertip, in particular a pressure sensor configured to generate and transmit a signal to a third control device when pressed.

[0125] In another embodiment, the device wearable on a user's finger comprises a pressure sensor and / or an optical sensor adapted to measure the pressure exerted by the finger on the navigation surface.

[0126] Preferably, the device wearable on a user's finger may have an additional portion extending beyond the distal phalanx of the finger, the additional portion having a movement sensor located thereon.

[0127] Preferably, said device wearable on a user's finger comprises a button under the fingertip, in particular a pressure sensor configured to generate and transmit a signal to a third control device when pressed.

[0128] Preferably, said third control device and / or the device wearable on a user's finger and / or the arrangement for determining a value of at least one parameter of the user's finger and / or the navigation surface and / or the display is configured for communication via wireless communication, preferably via Bluetooth and / or Wi-Fi and / or near field communication NFC and / or Li-Fi.

[0129] Preferably, the system for determining the value of at least one parameter of a user's finger is configured for the user to define a portion of the surface of the touch layer in the form of a touchscreen of the smartphone or tablet, which in particular constitutes a navigation surface for the smartphone or tablet, the touchscreen of the smartphone or tablet is configured to determine the value of the at least one parameter of the user's finger, and the navigation surface is configured to transmit the value of the at least one parameter of the user's finger to the third control device.

[0130] Another aspect of the present invention is a system for determining and presenting a user's finger position on a display, the system comprising: a fourth control device configured to determine the position of the finger across the navigation surface, the navigation surface comprising a processor, a memory, and a fourth control application configured to then transmit information about the position to the fourth control device; and a display. The fourth control device configured to determine the finger position on the navigation surface also comprises an arrangement for determining the position of the user's finger on the navigation surface, the arrangement comprising at least one sensor, configured to determine the position of the user's finger and transmit information about the determined position to the fourth control device. The fourth control device is configured to receive the information from the arrangement for determining the user's finger position and transmit data, particularly video data, to a display, the video data comprising a finger position indicator corresponding to the position of the user's finger across the navigation surface. The display is configured to receive the data, particularly video data, from the fourth control device and display a video image based thereon, including the finger position indicator.

[0131] The system for determining and presenting the position of a user's finger on a display may also comprise a system for controlling a video game, the system comprising a first control device, a processor, a memory, and an application for controlling the video game. The application for controlling the video game is configured to receive information from a device with a touch layer and data from an additional device for inputting data, and to control the progress of the game in accordance with the information and data. Preferably, the application for controlling the video game is further configured to transmit data, in particular video data, to a display. Thus, the system further comprises a device with a touch layer configured to determine the position of a user's finger on the touch layer and transmit information about the position to the first control device. The system also comprises the additional device for inputting data, configured to transmit data to the first control device, and a display. The first control device is configured to receive the information from the device with a touch layer and data from the additional device for inputting data, control the progress of the game in accordance with the information and data, and transmit data, in particular video data, to a display. The display is then configured to receive data, in particular video data, from the first control device and display a video image based thereon.

[0132] Additionally, the system for determining and presenting the position of a user's finger on a display may include a system for displaying virtual buttons, the system including a second control device including a processor, a memory, and an application for generating the virtual buttons. The application for generating data, particularly video data, in the form of a context menu is configured to display them across applications or games. The virtual buttons have the form of a context menu and are configured to receive information from a device including a touch layer and possibly data from an additional device, and generate data, particularly video data, in the form of a context menu depending on the information and data. The application is additionally configured so that the virtual buttons can be assigned selected actions of physical buttons in a game or on a game controller, gamepad, mouse button, or keyboard key. The application may also be configured to send information about activated buttons to a display or additional device for inputting data and to transmit data, particularly video data, to the display. The system may further include a device including a touch layer configured to determine the position of a user's finger on the touch layer and transmit information about the position to a second control device. The system may include an additional device for inputting data configured to transmit data to the second control device. The system also has a display for displaying, for example, a context menu. The second control device is configured to receive the information from the device with the touch layer and possibly data from further devices, to generate data in the form of a context menu, in particular video data, in response to this information and these data, and to transmit this data, in particular video data, to the display. The display is configured to receive the data, in particular video data, from the second control device and to display a video image based thereon.

[0133] Additionally, the system for determining and presenting the position of a user's finger on a display may also include a system for determining a value of at least one parameter of a user's finger, the system comprising: a device wearable on the user's finger; at least one sensor disposed in or on the device wearable on the user's finger; and a third control device comprising a processor, a memory, and a third control application. The at least one sensor is adapted to transmit a signal to the third control device. The system further comprises a configuration for determining the value of the at least one parameter of the user's finger, the configuration comprising the at least one sensor and configured to determine the value of the at least one parameter of the user's finger and transmit information regarding the determined value of the at least one parameter of the user's finger to the third control device. The third control device is configured to receive information regarding the determined value of the at least one parameter of the user's finger.

[0134] The fourth control device may be a tablet, a smartphone, or a video game console. The fourth control device is integrated with a display, in particular the fourth control device is a smart TV.

[0135] The display may have the form of a monitor, a smart TV, a projector or a VR or AR device, a smartphone screen or a tablet screen, and is configured to receive additional data, in particular video data, from additional external sources, for example a video game console, a smartphone, a tablet or to receive video streaming from the Internet, and to display video images including images on their basis.

[0136] Preferably, the navigation surface of the system for determining and presenting the position of the user's finger on the display is the surface of a touch layer, in particular the touch screen of a tablet or smartphone.

[0137] Preferably, the system for determining and presenting the position of a user's finger on the display is configured so that the user defines a portion of the surface of the touch layer in the form of a touchscreen of a smartphone or tablet, which constitutes the navigation surface presented in Figures 7-11.

[0138] Preferably, the arrangement for determining the position of a user's finger across a navigation surface is adapted to be placed on at least one of the user's fingers, the arrangement comprising an accelerometer and / or a gyroscope and configured to use data from the accelerometer and / or gyroscope to determine the position of the user's finger on the navigation surface in real time.

[0139] In yet another embodiment of the present invention, an arrangement for determining a position of a user's finger on a navigation surface is adapted to be placed on at least one of the user's fingers, the arrangement comprising a magnetometer and configured to use data from the magnetometer to determine the position of the user's finger on the navigation surface in real time.

[0140] In another embodiment of the invention, the arrangement for determining the position of a user's fingers on the navigation surface comprises at least one camera and is configured to use data from the at least one camera to determine the position of the user's fingers on the navigation surface in real time, preferably said arrangement being a device wearable on at least one of the user's fingers having an indicator.

[0141] According to yet another aspect of the present invention, an arrangement for determining the position of a user's finger on a navigation surface comprises a device configured to detect light intensity and configured to use data from said device to determine in real time the position of a user's finger across the navigation surface, said arrangement configuring a light barrier or reflector of incident light by said navigation.

[0142] Preferably, the arrangement for determining the position of the user's finger across the navigation surface further comprises an optical sensor configured to assist in determining the position of the user's finger on the navigation surface via determining the angle between the distal and proximal phalanges and thereby determining the finger flexion level.

[0143] Preferably, the fourth control device and / or the arrangement for determining the position of a user's finger across the navigation surface and / or the navigation surface and / or the display are configured to communicate wirelessly, preferably via Bluetooth and / or Wi-Fi and / or near field communication NFC and / or Li-Fi.

[0144] Preferably, the system for determining and presenting the position of a user's fingers on the display is configured to recognize gestures performed by the user's fingers above and / or on the navigation surface and / or in the form of a particular configuration of one or more of the user's fingers.

[0145] According to another embodiment of the invention, the first control device and / or the second control device and / or the third control device and / or the fourth control device are the same device.

[0146] Each of the systems according to the present invention, i.e. the system for dual control of a game, the system for displaying virtual buttons, the system for determining a value of at least one parameter of a user's fingers, and the system for determining and presenting the position of a user's fingers on a display, may be implemented individually as a sole system without the implementation of the remaining systems, or any two of them may be implemented, i.e. 1) a combination of a system for dual control of a game and a system for displaying data, in particular video data, in the form of a context menu; or 2) a combination of a system for dual control of a game and a system for determining a value of at least one parameter of a user's fingers; or 3) a combination of a system for dual control of a game and a system for determining and presenting the position of a user's fingers on a display; 4) a combination of a system for displaying data, in particular video data, in the form of a context menu and a system for determining a value of at least one parameter of a user's fingers; or 5) a combination of a system for displaying data, in particular video data, in the form of a context menu and a system for determining a value of at least one parameter of a user's fingers and a system for determining and presenting the position of a user's fingers on a display; or 6) a combination of a system for determining a value of at least one parameter of a user's fingers and a system for determining and presenting the position of a user's fingers on a display.

[0147] Three combinations of the above-mentioned systems are possible: 1) a system for dual control of a game, a system for displaying data, in particular video data, in the form of a context menu, and a system for determining the value of at least one parameter of a user's finger, 2) a system for displaying data, in particular video data, in the form of a context menu, a system for determining the value of at least one parameter of a user's finger, and a system for determining the value of at least one parameter of a user's finger and presenting the position of the user's finger, 3) a system for dual control of a game, a system for displaying data, in particular video data, in the form of a context menu, a system for determining and presenting the position of a user's finger on a display, and a system for dual control of a game; and 4) a system for dual control of a game, a system for determining the value of at least one parameter of a user's finger, and a system for determining the value of at least one parameter of a user's finger and presenting the position of the user's finger.

[0148] It is also possible to implement all four of the above-mentioned systems simultaneously, i.e. a system for dual control of a game, a system for displaying data in the form of a context menu, in particular video data, a system for determining the value of at least one parameter of a user's finger, and a system for determining and presenting the position of the user's finger on the display.

[0149] As used herein, an application refers to a computer program that, when launched on appropriate hardware, performs the actions or operations referred to herein. They refer to computer programs that contain instructions that, when executed by a computer, cause the computer to perform the appropriate operations. In the context of this invention, the above description of what an application is includes all applications referred to herein, i.e., those relating to dual control of a game, display of a context menu, and display of third and fourth applications.

[0150] In a sample use of the system according to the present invention, touch navigation is performed by contact with a navigation surface, particularly a touch-sensitive one, especially a phone or tablet, but this may also be a different surface covered with any touch sensor, or a non-touch-sensitive surface. The purpose of the navigation surface is to return information about the user's finger position and / or input and / or data and / or trigger action by finger means. When the surface is touch-sensitive, this is done similarly to the operation of a touchscreen on a mobile device or trackpad. For non-touch-sensitive surfaces, this is done based on measurements from system sensors. The purpose of the navigation surface is to function similar to touch control in mobile games, or a computer mouse, or an analog knob on a traditional pad. The player does not need to retract their thumbs and touch virtual buttons to trigger targeting or shooting actions. They can also use their left thumb to move their character and their right thumb to rotate the camera. At the same time, slightly pressing down on the left thumb triggers a targeting action, and slightly pressing down on the right thumb triggers a shooting action.

[0151] An example of usage could be a user wearing two wearable devices on the user's fingers, one on each thumb, and playing a "first-person shooter" type mobile game on the phone. Basic control is provided by the phone's native capabilities and the game's functions, with the left thumb being used to move the character by means of a virtual knob and the right thumb being used to rotate the camera by swiping a finger against the right part of the screen. They are also free to use actions assigned to virtual buttons located on the interface in the form of context menus. At the same time, the user can use additional inputs permitted by the wearable devices on the user's fingers.

[0152] Another use case might be a user playing a "first-person shooter" type game streamed from the cloud to a television set. These involve a phone and two devices worn on the user's fingers, one on each thumb. These three devices, along with an application installed on the phone, constitute the overall controller. The player transmits information about the tilt of the virtual knob and the movement of the fingers relative to the screen directly to the television set or to the device worn on the user's fingers. These control the character's movement and camera rotation in the application on the phone, which then transmits it, along with additional input provided by the player, to the television set via the device worn on the user's fingers. Controlling a game streamed to a television set requires more actions than simply moving the character and rotating the camera. For the external main action, the player can also perform other necessary actions, such as targeting and shooting.

[0153] The primary external input involves pressing a finger against the navigation surface. Several distinct inputs may, but need not, be distinguished depending on the angle at which the finger, particularly the distal phalanges, are pressed against the navigation surface, or depending on the specific part of the fingertip that is pressed against the navigation surface. Additional external inputs include selecting an action from a context menu. The menu is displayed on the navigation surface and / or on the external screen, preferably in a location corresponding to the position of one of the fingers. Selecting a location from the context menu involves swiping the finger or fingers displayed on the external screen toward the selected element by means of the finger and releasing it when the element is selected, or typically involves pressing the selected element. External spatial input involves performing a specific gesture in space with one or more fingers wearing a wearable device on the user's finger. This gesture may be performed without reference to another object in space and / or may be performed with respect to the navigation surface and / or a second or multiple devices wearable on the user's finger. The purpose of external input is to allow a single finger to perform more actions simultaneously without interrupting other actions, for example, to use a screen touch or control plane to control a pointer similar to a mouse pointer. Touching the screen or control plane with a finger and swiping across it controls the pointer displayed on the external screen. At the same time, the finger can also be pressed down firmly to generate a click.

[0154] The purpose of adding external inputs is to perform multiple actions simultaneously during the game without the need to use additional fingers. This solution also aims to improve ergonomics, efficiency, comfort, and the playing experience, for example, allowing a player to touch the screen with their right thumb to rotate the camera and start shooting without stopping the camera rotation even for a split second.

[0155] The feedback includes feedback of information about the position of the finger relative to the navigation surface, particularly via a phone or tablet, and also via another external navigation surface, such as a trackpad, or another navigation surface, particularly but not necessarily having a touch layer, including gathering data from motion sensors. The goal is to be able to precisely touch any point on the navigation surface without looking at it and / or one's hand. Instead, the only information used includes the finger's position, as presented by a visible pointer on an external screen, including being able to display additional information about a game, application, or system (e.g., an application icon) over which one or more fingers are positioned, similar to hovering a particular interface element with a mouse pointer, even when the pointer is not visible.

[0156] The primary purpose of the feedback is to display a pointer that indicates the position of a finger relative to a navigation surface, particularly a phone or tablet, allowing a player to easily tap interface elements of a game or application with their finger on the navigation surface, particularly a phone or tablet, without having to look at the navigation surface, particularly a phone or tablet. The pointer and interface elements may be displayed directly on the navigation surface, particularly a mobile device, or may be displayed only on an external screen. Hovering over an interface element with a pointer (visible or hidden) can cause additional information to be displayed.

[0157] In another use of the present invention, a user is wearing two devices wearable on the user's fingers, one for each thumb on each hand, and is playing a "first-person shooter" type mobile game on the phone, with the images displayed on the phone being transmitted to a television set. A pointer displayed on the phone screen, or directly on the television set screen, indicates the position of the thumb on the phone screen. This allows the player to look only at the television set and still use their thumb to hit virtual buttons displayed on the interface on the phone without having to look at the phone or their hands.

[0158] Another usage example is where a user plays a "first-person shooter" type game streamed from the cloud to a television set. They use a phone and two devices that can be worn on the user's fingers, one on each thumb. These three devices, along with an application installed on the phone, make up the entire controller. Aside from transmitting information about knob tilt or camera rotation, the phone also serves as a reference point for displaying a pointer on an external navigation surface and on the television set. The application transforms the entire phone screen, or a portion of it, into a space that corresponds to the ratio of the television set's screen.

[0159] One additional use of the system for determining and presenting the user's finger position on the display is in the form of so-called feedback, i.e., in the form of a pointer visible on the external screen. The pointer's position corresponds to the thumb's position on the phone's screen. This allows the player to touch selected buttons on the interface without looking at their hands, focusing only on the external screen. Another use is the use of additional input in the form of one or more buttons located on the fingertips, allowing the player to simultaneously perform several primary actions using only their thumb. Input is further supported by context menus, which allow for the execution of even more side actions.

[0160] Ways to achieve additional input: a) Mechanical Button Switch One or more mechanical buttons of any type are placed on the fingertip. The controller element that abuts the fingertip may be flexible and / or replaceable to allow for proper sizing to fit a person's finger or to allow replacement when worn, or may be fixedly attached to the rest of the device. b) Pressure sensor One or more pressure sensors of any type are attached to the fingertip, each capable of transmitting one or more inputs in response to applied pressure. The controller element that abuts the fingertip may be flexible and / or replaceable to allow for proper sizing to fit a person's finger or to allow replacement when worn, or may be fixedly attached to the rest of the device. c) Touch sensor The fingertip is fitted with one or more touch sensors of any kind, and the controller element that abuts the fingertip may be flexible and / or replaceable to allow for proper sizing to fit a person's finger or to allow for replacement when worn, or it may be fixedly attached to the rest of the device. d) Motion Sensor This could be a simple gyroscope, a gyroscope with an accelerometer, or an entire set of sensors constituting a motion sensor placed anywhere on the finger. When performing a tap / click, the finger performs a specific micro-movement. Collecting data from the motion sensor allows this movement / gesture to be distinguished from a regular touch and swipe across the screen. The tilt angle of the distal phalanges relative to the navigation surface, particularly but not exclusively, allows motion sensors, i.e., movement sensors placed on the distal phalanges, to distinguish different types of inputs. For example, Input A is a flat finger placed parallel to the navigation surface and pressing it with the distal phalanges straight. Input B is a finger bent at a 45-degree angle. Input C is a finger bent at a 90-degree angle relative to the navigation surface and pressing with the tip of the finger (the part of the fingertip just below the nail). Preferably, the movement sensor is further supported by machine learning to achieve better accuracy. e) Motion sensor + heart rate monitor / EMG A heart rate monitor is added to the solution from point d to receive additional data. f) Optical sensor It refers to an optical sensor that measures finger tension via infrared light to obtain information about pressure.

Claims

1. An application for controlling a video game, configured to receive information from a device including a touch layer and data from an additional device for inputting data, and to control the progress of the game in accordance with this information and data.

2. 2. The application according to claim 1, further configured to transmit data, in particular video data, to a display or to a device comprising a display, wherein the display has the form of a monitor, a smart TV, a projector, a VR or AR device, a smartphone display or a tablet display, in particular when the data input device is a game controller, a gamepad, a joystick or a device wearable on a user's finger, and / or to an external device.

3. 1. A system for controlling a video game, comprising: a first control device for determining the position of a user's finger on the touch layer, the first control device comprising a processor, a memory, and an application according to claim 1 or 2 for controlling the video game; a device having a touch layer, the device determining the position of a user's finger on the touch layer and transmitting this information to the first controller; a display; said first control device is configured to receive said information from said device with the touch layer and to input data from said further device, to control the progress of the game depending on said information and these data, and to transmit said data, in particular video data, to said display; The display device is configured to receive data, particularly video data, from the first control device and to display a video image based thereon.

4. 4. The system according to claim 3, characterized in that the first control device, the device with a touch layer and the display are a single integrated device, in particular a smartphone or a tablet.

5. 5. The system according to claim 3, wherein the device including the touch layer is a smartphone or a tablet, and the touch layer is the screen of the smartphone or the tablet.

6. 6. The system of claim 3, 4 or 5, characterized in that the first control device is a smartphone, a tablet, a smart TV or a video game console.

7. 7. The system according to claim 3, wherein the additional device for inputting data is a game controller, a gamepad, a joystick, or a device that can be worn on a user's finger.

8. 8. The system according to claim 3, wherein the device with a touch layer is configured to transmit information about the position of the user's finger and / or the input and / or the data and / or the trigger action input by means of the finger on the touch layer directly to the first control device.

9. 8. The system according to claim 3, wherein the device with a touch layer is configured to transmit information about the position of the user's finger and / or the input and / or the data and / or the triggered action input by means of the finger on the touch layer to the first control device via the additional device for inputting data.

10. 10. The system according to any one of claims 3 to 9, characterized in that the additional device for inputting data is configured to transmit said data directly to the first control device.

11. 10. The system according to claim 3, wherein the additional device for inputting data is configured to transmit the data to the first control device via the device comprising a touch layer.

12. The system according to claims 3 to 11, characterized in that the display is configured to receive additional data, in particular video data, from an additional external source, for example a video game console, a smartphone, a tablet, or to receive video streaming from the Internet, and to display video images based thereon.

13. 13. The system according to any one of claims 3 to 12, characterized in that the display has the form of a monitor, a smart TV, a projector or a VR or AR device, a smartphone screen or a tablet screen.

14. System according to any one of claims 3 to 13, characterized in that the touch layer and / or the additional device for inputting data and / or the first control device and / or the device comprising the display are configured for wireless communication, preferably via Bluetooth and / or Wi-Fi and / or Near Field Communication NFC and / or Li-Fi.

15. An application for generating virtual buttons, the application configured to display the virtual buttons on an application or game.

16. 16. The application according to claim 15, characterized in that the virtual button has the form of a context menu and is adapted to receive information from the device having a touch layer, possibly comprising data from the device, and possibly data from the additional device, and to generate data in the form of a context menu, in particular video data, that depend on this information and these data.

17. The application, wherein the virtual buttons are additionally configured to be assignable to selected actions of a game or physical buttons on a game controller, gamepad, mouse buttons, or keyboard keys.

18. 18. The application according to any one of claims 15 to 17, further comprising: a means for transmitting information about the activation button to a display or an additional device for inputting data.

19. 15. The application according to claim 14, further configured to transmit data, in particular video data, to the display.

20. 1. A system for displaying virtual buttons, comprising: a second control device comprising a processor, a memory and an application for generating virtual buttons according to claims 15 to 19; Optionally, a system for controlling a video game according to claims 3 to 14, a device comprising a touch layer, the touch layer configured to determine a position of a user's finger on the touch layer and to transmit information regarding the position to a second controller; Optionally, an additional device configured to transmit data to a second controller; The display and The second control device receives the information of the device with a touch layer, possibly data from an additional device, and generates data, in particular video data, in the form of a context menu based on this information and data, and transmits this data, in particular video data, to a display, which is configured to receive the data, in particular video data, from the second control device and display a video image based thereon, for displaying virtual buttons.

21. 21. The system of claim 20, wherein the context menu includes any number of buttons distributed in a circle around a location corresponding to the position of the user's finger as determined by the device including the touch layer.

22. 22. A system according to claim 20 or 21, wherein the button is configured, after activation, to transmit data to an external device, in particular an additional device for inputting data, and these data are transmitted to the second control device, in particular a tablet, phone or television, as information about pressing a physical button corresponding to a selected button of a game controller or a keyboard and / or a mouse.

23. 23. The system according to any of claims 20 to 22, characterized in that the device comprising a touch layer is a smartphone or a tablet, the touch layer being the screen of the smartphone or tablet.

24. The system according to any one of claims 20 to 23, characterized in that the second control device is a smartphone, a tablet, a smart TV or a video game console.

25. 25. The system according to any one of claims 20 to 24, characterized in that the additional device for inputting data is a game controller, a gamepad, a joystick, or a device that can be worn on a user's finger.

26. 26. The system according to any one of claims 20 to 25, characterized in that the display is configured to receive additional data, in particular video data, from additional external sources, such as video game consoles, smartphones, tablets, or video streaming from the Internet, and to display video images based thereon.

27. 27. The system according to any one of claims 20 to 26, characterized in that the display has the form of a monitor, a smart TV, a projector or a VR or AR device, a smartphone screen or a tablet screen.

28. System according to any one of claims 20 to 27, characterized in that the touch layer and / or the additional device for inputting data and / or the second control device and / or the device comprising the display are configured to communicate wirelessly, preferably via Bluetooth and / or Wi-Fi and / or Near Field Communication NFC and / or Li-Fi.

29. 1. A system for determining a value of at least one parameter of a user's finger, comprising: a device wearable on a user's finger; at least one sensor disposed in or on the device wearable on the user's finger; a third control device comprising a processor, a memory, and a third control application; Optionally, a system for controlling a video game according to claims 3 to 14; Optionally, a system for displaying virtual buttons according to claims 20 to 28; the at least one sensor is configured to send a signal to the third controller; a configuration for determining a value of at least one parameter of the user's finger, configured to determine the value of the at least one parameter of the user's finger and to transmit information about the determined value of the at least one parameter of the user's finger to the third control device; The system, wherein the third controller is configured to receive information about a determined value of at least one parameter of the user's finger.

30. 30. The system according to claim 29, characterized in that the sensor has the form of a motion sensor and / or a pressure sensor and / or an optical sensor and / or a proximity sensor.

31. 31. A system according to claim 29 or 30, further comprising a navigation surface, in particular in the form of a touch layer, in particular in the form of a touchscreen of a smartphone or tablet, or a touchpad or trackpad.

32. 32. The system of claim 31, wherein the navigation surface is configured to determine a value of at least one parameter of the user's finger, and wherein the navigation surface is configured to transmit the value of the at least one parameter of the user's finger to the third control device.

33. 32. The system according to any one of claims 29 to 31, characterized in that it is configured to determine the value of a parameter of the user's finger, namely the angle between the distal phalange and the proximal phalange.

34. 34. The system according to claim 29, 30, 31 or 33, characterized in that it is configured to determine the value of a parameter of the user's finger, i.e. the pressure exerted by the finger on the sensor.

35. 35. The system according to claim 29, 30, 31, 33 or 34, characterized in that it is configured to determine values ​​of parameters of the user's finger, i.e. the pressure exerted by the finger on the navigation surface and / or the angle between the distal and proximal phalanges.

36. 32. The system of claim 31, configured to determine a value of a parameter of the user's finger, i.e., the pressure exerted by the finger on the navigation surface.

37. The system according to any one of claims 29 to 36, further comprising a display.

38. 38. The system of claim 37, wherein the display is a projector, a monitor, a smart TV, a VR or AR device, a smartphone screen or a tablet screen.

39. 39. The system of claim 37 or 38, wherein the display is configured to receive data from the third control device and display an image based thereon.

40. A system according to any one of claims 37 to 39, characterized in that the display is configured to receive additional data, in particular video data from an additional external source, for example a video game console, a smartphone, a tablet, or to receive video streaming from the internet, and to display video images based thereon, including an image according to claim 39.

41. The system according to any one of claims 29 to 40, characterized in that the third control device is configured to perform an action based on the value of a parameter of the user's finger.

42. The system according to any one of claims 29 to 41, characterized in that the third control device is a tablet or a smartphone.

43. System according to any one of claims 29 to 42, characterized in that the third control device is integrated with the display, in particular the third control device is a smart TV.

44. A system according to any one of claims 29 to 43, characterized in that the third control device is a video game console.

45. 45. The system of any one of claims 29 to 44, characterized in that the device wearable on the user's finger has the form of a clip intended to be placed on the user's finger and has on top, or on the side, or on an open or closed ring with a sensor.

46. 46. ​​The system of any one of claims 29 to 45, wherein the device wearable on the user's finger comprises a sensor adapted to measure the angle between the distal phalanges and the proximal phalanges of the user's finger.

47. 47. The system of claim 29, wherein the device wearable on the user's finger comprises a motion sensor adapted to measure the pressure exerted by the finger on the navigation surface and to measure position and angular differences between the finger and the navigation surface.

48. 48. The system of any one of claims 29 to 47, wherein the device wearable on the user's finger comprises a pressure sensor and / or an optical sensor adapted to measure the pressure exerted by the finger on the navigation surface.

49. 49. The system of claim 29, wherein the device wearable on the user's finger has an additional portion extending over the distal phalanx of the finger, the additional portion having a motion sensor located thereon.

50. 50. The system of any one of claims 29 to 49, characterized in that the device wearable on the user's finger comprises a button under the fingertip, in particular a pressure sensor configured to generate and transmit a signal to the third control device when pressed.

51. System according to any one of claims 29 to 50, characterised in that the third control device and / or the device for determining the value of at least one parameter of the user's finger and / or the navigation surface and / or the display are configured to communicate wirelessly, preferably via Bluetooth and / or Wi-Fi and / or Near Field Communication NFC and / or Li-Fi.

52. A system according to any one of claims 29 to 51, characterized in that the user is configured to define a part of the surface of the touch layer in the form of a touchscreen of a smartphone or tablet, thereby constituting a navigation surface according to claim 31.

53. 1. A system for determining and presenting on a display the position of a user's finger, comprising: a fourth control device configured to determine a position of a finger on the navigation surface, the fourth control device comprising a processor, a memory, and a fourth control application; The display and a navigation surface, the processor configured to transmit information regarding its position to the fourth control device; The display and Optionally, a system for controlling a video game according to claims 3 to 14; Optionally, a system for displaying virtual buttons according to claims 20 to 28; Optionally, a system for determining a value of at least one parameter of a finger according to claims 29 to 52, and an arrangement for determining a position of a user's finger on a navigation surface, the arrangement comprises at least one sensor configured to determine the position of a user's finger and to transmit information about the determined position to the fourth control device; the fourth control device is configured to receive the information from the arrangement for determining the position of a user's finger and to transmit data, in particular video data, to the display, the data, in particular video data, comprising a finger position indicator corresponding to the position of the user's finger on the navigation surface; The system wherein the display is configured to receive data, particularly video data, from a fourth control device and display a video image based thereon including a finger position indicator.

54. 54. The system of claim 53, wherein the fourth control device is a tablet, a smartphone, or a video game console.

55. 55. The system according to claim 53 or 54, characterized in that the fourth control device is integrated with the display, in particular the fourth control device is a smart TV.

56. System according to any one of claims 53 to 45, characterised in that the display is in the form of a monitor, a smart TV, a projector or a VR or AR device, or has the screen of a smartphone or the screen of a tablet.

57. A system according to any one of claims 53 to 56, characterized in that the display is configured to receive additional data, in particular video data from an additional external source, for example a video game console, a smartphone, a tablet, or to receive video streaming from the internet, and is configured to display video images based thereon, including an image according to claim 53.

58. System according to any one of claims 43 to 57, characterized in that the navigation surface is a touch layer, in particular the surface of a touchscreen of a tablet or smartphone.

59. 59. A system according to claim 58, characterized in that the system is configured for the user to define a portion of the surface of the touch layer in the form of a touch screen of a smartphone or tablet constituting the navigation surface according to claim 58.

60. The arrangement for determining the position of the user's finger on the navigation surface.

60. The system of any one of claims 53 to 59, adapted to be placed on at least one of the user's fingers, the arrangement comprising an accelerometer and / or a gyroscope, and configured to use data from the accelerometer and / or gyroscope to determine in real time the position of the user's finger on the navigation surface.

61. 61. The system of claim 60, wherein the configuration for determining a position of the user's finger on the navigation surface further comprises an optical sensor configured to assist in determining the position of the user's finger on the navigation surface by determining the finger flexion level by determining an angle between the distal phalange and the proximal phalange.

62. A system according to any of claims 53 to 60, characterised in that the fourth control device and / or the navigation surface and / or the arrangement for determining the position of the user's finger on the navigation surface and / or the display are configured to communicate wirelessly, preferably via Bluetooth and / or Wi-Fi and / or Near Field Communication NFC and / or Li-Fi.

63. 63. A system according to any one of claims 60 to 62, characterized in that it is configured to recognize the gestures performed by the fingers of the user across and / or on the navigation surface and / or in the form of a particular configuration of one or more of the user's fingers.

64. A system according to any one of claims 3 to 63, wherein the first control device and / or the second control device and / or the third control device and / or the fourth control device are the same device.

Images