Interactive illuminated floor

Abstract

The invention relates to the field of computing. An interactive illuminated floor made up of an array of panels arranged in a grid comprises: printed circuit boards with arrays of LEDs, designed to be capable of illuminating each panel of the floor array independently, and capacitive sensors and strain gauge weight sensors designed to be capable of detecting the presence of a participant on any panel of the floor array. The illuminated floor comprises a game microcomputer which is connected to the panel array and to the sensors and is designed to be capable of sending illumination control signals to the arrays of LEDs. Prior to commencement of a game process, the game microcomputer records the weight of each participant, and during the game process, it tracks panel depression time, depression weight and pixel depression coordinates. Using a first trained neural network, the microcomputer tracks the movement of a given participant across the game field and the execution of a set sequence of the game process. Using a second trained neural network, the microcomputer identifies instances of incorrect depression of a panel. The technical result is that of expanding the existing range of technical means by improving the process of tracking the movement of a given participant across the game field and identifying instances of incorrect depression of a panel.

Description

[0001] INTERACTIVE LIGHT FLOOR

[0002] AREA OF TECHNOLOGY

[0003] This technical solution relates to the field of computer technology, in particular to the interactive light floor.

[0004] LEVEL OF TECHNOLOGY

[0005] An interactive floor is a system for displaying graphic images on any surface that responds to human movement. It is used in educational settings, to create eye-catching presentations, to display next-generation advertising, and to decorate dance halls, exhibition pavilions, commercial facilities, and shopping malls.

[0006] Known in the prior art is patent RU93428U1, Shcherbakov Alexander Ivanovich (RU), published 27.04.2010. This solution describes a dance floor comprising an upper deck and a lower deck, which is secured to the floor base, characterized in that a layer of mirror plastic is introduced, installed on the lower deck with the reflective surface facing up and in which a group of light-emitting diodes with a power supply and a controller are secured, a grid of plexiglass installed on the layer of mirror plastic, a layer of transparent plexiglass installed on the plexiglass grid, and a layer of mirror film secured with the reflective surface facing down between the layer of transparent plexiglass and the upper deck made of tempered glass.

[0007] In addition, the prior art discloses patent RU2669534C2, UNIVERSAL CITY STUDIOS LLC (US), published 11.10.2018, which describes a system including a surface that displays a plurality of images related to a game, wherein the vehicle comprises an interface circuit configured to receive input from a rider related to the path of the vehicle along the surface, wherein the vehicle operates in accordance with the input to move along the surface along the path of the vehicle, and a controller that determines that the vehicle has moved over a first image from a plurality of images along the path of the vehicle based on a signal from the vehicle, the surface, an external sensor, or a combination thereof; issues commands to the display circuit associated with the surface to change the first image when the vehicle has moved over the first image along the path of the vehicle;and updates the score associated with the vehicle when the vehicle passes over the first image in the vehicle's path. A common drawback of existing solutions in this area is their limited functionality, as they lack the ability to use both capacitive and strain gauge load cells simultaneously. This would provide more reliable operation due to redundancy in the event of a sensor failure, and also expand the scope of application by providing information on the applied force from the load cells. Furthermore, the above solutions do not utilize neural networks.

[0008] The proposed technical solution addresses the shortcomings of current technology and differs from existing solutions in that it ensures more reliable operation of the interactive light floor by providing redundancy in the event of a sensor failure. Furthermore, a capacitive sensor allows for the detection of pressure at a short distance from the glass surface without actual contact, while a strain gauge sensor requires force and provides information on the force applied. Furthermore, the proposed solution focuses on an immersive experience utilizing a new interactive floor environment and associated gaming techniques using at least one neural network.

[0009] ESSENCE OF THE INVENTION

[0010] The technical problem addressed by the proposed solution is the development of a new interactive light floor. This solution utilizes machine learning algorithms to improve the process of tracking a specific participant's movements across the playing field and detecting false key presses on the panel.

[0011] The technical result consists in expanding the arsenal of technical means. The proposed solution is a game room or platform of arbitrary size, covered with sensitive LED panels, and contains a gaming microcomputer that controls various game scenarios requiring participants to move around the room / platform, press the panels with their feet, or, conversely, dodge oncoming "lava." This technical result is achieved thanks to an interactive light floor, made from an array of panels laid out in at least one grid, wherein the panels of each grid lie at a common equal height in a common coplanar relationship with one another to jointly define the corresponding areas of the interactive floor surface and comprising: - printed circuit boards with LED matrices, configured to independently illuminate each panel of the array;

[0012] - capacitive and strain gauge weight sensors, designed with the ability to detect the presence of a participant on any panel of the array;

[0013] - a gaming microcomputer connected to an array of panels and their sensors and configured to send lighting control signals to the LED matrices to change the lighting states of the array panels and receive detection signals from a system of sensors indicating the detected presence of participants on the array panels, wherein the gaming microcomputer, before the start of the game process, registers the weight of each participant through information received from the sensors and, during the game process, tracks the time of pressing the panel, the weight of the press and the coordinates of the pixel press, and, using the first trained neural network (NN), tracks the movements of a specific participant across the playing field and the successful completion of a given sequence of the game process, and, using the second trained NN, determines false presses on the panel.

[0014] In a particular embodiment of the described solution, capacitive sensors detect the presence of a participant on any panel of the array by registering a press at a distance from the surface of the panel of the array without actually touching it.

[0015] In another particular embodiment of the described solution, strain gauge weight sensors detect the presence of a participant on any panel of the array by registering the force of pressure or weight on the surface of the panel.

[0016] In another particular embodiment of the described solution, the gaming microcomputer is designed with the ability to, using a second neural network, turn off non-working pixels without interrupting the gaming process.

[0017] BRIEF DESCRIPTION OF DRAWINGS

[0018] The features and advantages of the present technical solution will become apparent from the detailed description below and the attached drawings.

[0019] Fig. 1 - illustrates the circuit diagram of an interactive LED cell.

[0020] Fig. 2 - illustrates a diagram of an interactive LED cell with strain gauges made in the form of a frame along the contour of the glass.

[0021] Fig. 3 - illustrates the diagram of the movement of participants around the game room and the analysis of this information using the game microcomputer

[0022] DETAILED DESCRIPTION OF THE INVENTION The following detailed description of the invention includes numerous implementation details to provide a clear understanding of the present invention. However, one skilled in the art will readily appreciate how the present invention may be used with or without these implementation details. In other instances, well-known methods, procedures, and components have not been described in detail to avoid unnecessarily obscuring the features of the present invention.

[0023] Furthermore, it will be clear from the foregoing description that the invention is not limited to the embodiment described. Numerous possible modifications, changes, variations, and substitutions, while preserving the spirit and form of the present invention, will be apparent to those skilled in the art.

[0024] The claimed interactive light floor is made from an array of panels laid out in at least one grid, wherein the panels of each grid lie at a common equal height in a common coplanar relationship with each other in order to jointly define the corresponding areas of the interactive floor surface. The interactive LED platform (pixels - 30x30 cm with pressure sensors), which completely covers the floor of a playroom measuring from 1 to 100 m 2 , as well as buttons located on the walls.

[0025] For example, an immersive entertainment environment could be represented by an enclosed game room with a vertical perimeter wall structure consisting of a front wall, an opposite rear wall, a first side wall, and an opposite second side wall. The front wall has both an entrance to the room and a separate exit, for example, near the respective ends of the front wall. Alternatively, the game room could have a common entrance / exit through which the interior space of the game room is both entered and exited. In separate entrance / exit configurations, they do not necessarily need to be located on the same wall.

[0026] The perimeter wall structure defines the room's floor space, a significant portion of which is occupied by an interactive floor system defining an array of rectangular cells that, taken together, encompass a substantially complete dimension of the room's interior space in each of its two horizontal dimensions. Thus, the cellular array largely encompasses both the room's width, measured perpendicularly between two opposing side walls, and the room's depth, measured perpendicularly between opposing front and back walls. Accordingly, upon entering the room, the gamer is essentially forced to stand on the interactive floor, thereby ensuring mandatory interaction with it during gameplay sessions within the room, such as "The Floor is Lava!", "Hopscotch," "The Sea is Worried," "Dancing," "Base Defense," and so on.

[0027] In addition to the interactive floor, the room also has one or more interactive target devices mounted in elevated positions above the interactive floor. As a result, gameplay within the room can include both physical interaction of the feet with the interactive floor cells (stepping, jumping, tapping, etc.) and physical interaction of the hands or other upper body parts with individual target devices. Preferably, multiple target devices are distributed throughout the room. Target devices can be provided as wall-mounted target devices, with at least one on each perimeter wall of the room.Each target device may be a push-button device, a touchpad, or any other electronic device capable of receiving physical input indicating the activation of said device, preferably through a hand press or other upper body interaction of the participant, although participants may optionally use contact with a raised leg, knee, hip, or other body part.

[0028] In addition to the target devices, the interior of the gaming space also includes at least one display operative to provide feedback to the participant(s) during the gaming session. The display(s) may, for example, show at least one of: a scoreboard with increasing / decreasing score values ​​upon successful completion or failure of assigned tasks during gameplay; a running timer; and / or a status counter whose level increases or decreases in response to detected failures, similar to a life bar or health counter in a video game.

[0029] Moreover, the interactive light floor contains printed circuit boards with LED matrices, designed with the ability to independently illuminate each panel of the array;

[0030] - Capacitive and strain gauge load sensors capable of detecting the presence of a participant on any panel in the array. Using both capacitive and strain gauge load sensors ensures more reliable operation of the interactive light floor due to redundancy in the event of failure of one sensor. Furthermore, it offers a new user experience, as the capacitive sensor is highly sensitive and registers pressure at a distance from the glass surface without actually touching it, while the strain gauge requires a force that adjusts to the age of the participant. By combining the two sensor types, it is possible to achieve reliable operation and an improved user experience when interacting with the floor.It allows for varying touch force settings on individual pixels: up to 50 kg on one, and even no touch on another, opening up new possibilities for gameplay and enabling new game mechanics for athletes requiring different interactions with pixels.

[0031] Simultaneous and independent sensor polling is achieved through a microcontroller embedded in each panel (see Fig. 1-2). Each microcontroller polls its panel's sensors up to 80 times per second and sends information about keystrokes and their force to the game microcomputer.

[0032] A gaming microcomputer connected to the array of panels and their sensors and configured to send lighting control signals to the LED matrices to change the lighting states of the array panels and to receive detection signals from the sensor system indicating the detected presence of participants on the array panels.

[0033] In this case, before the start of the game process, the microcomputer registers the weight of each participant through information coming from the sensors and during the game process, it tracks the time of pressing the panel, the weight of the press and the coordinates of the pixel press, and with the help of the first trained neural network (NN), it tracks the movements of a specific participant on the playing field and the successful completion of a given sequence of the game process, with the help of the second trained NN, it determines false presses on the panel.

[0034] Before each game session begins, each participant's weight is recorded. During the game, each participant's click data is recorded, representing their weight. After each click, information is received through the sensors, and it is necessary to determine which participant performed the click. This task boils down to a classic classification problem using the KNN (k-nearest neighbors) algorithm. The main challenge is creating a similarity metric between all participants. Each participant will be represented by a vector of values ​​(click time, click weight, click pixel) - (t, w, x, y), where n is the click number.

[0035] We'll also factor in the values ​​for the last three clicks in the metric. The weight of each click will vary for each user, and distance (pixels) and time must be taken into account.

[0036] Once the value vector for each participant has been determined, it must be classified. Initial class values ​​(participant numbers) are available at the start. Next, for each keystroke vector, the closest classes (participant numbers) from history are compared using the KNN algorithm to determine the current keystroke. After the current keystroke, the values ​​are entered into the database for subsequent classification.

[0037] This algorithm runs in real time on a Raspberry Pi 4.21 and is ready to process clicks per second. The project technologies are Python, NumPy, and skleam (KNN).

[0038] In addition, the microcomputer is designed with the ability to use the second NS to turn off non-working pixels without interrupting the game process.

[0039] A common situation is that after a pixel is clicked, it begins to signal a click every second, even though no clicks have actually occurred. It also sometimes happens that neighboring pixels begin to blink as well. Therefore, the challenge is to determine whether this was a human click or a system glitch. To address this, the Outlier Detection method is proposed. This task requires collecting a dataset of clicks and classifying them into two classes: human and false. The features used to train the model include: click duration, click weight, and pixel coordinates.

[0040] The number of training examples is 3,000-5,000. This number of keystrokes, with varying weights and durations, will take 3-6 months. Therefore, we will use data augmentation methods.

[0041] Dataset collection is as follows. Data collection is performed with different weights (10-20 people). Then, using linear and polynomial interpolation, samples are generated for another 200 people.

[0042] After this, positive examples are given for training (where the person is).

[0043] The model, based on positive examples, learns patterns and dependencies in the data. When an input example deviates from the "standard," it will mark the large / small weight as an outlier.

[0044] The model's performance is 25 clicks per second, allowing it to run in real time on a Raspberry Pi 4 or similar microcomputer. The project's technologies include Python, NumPy, and scikit-learn (KNN).

[0045] In these application materials, a preferred disclosure of the implementation of the claimed technical solution was presented, which should not be used as limiting other, particular embodiments of its implementation that do not go beyond the scope of the requested scope of legal protection and are obvious to specialists in the relevant field of technology.

Claims

CLAUSES OF THE INVENTION 1. An interactive light floor made from an array of panels laid out in at least one grid, wherein the panels of each grid lie at a common equal height in a common coplanar relationship with each other in order to jointly define corresponding areas of the interactive floor surface and comprising: - printed circuit boards with LED matrices, designed with the ability to independently illuminate each panel of the array; - capacitive and strain gauge weight sensors, designed with the ability to detect the presence of a participant on any panel of the array; - a gaming microcomputer connected to the array of panels and their sensors and configured to send lighting control signals to the LED matrices to change the lighting states of the array panels and receive detection signals from a system of sensors indicating the detected presence of participants on the array panels, wherein the gaming microcomputer, before the start of the game process, registers the weight of each participant through information received from the sensors and, during the game process, tracks the time of pressing the panel, the weight of the press and the coordinates of the pixel press, and, using the first trained neural network (NN), tracks the movements of a specific participant across the playing field and the successful completion of a given sequence of the game process, and, using the second trained NN, determines false presses on the panel.

2. An interactive light floor according to claim 1, characterized in that the capacitive sensors detect the presence of a participant on any panel of the array by registering a press at a distance from the surface of the panel of the array without actual contact.

3. An interactive light floor according to claim 1, characterized in that the strain gauge weight sensors detect the presence of a participant on any panel of the array by registering the force of pressure or weight on the surface of the panel.

4. An interactive light floor according to claim 1, characterized in that the gaming microcomputer is designed with the ability to, using a second neural network, turn off non-working pixels without interrupting the gaming process.

Images