Somatosensory networked game method, device and computer-readable storage medium

By using convolutional neural network on the server side to identify the spectrum grayscale map provided by the client, and generate game operation instructions, the problem of different computing power of somatosensory games when interconnecting multiple terminals is solved, and the unified experience and high smoothness of somatosensory games in wide-area networks is achieved. sex.

CN115006840BActive Publication Date: 2024-11-01SHENZHEN SHIMI NETWORK TECH CO LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202210697254.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-06-20
Publication Date
2024-11-01
Estimated Expiration
2042-06-20

AI Technical Summary

Technical Problem

When existing somatosensory games are interconnected by multiple terminals, the difference in computing power results in inconsistent game screens, affecting user experience, and it is difficult to realize efficient somatosensory games on the wide area network.

Method used

The client obtains the player's posture data, generates a spectrum grayscale map and imports a preset convolutional neural network, recognizes the player's actions and generates game operation instructions, and the server issues instructions to the client to achieve action matching and eliminates calculations to the client. Force requirements and equipment model restrictions.

Benefits of technology

It realizes somatosensory networked games for different models of terminals, reduces the computing power requirements of the client, improves the game fluency and action recognition accuracy, and ensures the consistency of the game experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115006840B_ABST
    Figure CN115006840B_ABST
Patent Text Reader

Abstract

The present invention discloses a somatosensory networked game method, device and computer-readable storage medium, the method comprising: after the client starts a preset somatosensory game, obtaining the player's original posture data from the client; generating a spectrum grayscale image according to the original posture data; importing the spectrum grayscale image into a preset convolutional neural network to identify the player's actions; generating game operation instructions according to the recognition results of the preset convolutional neural network; and sending the game operation instructions to the client. The method has the advantages of being able to realize somatosensory networked games, having low requirements on the client's computing power, high accuracy in identifying player actions, and good game fluency.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of somatosensory gaming technology, and in particular to a somatosensory networked gaming method, device, and computer-readable storage medium. Background Art

[0002] At present, motion sensing games are usually run on local terminals, relying on local terminals to recognize user actions to generate game operation instructions. Among them, the difference in terminal computing power will affect the accuracy and speed of user action recognition. Due to the different computing power of different terminals, if multiple terminals are interconnected for motion sensing networked games, the screen displayed on each terminal may be inconsistent, seriously affecting the user's gaming experience.

[0003] Therefore, there is an urgent need to provide a somatosensory networked gaming method to facilitate users to play somatosensory games on a wide area network. Summary of the invention

[0004] The embodiment of the present application provides a somatosensory networked gaming method, which aims to implement somatosensory gaming on a wide area network and expand the user's gaming methods.

[0005] To achieve the above objectives, the present application provides a method for playing a somatosensory online game, including:

[0006] After the client starts the preset somatosensory game, the player's original posture data is obtained from the client;

[0007] Generate a spectrum grayscale image according to the original posture data;

[0008] Importing the spectrum grayscale image into a preset convolutional neural network to recognize player actions;

[0009] Generate game operation instructions according to the recognition result of the preset convolutional neural network;

[0010] The game operation instruction is sent to the client.

[0011] In one embodiment, generating a spectrum grayscale image according to the original posture data includes:

[0012] Generate a spatial posture trajectory of the player's action according to the player's posture raw data;

[0013] generating a waveform graph according to the spatial posture trajectory;

[0014] Performing discrete Fourier transform on the original sampling data in the waveform diagram to obtain spectrum data;

[0015] Merging the spectrum data according to the time domain information to obtain a two-dimensional spectrum waterfall diagram;

[0016] The element values ​​in the two-dimensional spectrum waterfall diagram are mapped according to the image grayscale values ​​to obtain the spectrum grayscale diagram.

[0017] In one embodiment, the original sampled data in the waveform diagram includes sampled data of channel I and sampled data of channel Q;

[0018] Before performing discrete Fourier transform on the original sampled data in the waveform, the method further includes:

[0019] The sampled data of channel I and channel Q are combined into channel S according to the following formula:

[0020]

[0021] In the formula, S I is the sampling data of channel I, S Q is the sampling data of channel Q, and S is the combined power density.

[0022] In one embodiment, the discrete Fourier transform of the original sampled data in the waveform is performed using the following formula:

[0023]

[0024] Wherein, N is the number of sampling points when performing discrete Fourier transform, and the value of N is limited to a positive integer power of 2.

[0025] In one embodiment, obtaining the player's raw posture data from the client includes:

[0026] The player's original posture data matching the preset target axis of the current somatosensory game is obtained from the client.

[0027] In one embodiment, before the spectrogram is introduced into a preset convolutional neural network to identify the player's actions, the method further includes:

[0028] Collecting the posture data of the standard somatosensory movements of the players completing the preset somatosensory game to establish a preset data set;

[0029] Obtaining a training spectrum waterfall diagram of the preset data set;

[0030] Determining a preset target axis of the preset somatosensory game according to the waveform diagram of the preset data set; and

[0031] The convolutional neural network is trained using the training spectrum data graph of the preset target axis.

[0032] In one embodiment, the raw posture data obtained from the client is compressed using a Varints algorithm.

[0033] In one embodiment, sending the game operation instruction to the client includes:

[0034] When the gateway node information bound to the client is inconsistent with the distribution gateway node of the current server, publishing the game operation instruction to the target gateway node bound to the client;

[0035] The game operation instruction is sent to the client through the target gateway node.

[0036] To achieve the above objectives, an embodiment of the present application also proposes a somatosensory networked game device, comprising a memory, a processor, and a somatosensory networked game program stored in the memory and executable on the processor, wherein the processor implements a somatosensory networked game method as described in any one of the above items when executing the somatosensory networked game program.

[0037] To achieve the above objectives, an embodiment of the present application further proposes a computer-readable storage medium, on which a somatosensory networked game program is stored. When the somatosensory networked game program is executed by a processor, the somatosensory networked game method as described in any one of the above items is implemented.

[0038] In the somatosensory networked game method of the technical solution of the present application, the server can convert the player posture data provided by the client into a spectrum grayscale image, and then classify the spectrum grayscale image through a preset convolutional neural network to identify the player's action, and then generate game operation instructions according to the player's action, and send the game operation instructions to the client, and the client can execute the game operation instructions to match the game character action with the player's action. In this way, during the entire game process, the client only performs data exchange and execution of operation instructions. Therefore, there is no special limitation on the model of the client, so that clients of different models and types can all play somatosensory networked games. In addition, the player's action recognition is performed on the server, so the computing power requirements of the client are extremely low, eliminating the action recognition delay caused by the inconsistent computing power between different devices, and improving the fluency of the somatosensory networked game. It can be seen that the somatosensory networked game method of the technical solution of the present application has the advantages of being able to realize somatosensory networked games, and having low requirements for client computing power, high accuracy in player action recognition, and good game fluency. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0040] Figure 1This is a module structure diagram of an embodiment of a somatosensory networked game device of the present invention;

[0041] Figure 2 A schematic diagram of a flow chart of an embodiment of a somatosensory networked gaming method of the present invention;

[0042] Figure 3 A schematic diagram of a flow chart of another embodiment of the somatosensory networked gaming method of the present invention;

[0043] Figure 4 The figure is a flow chart of another embodiment of the somatosensory networked gaming method of the present invention.

[0044] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0045] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0046] In order to better understand the above technical solution, exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0047] It should be noted that in the claims, any reference symbols placed between brackets shall not be constructed as limitations to the claims. The presence of "comprising" in the text does not exclude the presence of components or steps not listed in the claims. The quantifier "one" or "an" preceding a component does not exclude the presence of a plurality of such components. The present invention can be implemented by means of hardware comprising several different components and by means of appropriately programmed computers. In a unit claim that lists several means, several of these means may be embodied by the same hardware item. The use of "first", "second", and "third" etc. does not indicate any order, and these words may be interpreted as names.

[0048] like Figure 1 As shown, Figure 1 It is a structural diagram of a server 1 (also called a somatosensory networked gaming device) in a hardware operating environment involved in an embodiment of the present invention.

[0049] The server of the embodiment of the present invention is a device with display function, such as "Internet of Things devices", AR / VR devices with networking functions, smart speakers, self-driving cars, PCs, smart phones, tablet computers, e-book readers, portable computers, etc.

[0050] like Figure 1 As shown, the server 1 includes: a memory 11, a processor 12 and a network interface 13.

[0051] The memory 11 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory, etc.), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 11 may be an internal storage unit of the server 1, such as a hard disk of the server 1. In other embodiments, the memory 11 may also be an external storage device of the server 1, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card, etc., equipped on the server 1.

[0052] Furthermore, the memory 11 may include an internal storage unit of the server 1 as well as an external storage device. The memory 11 may be used not only to store application software and various data installed on the server 1, such as the code of the somatosensory networked game program 10, but also to temporarily store data that has been output or is to be output.

[0053] In some embodiments, the processor 12 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor or other data processing chip, used to run program codes or process data stored in the memory 11, such as executing the somatosensory networked game program 10.

[0054] The network interface 13 may optionally include a standard wired interface, a wireless interface (such as a WI-FI interface), and is generally used to establish a communication connection between the server 1 and other electronic devices.

[0055] The network may be the Internet, a cloud network, a wireless fidelity (Wi-Fi) network, a personal area network (PAN), a local area network (LAN), and / or a metropolitan area network (MAN). Various devices in the network environment may be configured to connect to the communication network according to various wired and wireless communication protocols. Examples of such wired and wireless communication protocols may include, but are not limited to, at least one of the following: Transmission Control Protocol and Internet Protocol (TCP / IP), User Datagram Protocol (UDP), Hypertext Transfer Protocol (HTTP), File Transfer Protocol (FTP), ZigBee, EDGE, IEEE 802.11, Light Fidelity (Li-Fi), 802.16, IEEE 802.11s, IEEE 802.11g, multi-hop communication, wireless access point (AP), device-to-device communication, cellular communication protocol, and / or Bluetooth (Blue Tooth) communication protocol or a combination thereof.

[0056] Optionally, the server may further include a user interface, which may include a display, an input unit such as a keyboard, and an optional user interface may further include a standard wired interface and a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode) touch device, etc. The display may also be referred to as a display screen or a display unit, which is used to display information processed in the server 1 and to display a visual user interface.

[0057] Figure 1 Only the server 1 having components 11-13 and the somatosensory networked game program 10 is shown. It can be understood by those skilled in the art that Figure 1 The structure shown does not constitute a limitation on the server 1, and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0058] In this embodiment, the processor 12 can be used to call the somatosensory networked game program stored in the memory 11 and perform the following operations:

[0059] After the client starts the preset somatosensory game, the player's original posture data is obtained from the client;

[0060] Generate a spectrum grayscale image according to the original posture data;

[0061] Importing the spectrum grayscale image into a preset convolutional neural network to recognize player actions;

[0062] Generate game operation instructions according to the recognition result of the preset convolutional neural network;

[0063] The game operation instruction is sent to the client.

[0064] In one embodiment, the processor 12 may be used to call the somatosensory networked game program stored in the memory 11 and perform the following operations:

[0065] Generate a spatial posture trajectory of the player's action according to the player's posture raw data;

[0066] generating a waveform graph according to the spatial posture trajectory;

[0067] Performing discrete Fourier transform on the original sampling data in the waveform diagram to obtain spectrum data;

[0068] Merging the spectrum data according to the time domain information to obtain a two-dimensional spectrum waterfall diagram;

[0069] The element values ​​in the two-dimensional spectrum waterfall diagram are mapped according to the image grayscale values ​​to obtain the spectrum grayscale diagram.

[0070] In one embodiment, the processor 12 may be used to call the somatosensory networked game program stored in the memory 11 and perform the following operations:

[0071] The sampled data of channel I and channel Q are combined into channel S according to the following formula:

[0072]

[0073] In the formula, S I is the sampling data of channel I, S Q is the sampling data of channel Q, and S is the combined power density.

[0074] In one embodiment, the processor 12 may be used to call the somatosensory networked game program stored in the memory 11 and perform the following operations:

[0075] The following formula is used to perform discrete Fourier transform on the original sampled data in the waveform:

[0076]

[0077] Wherein, N is the number of sampling points when performing discrete Fourier transform, and the value of N is limited to a positive integer power of 2.

[0078] In one embodiment, the processor 12 may be used to call the somatosensory networked game program stored in the memory 11 and perform the following operations:

[0079] The player's original posture data matching the preset target axis of the current somatosensory game is obtained from the client.

[0080] In one embodiment, the processor 12 may be used to call the somatosensory networked game program stored in the memory 11 and perform the following operations:

[0081] Collecting the posture data of the standard somatosensory movements of the players completing the preset somatosensory game to establish a preset data set;

[0082] Obtaining a training spectrum waterfall diagram of the preset data set;

[0083] Determining a preset target axis of the preset somatosensory game according to the waveform diagram of the preset data set; and

[0084] The convolutional neural network is trained using the training spectrum data graph of the preset target axis.

[0085] In one embodiment, the processor 12 may be used to call the somatosensory networked game program stored in the memory 11 and perform the following operations:

[0086] The original posture data obtained from the client is compressed by the Varints algorithm.

[0087] In one embodiment, the processor 12 may be used to call the somatosensory networked game program stored in the memory 11 and perform the following operations:

[0088] When the gateway node information bound to the client is inconsistent with the distribution gateway node of the current server, publishing the game operation instruction to the target gateway node bound to the client;

[0089] The game operation instruction is sent to the client through the target gateway node.

[0090] Based on the hardware architecture of the above-mentioned somatosensory networked game device, an embodiment of the somatosensory networked game method of the present invention is proposed. The somatosensory networked game method of the present invention is intended to realize somatosensory games in a wide area network and expand the user's game mode.

[0091] Reference Figure 2 , Figure 2 This is an embodiment of the somatosensory networked game method of the present invention, and the somatosensory networked game method comprises the following steps:

[0092] S10: After the client starts the preset somatosensory game, the original posture data of the player is obtained from the client.

[0093] The client refers to the terminal that runs the game, including but not limited to personal PC, game console, portable game console, mobile client. The mobile client includes but is not limited to smart phones, tablet computers, and portable computers.

[0094] Furthermore, the player's posture data can be detected by a six-axis IMU sensor, which includes a three-axis accelerometer and a three-axis gyroscope. The six-axis IMU sensor can detect the three-axis acceleration data and three-axis angular velocity data of the player during movement, and finally generate the player's posture data. Usually, the player's posture data is provided by a somatosensory device connected to the client, wherein the somatosensory device includes but is not limited to a bracelet, a glove watch, a headscarf, a hat, a vest, a fitness ring, and a game controller. If the client itself has a six-axis IMU sensor, the player's posture data can be provided directly by the client.

[0095] Specifically, after the game is started, the client establishes a communication connection with the server. During the game, the client can transmit the player's posture data to the server based on a specific communication protocol (usually TCP / IP protocol).

[0096] S20: Generate a frequency spectrum grayscale image according to the original posture data.

[0097] Specifically, the spectrum grayscale image is a grayscale representation of the spectrum of the time domain and frequency domain of the posture data, wherein the dual axes of the spectrum grayscale image are frequency and time. The spectrum grayscale image can intuitively and realistically display the player's action posture.

[0098] It is worth noting that when generating a spectral grayscale graph, the server will generate a corresponding number of spectral grayscale graphs according to the number of axes in the acquired data. That is, each time the data of an axis is acquired, the server generates a spectral grayscale graph of the axis data. For example, if the player posture data acquired by the server from the client includes the X-axis data and Y-axis data of the accelerometer, then the server will generate two spectral grayscale graphs, which are the spectral waterfall graphs of the X-axis and Y-axis of the accelerometer. Among them, the player posture data provided by the client to the server usually includes the data of the three axes of the accelerometer and the data of the three axes of the gyroscope, that is, a total of six-axis data. Therefore, when the server generates a spectral grayscale graph, it will generate six spectral grayscale graphs at the same time.

[0099] S30, importing the frequency spectrum grayscale image into a preset convolutional neural network to recognize player actions.

[0100] Specifically, convolutional neural networks have excellent image recognition capabilities, and spectral grayscale images can display the player's movements and postures in a graphical manner. Therefore, we conduct targeted training on the convolutional neural network so that the convolutional neural network can identify the player's somatosensory movements by identifying the spectral grayscale image. In this way, the characteristics of the convolutional neural network can be used to quickly identify the player's movements. In addition, training can improve the speed of the convolutional neural network in recognizing the player's movements, and can also improve the recognition accuracy of the player's movements.

[0101] S40, generating game operation instructions according to the recognition result of the preset convolutional neural network.

[0102] Specifically, the output result of the convolutional neural network is the recognized player action. If different player actions have corresponding game operation instructions in different somatosensory games, the game operation instructions are mainly used to control the player's game character to perform game actions that match the player's real somatosensory actions.

[0103] S50: Send the game operation instruction to the corresponding client.

[0104] Specifically, after obtaining the game operation instructions that match the player's real actions, the server sends the game operation instructions to the client through the communication protocol. After receiving the game operation instructions, the client will execute the game operation instructions so that the game character on the client can perform game actions that match the player's real actions.

[0105] Exemplarily, when the somatosensory game run by the client is a tennis somatosensory game, and the player performs a forehand swing, the client can send the posture data of the player during the game to the server, and the server can recognize that the player is currently performing a forehand swing through a preset convolutional neural network. After that, the server can obtain a game operation instruction for the forehand swing according to the forehand swing action, and send the game operation instruction to the client. After the client receives the game operation instruction, the client executes the game operation instruction so that the player character in the somatosensory game also performs a forehand swing. In this way, the mapping between the player's action and the game character's action can be achieved. In addition, since the client only performs data exchange and execution of operation instructions during the entire game process, there is no special limitation on the client model, so that clients of different models and types can all play somatosensory networked games. In addition, the player's action recognition is performed on the server, so the computing power requirements for the client are extremely low, eliminating the action recognition delay caused by the inconsistent computing power between different devices, and improving the fluency of somatosensory networked games.

[0106] It can be understood that in the somatosensory networked game method of the technical solution of the present application, the server can convert the player posture data provided by the client into a spectral grayscale image, and then classify the spectral grayscale image through a preset convolutional neural network to identify the player's action, and then generate game operation instructions according to the player's action, and send the game operation instructions to the client, and the client can execute the game operation instructions to match the game character action with the player's action. In this way, during the entire game process, the client only performs data exchange and operation instruction execution. Therefore, there is no special limitation on the client model, so that clients of different models and types can all play somatosensory networked games. In addition, the player's action recognition is performed on the server, so the computing power requirements for the client are extremely low, eliminating the action recognition delay caused by the inconsistent computing power between different devices, and improving the fluency of the somatosensory networked game. It can be seen that the somatosensory networked game method of the technical solution of the present application has the advantages of being able to realize somatosensory networked games, and having low requirements for client computing power, high accuracy in player action recognition, and good game fluency.

[0107] like Figure 3 As shown, in some embodiments, generating a spectrum grayscale image according to the original posture data includes:

[0108] S21, generating a spatial posture trajectory of the player's action according to the player's original posture data.

[0109] Specifically, after obtaining the original data of the player's posture, the original data of the player's posture can be solved by the Euler angle solution algorithm to obtain the player's spatial posture trajectory.

[0110] S22. Generate a waveform graph according to the spatial posture trajectory.

[0111] Specifically, the player's posture trajectory can be decomposed into waveforms of six axes, namely, three acceleration axes and three gyroscope axes, to obtain data of six dimensions. Through the above operation, the player's posture data can be reduced in dimension to facilitate subsequent processing of the player's posture data.

[0112] S23, performing discrete Fourier transform on the original sampling data in the waveform diagram to obtain spectrum data.

[0113] The original sampling data in the waveform diagram includes sampling data of channel I and sampling data of channel Q, that is, I / Q signals in digital communication.

[0114] Specifically, the discrete Fourier transform formula can be used to perform discrete Fourier transform on the original sampled data in the waveform. Specifically, the discrete Fourier transform formula is as follows:

[0115]

[0116] Where N is the number of sampling points when performing discrete Fourier transform.

[0117] After performing discrete Fourier transform on the original sampling data in the waveform, we can obtain the spectrum data of the original sampling data. The spectrum data represents the spectrum at a certain time point.

[0118] It is worth noting that through discrete Fourier transform, the data loss from original sampled data to spectrum data can be reduced.

[0119] S24, combining the spectrum data according to the time domain information to obtain a two-dimensional spectrum waterfall diagram.

[0120] Among them, the time domain information refers to the time information possessed by each spectrum data.

[0121] Specifically, according to the time information of each spectrum data, the order of each spectrum data can be determined, and then based on the order, multiple spectrum data belonging to the same axis are merged to obtain a two-dimensional spectrum waterfall diagram, in which the two coordinate axes are the time axis and the frequency axis.

[0122] S25, mapping the element values ​​in the two-dimensional spectrum waterfall diagram according to the image grayscale values ​​to obtain the spectrum grayscale diagram.

[0123] Specifically, the element value in the spectrum data diagram is the result value obtained by discrete Fourier transform.

[0124] Furthermore, after obtaining the above spectrum diagram, the element values ​​in the above two-dimensional spectrum diagram can be mapped according to the image grayscale value, so that we can obtain a grayscale image, which is the spectrum waterfall diagram we finally obtain.

[0125] It can be understood that since we use convolutional neural networks to perform feature recognition on the spectrum grayscale image, converting the two-dimensional spectrum waterfall image into a grayscale image can fit the image recognition characteristics of the convolutional neural network to accurately identify the player's actions.

[0126] In some embodiments, when performing Fourier transform on the original sampled data in the waveform, we limit the value of N in the discrete Fourier transform formula to an integer power of 2, that is, N=2 K , k=1, 2, 3….

[0127] Through the above settings, the calculation speed of discrete Fourier transform can be accelerated, thereby improving the recognition speed of user actions.

[0128] In some embodiments, before performing discrete Fourier transform on the original sampled data in the waveform, the method further includes:

[0129] The sampled data of channel I and channel Q are combined into channel S according to the following formula:

[0130]

[0131] In the formula, S I is the sampling data of channel I, S Q is the sampling data of channel Q, and S is the combined power density.

[0132] Specifically, the above formula can reduce the amount of original sampling data, so that the server can perform discrete Fourier transform and improve the recognition speed of player actions.

[0133] In some embodiments, obtaining the player's raw gesture data from the client includes:

[0134] The player's original posture data matching the preset target axis of the current somatosensory game is obtained from the client.

[0135] The preset target axis refers to the axis on which the signal changes most dramatically when the player performs a standard action that conforms to the somatosensory game. In other words, the signal of the preset target axis can best reflect the player's action. It is worth pointing out that the preset target axis can be one or more of the six axes of the three axes of the accelerometer and the three axes of the gyroscope.

[0136] Specifically, we can obtain the preset target axes corresponding to different somatosensory games by analyzing the waveform graph of a known data set.

[0137] Specifically, different somatosensory games have different types and numbers of axes that can best reflect player actions. Therefore, obtaining posture data of the preset target axis of the current somatosensory game from the client and using it for player action recognition can not only improve the recognition accuracy of player actions, but also reduce the terminal's computing power to increase the recognition speed of player actions.

[0138] like Figure 4 As shown, in some embodiments, before importing the spectrum waterfall diagram into a preset convolutional neural network to identify the player's action, the method further includes:

[0139] S110, collecting posture data of standard somatosensory movements of a player completing a preset somatosensory game to establish a preset data set.

[0140] Specifically, somatosensory devices can be set on a set number of players, and these players can be made to repeatedly complete the set somatosensory actions of the preset somatosensory game, and the posture data of these players when completing the set somatosensory actions can be collected to establish the required preset data set. It is worth noting that when collecting the preset data set for training, labels can be added to the data in the preset data set to facilitate the subsequent training of the convolutional neural network.

[0141] S120: Obtain a training spectrogram of the preset data set.

[0142] Specifically, based on the method of step S21 to step S25, a training spectrum grayscale image can be established for the preset data set.

[0143] S130: Determine a preset target axis of the preset somatosensory game according to the waveform diagram of the preset data set.

[0144] Specifically, the corresponding waveforms can be generated based on the preset data sets, and the preset target axes corresponding to different preset motion sensing games can be determined by observing the amplitude changes of these waveforms. Specifically, we can determine the required preset target axis based on the intensity of the waveform changes in the waveform corresponding to a complete action.

[0145] S140, training the convolutional neural network using the training spectrum grayscale image of the preset target axis.

[0146] Specifically, after determining the preset target axis for the preset somatosensory game, a spectral grayscale image can be generated based on the data of this or these preset axes. Thereafter, the spectral grayscale image can be imported into the convolutional neural network to train the neural network until the recognition accuracy of the convolutional neural network reaches the set accuracy.

[0147] It can be understood that the above method can be used to specifically train the convolutional neural network to improve the sensitivity of the convolutional neural network to the spectral grayscale image, thereby improving the recognition accuracy and recognition speed of the convolutional neural network to the player's actions.

[0148] In some embodiments, the raw gesture data obtained from the client is compressed using a Varints algorithm.

[0149] Specifically, the Varints algorithm is a typical data compression algorithm that can greatly reduce the amount of data while ensuring the amount of data information. It can be understood that by compressing the player's posture data through the Varints algorithm and then transmitting it to the server, it can reduce the packet loss from the client to the server and reduce the transmission delay to improve the player's gaming experience.

[0150] In some embodiments, sending the game operation instruction to the client includes:

[0151] S210. When the gateway node information bound to the client is inconsistent with the distribution gateway node of the current server, the game operation instruction is published to the target gateway node bound to the client.

[0152] It is worth noting that the server of the technical solution of the present application includes multiple distribution gateways. When the client accesses the server for the first time, the client's connection request will pass through a gateway node of the server. After the client successfully establishes a connection with the server, the server will bind the client to the gateway node.

[0153] Specifically, after the server generates the game operation instruction, it will send it to the client through any gateway node. If the distribution gateway node of the server currently distributing the game operation instruction is consistent with the gateway node information bound to the client, then the current gateway can directly send the game operation instruction to the corresponding client. If the server's current distribution gateway node is inconsistent with the gateway node information bound to the client, the server will publish the game operation instruction to the target gateway node bound to the client.

[0154] S230: Send the game operation instruction to the client through the target gateway node.

[0155] Specifically, after publishing the game operation to the target gateway node, the server directly sends the game operation instruction to the client through the target gateway node.

[0156] It can be understood that through the above method, the high concurrency performance and long connection performance of the service can be guaranteed, thereby improving the stability and low latency of the somatosensory networked games.

[0157] In addition, the embodiment of the present invention also proposes a computer-readable storage medium, which can be any one or any combination of a hard disk, a multimedia card, an SD card, a flash memory card, an SMC, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disk read-only memory (CD-ROM), a USB memory, etc. The computer-readable storage medium includes a somatosensory networked game program 10. The specific implementation of the computer-readable storage medium of the present invention is roughly the same as the specific implementation of the somatosensory networked game method and the server 1 described above, and will not be repeated here.

[0158] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0159] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0160] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1A function specified in one or more boxes.

[0161] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0162] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0163] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A somatosensory online gaming method, characterized in that: include: After the client starts the preset somatosensory game, the player's original posture data is obtained from the client; Generate a spectrum grayscale image according to the original posture data; Importing the spectrum grayscale image into a preset convolutional neural network to recognize player actions; Generate game operation instructions according to the recognition result of the preset convolutional neural network; Sending the game operation instruction to the client; Get the player's raw posture data from the client, including: Acquire the player's posture raw data matching the preset target axis of the current somatosensory game from the client; Before importing the frequency spectrum grayscale image into a preset convolutional neural network to identify the player's action, the method further includes: Collecting the posture data of the standard somatosensory movements of the players in the preset somatosensory game to establish a preset data set; Obtaining a training spectrum grayscale image of the preset data set; Determining a preset target axis of the preset somatosensory game according to the waveform diagram of the preset data set; and The convolutional neural network is trained using the training spectrum grayscale image of the preset target axis.

2. The somatosensory networked gaming method according to claim 1, wherein: Generating a spectrum grayscale image according to the original posture data includes: Generate a spatial posture trajectory of the player's action according to the player's posture raw data; generating a waveform graph according to the spatial posture trajectory; Performing discrete Fourier transform on the original sampling data in the waveform diagram to obtain spectrum data; Merging the spectrum data according to the time domain information to obtain a two-dimensional spectrum waterfall diagram; The element values ​​in the two-dimensional spectrum waterfall diagram are mapped according to the image grayscale values ​​to obtain the spectrum grayscale diagram.

3. The somatosensory networked gaming method according to claim 2, wherein: The original sampling data in the waveform diagram includes the sampling data of channel I and the sampling data of channel Q; Before performing discrete Fourier transform on the original sampled data in the waveform, the method further includes: The sampled data of channel I and channel Q are combined into channel S according to the following formula: ; In the formula, is the sampling data of channel I, is the sampling data of channel Q, and S is the combined power density.

4. The somatosensory networked gaming method according to claim 2, wherein: The following formula is used to perform discrete Fourier transform on the original sampled data in the waveform: ; Wherein, N is the number of sampling points when performing discrete Fourier transform, and the value of N is limited to a positive integer power of 2.

5. The somatosensory networked gaming method according to claim 1, wherein: The original posture data obtained from the client is compressed by the Varints algorithm.

6. The somatosensory networked gaming method according to claim 1, characterized in that: Sending the game operation instruction to the client includes: When the gateway node information bound to the client is inconsistent with the distribution gateway node of the current server, publishing the game operation instruction to the target gateway node bound to the client; The game operation instruction is sent to the client through the target gateway node.

7. A somatosensory networked gaming device, characterized in that: The invention comprises a memory, a processor and a somatosensory networked game program stored in the memory and executable on the processor. When the processor executes the somatosensory networked game program, the somatosensory networked game method as described in any one of claims 1 to 6 is implemented.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a somatosensory networked game program, and when the somatosensory networked game program is executed by the processor, the somatosensory networked game method according to any one of claims 1 to 6 is implemented.

Citation Information

Patent Citations

  • Three-dimensional micro-Doppler gesture identification method based on convolutional neural network

    CN108519812A

  • Radio signal identification method based on end-to-end convolutional neural network

    CN108764013A

  • Use state recognition method and device, storage medium and electronic equipment

    CN111176465A

  • Somatosensory health entertainment system based on wearable inertial perception and working method thereof

    CN111318009A

  • State identification method by characteristic analysis of histogram in time region and frequency region

    JP2018205292A