Somatic game method based on swing gesture

Abstract

The present invention discloses a swing-based somatosensory gaming method, device, and computer-readable storage medium. The method comprises: after a preset swing-based somatosensory game is launched, obtaining a raw player posture signal detected by a somatosensory device; filtering the player's swing action key signals from the raw player posture signal; importing the key signals into a preset swing posture recognition model to identify the player's swing posture; and controlling the player's game character to execute a matching swing action based on the recognition result of the preset swing posture recognition model. This method has the advantages of high swing posture recognition accuracy and a good player gaming experience.

Description

Technical Field

[0001] The present invention relates to the field of somatosensory game technology, and in particular to a somatosensory game method, device, and computer-readable storage medium based on a swinging posture. Background Art

[0002] Currently, motion-sensing gaming architectures on the market are typically based on IMUs (Inertial Measurement Units). During gaming, players wear a specific IMU device that detects their motion data. The system then converts this data into action commands that the game can recognize, allowing in-game objects to perform essentially the same actions as the user.

[0003] However, the problem with the IMU architecture is that during the game, problems such as gyroscope drift, angle random wandering, rate machine wandering, and rate slope are prone to occur, resulting in low action accuracy and easy misjudgment of player actions. Summary of the Invention

[0004] The embodiments of the present application provide a somatosensory gaming method based on a bat swinging posture, aiming to improve the recognition accuracy of a player's bat swinging posture in bat swinging-type somatosensory games.

[0005] To achieve the above objectives, the present invention provides a somatosensory gaming method based on a swinging posture, comprising:

[0006] After the preset swing-type motion sensing game is started, the original signal of the player's posture detected by the motion sensing device is obtained;

[0007] Filtering key signals of the swing action from the original player posture signal;

[0008] Importing the key signal into a preset swing posture recognition model to recognize the player's swing posture;

[0009] The player's game character is controlled to perform a matching swing action according to the recognition result of the preset swing posture recognition model.

[0010] In one embodiment, filtering out key signals of the swing action from the original player gesture signal includes:

[0011] determining a target signal interval for a complete swing action from the player's posture raw signal;

[0012] The key signal of the swing action is screened out from the target signal interval.

[0013] In one embodiment, filtering out a target signal interval of a complete swing action from the player's original posture signal includes:

[0014] generating a continuous waveform graph according to the player's posture original signal;

[0015] A first window with a control length of k1 and a step size of s1 slides on the continuous waveform graph;

[0016] Calculating a first amplitude average value of the waveform within the first window while sliding the first window;

[0017] If the first amplitude average value is less than a first amplitude threshold and greater than a second amplitude threshold, marking the current interval as a static interval;

[0018] If the first amplitude average value is greater than a first amplitude threshold, marking the current interval as a swing interval;

[0019] When the window passes through the static interval-swing interval-static interval in sequence, and the sliding time of the window in two adjacent static intervals is greater than the preset time threshold, the static interval, swing interval and static interval connected in sequence are marked as target signal intervals.

[0020] In one embodiment, filtering out key signals of a bat swing action from the target signal interval includes:

[0021] A second window with a control length of k2 and a step size of s2 slides within the target signal interval;

[0022] while sliding the second window, obtaining a maximum peak value within the second window;

[0023] Assume T is the maximum peak value in the second window, ΔT is the preset peak threshold, and Δt is the second preset duration threshold, then

[0024] If T n -T n-1 ≥ΔT, then mark T n The front end of the second window at is the starting moment of the swing interval;

[0025] If T n+1 -T n ≤-ΔT, then mark T n+1 The rear end of the second window is the end of the swing interval;

[0026] A signal within the swing interval is obtained as a key signal of the swing action.

[0027] In one embodiment, before filtering out the key signal of the swing action from the original player gesture signal, the method further includes:

[0028] Obtain a raw data set based on bat swing motion;

[0029] Performing noise reduction processing on the original data set to obtain a target data set;

[0030] The k1, s1, first amplitude threshold, preset duration threshold, k2, s2, second amplitude threshold and second preset duration threshold are determined according to the target data set.

[0031] In one embodiment, the player gesture signal includes a three-axis acceleration signal and a three-axis angular velocity signal;

[0032] Before filtering out the key signal of the swing action from the original player posture signal, the following steps are included:

[0033] The player's original posture signal of the axis with the most intense signal fluctuation among the three-axis acceleration signal and the three-axis angular velocity signal is selected to filter out the key signal of the swing action.

[0034] In one embodiment, before obtaining the original signal of the player's gesture detected by the somatosensory device, the method further includes:

[0035] Establish a swing posture recognition model based on neural network;

[0036] The swing gesture recognition model is trained using data associated with the swing action in the target dataset.

[0037] In one embodiment, the swing posture recognition model is based on a deep residual neural network, wherein the residual learning unit of the deep residual neural network has no less than 4 layers and no more than 20 layers.

[0038] To achieve the above-mentioned purpose, an embodiment of the present application also proposes a somatosensory gaming device based on a swinging posture, comprising a memory, a processor, and a somatosensory gaming program based on a swinging posture stored in the memory and runnable on the processor. When the processor executes the somatosensory gaming program based on a swinging posture, a somatosensory gaming method based on a swinging posture as described in any one of the above items is implemented.

[0039] To achieve the above-mentioned purpose, an embodiment of the present application also proposes a computer-readable storage medium, on which a somatosensory game program based on a swinging posture is stored. When the somatosensory game program based on a swinging posture is executed by a processor, a somatosensory game method based on a swinging posture as described in any one of the above items is implemented.

[0040] The present invention's swing-based somatosensory gaming method, which filters out key swing signals from the player's original posture signals and then imports these key signals into a preset swing-posture recognition model, can identify the player's swing posture. This improves the speed and accuracy of swing recognition, thereby enhancing the player's gaming experience. This demonstrates that, compared to traditional somatosensory gaming methods, the present invention offers the advantages of high swing-posture recognition accuracy and a better gaming experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying any creative work.

[0042] Figure 1 This is a module structure diagram of an embodiment of a somatosensory gaming device based on a bat swing posture according to the present invention;

[0043] Figure 2 1. It is a flow chart of an embodiment of a somatosensory gaming method based on a bat swing posture according to the present invention;

[0044] Figure 3 1. It is a flow chart of another embodiment of a somatosensory gaming method based on a bat swing posture according to the present invention;

[0045] Figure 4 This is a structural diagram of a first window sliding on a continuous waveform graph of a certain axis in the somatosensory game method based on a bat swing posture of the present invention;

[0046] Figure 5 2 is a flow chart of another embodiment of the somatosensory gaming method based on the swinging posture of the present invention.

[0047] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0048] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0049] To better understand the above technical solutions, exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments described herein. Instead, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0050] It should be noted that in the claims, any reference signs placed between brackets shall not be construed as limiting the claims. The presence of "comprising" in the text does not exclude the presence of components or steps not listed in the claims. The quantifier "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The present invention can be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In a unit claim that lists several means, several of these means may be embodied by the same hardware item. The use of "first", "second", and "third" etc. does not indicate any order and these words may be interpreted as names.

[0051] like Figure 1 As shown, Figure 1 It is a structural diagram of a server 1 (also called a swing-based somatosensory gaming device) in a hardware operating environment according to an embodiment of the present invention.

[0052] The server of the embodiment of the present invention is a device with display function, such as "Internet of Things devices", AR / VR devices with networking functions, smart speakers, self-driving cars, PCs, smart phones, tablets, e-book readers, portable computers, etc.

[0053] like Figure 1 As shown, the server 1 includes: a memory 11 , a processor 12 and a network interface 13 .

[0054] The memory 11 includes at least one type of readable storage medium, including a flash memory, a hard disk, a multimedia card, a card-type memory (e.g., an SD or DX memory), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the memory 11 may be an internal storage unit of the server 1, such as a hard disk of the server 1. In other embodiments, the memory 11 may also be an external storage device of the server 1, such as a plug-in hard disk equipped on the server 1, a smart media card (SMC), a secure digital (SD) card, a flash card, etc.

[0055] Furthermore, the memory 11 may include both an internal storage unit of the server 1 and an external storage device. The memory 11 may be used not only to store application software installed on the server 1 and various data, such as the code of the swing-based motion sensing game program 10, but also to temporarily store data that has been output or is about to be output.

[0056] In some embodiments, the processor 12 can be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip, used to run the program code or process data stored in the memory 11, such as executing the swing-based motion game program 10.

[0057] The network interface 13 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface), and is generally used to establish a communication connection between the server 1 and other electronic devices.

[0058] The network may be the Internet, a cloud network, a wireless fidelity (Wi-Fi) network, a personal area network (PAN), a local area network (LAN), and / or a metropolitan area network (MAN). Various devices in the network environment may be configured to connect to the communication network according to various wired and wireless communication protocols. Examples of such wired and wireless communication protocols may include, but are not limited to, at least one of the following: Transmission Control Protocol and Internet Protocol (TCP / IP), User Datagram Protocol (UDP), Hypertext Transfer Protocol (HTTP), File Transfer Protocol (FTP), ZigBee, EDGE, IEEE 802.11, Light Fidelity (Li-Fi), 802.16, IEEE 802.11s, IEEE 802.11g, multi-hop communication, wireless access point (AP), device-to-device communication, cellular communication protocol, and / or Bluetooth communication protocol, or a combination thereof.

[0059] Optionally, the server may further include a user interface, which may include a display and an input unit such as a keyboard. The optional user interface may also include a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. The display, which may also be referred to as a display screen or display unit, is used to display information processed in the server 1 and to display a visual user interface.

[0060] Figure 1 Only the server 1 having components 11-13 and the somatosensory game program 10 based on the swing posture is shown. It can be understood by those skilled in the art that Figure 1 The structure shown does not constitute a limitation on the server 1 , and the server 1 may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0061] In this embodiment, the processor 12 may be configured to call a swing-based motion sensing game program stored in the memory 11 and perform the following operations:

[0062] After the preset swing-type motion sensing game is started, the original signal of the player's posture detected by the motion sensing device is obtained;

[0063] Filtering key signals of the swing action from the original player posture signal;

[0064] Importing the key signal into a preset swing posture recognition model to recognize the player's swing posture;

[0065] The player's game character is controlled to perform a matching swing action according to the recognition result of the preset swing posture recognition model.

[0066] In one embodiment, the processor 12 may be configured to call a swing-based motion sensing game program stored in the memory 11 and perform the following operations:

[0067] determining a target signal interval for a complete swing action from the player's posture raw signal;

[0068] The key signal of the swing action is screened out from the target signal interval.

[0069] In one embodiment, the processor 12 may be configured to call a swing-based motion sensing game program stored in the memory 11 and perform the following operations:

[0070] generating a continuous waveform graph according to the player's posture original signal;

[0071] A first window with a control length of k1 and a step size of s1 slides on the continuous waveform graph;

[0072] Calculating a first amplitude average value of the waveform within the first window while sliding the first window;

[0073] If the first amplitude average value is less than a first amplitude threshold and greater than a second amplitude threshold, marking the current interval as a static interval;

[0074] If the first amplitude average value is greater than a first amplitude threshold, marking the current interval as a swing interval;

[0075] When the window passes through the static interval-swing interval-static interval in sequence, and the sliding time of the window in two adjacent static intervals is greater than the preset time threshold, the static interval, swing interval and static interval connected in sequence are marked as target signal intervals.

[0076] In one embodiment, the processor 12 may be configured to call a swing-based motion sensing game program stored in the memory 11 and perform the following operations:

[0077] A second window with a control length of k2 and a step size of s2 slides within the target signal interval;

[0078] while sliding the second window, obtaining a maximum peak value within the second window;

[0079] Assume T is the maximum peak value in the second window, ΔT is the preset peak threshold, and Δt is the second preset duration threshold, then

[0080] If T n -T n-1 ≥ΔT, then mark T n The front end of the second window at is the starting moment of the swing interval;

[0081] If T n+1 -T n ≤-ΔT, then mark T n+1 The rear end of the second window is the end of the swing interval;

[0082] A signal within the swing interval is obtained as a key signal of the swing action.

[0083] In one embodiment, the processor 12 may be configured to call a swing-based motion sensing game program stored in the memory 11 and perform the following operations:

[0084] Obtain a raw data set based on bat swing motion;

[0085] Performing noise reduction processing on the original data set to obtain a target data set;

[0086] The k1, s1, first amplitude threshold, preset duration threshold, k2, s2, second amplitude threshold and second preset duration threshold are determined according to the target data set.

[0087] In one embodiment, the processor 12 may be configured to call a swing-based motion sensing game program stored in the memory 11 and perform the following operations:

[0088] The player's original posture signal of the axis with the most intense signal fluctuation among the three-axis acceleration signal and the three-axis angular velocity signal is selected to filter out the key signal of the swing action.

[0089] In one embodiment, the processor 12 may be configured to call a swing-based motion sensing game program stored in the memory 11 and perform the following operations:

[0090] Establish a swing posture recognition model based on neural network;

[0091] The swing gesture recognition model is trained using data associated with the swing action in the target dataset.

[0092] Based on the hardware architecture of the aforementioned swing-based somatosensory gaming device, an embodiment of the present invention is provided for a somatosensory gaming method based on a swinging posture. The present invention aims to improve the accuracy of swing-based somatosensory gaming in recognizing a player's swinging posture.

[0093] Reference Figure 2 , Figure 2 This is an embodiment of a somatosensory gaming method based on a bat swinging posture according to the present invention, and the somatosensory gaming method based on a bat swinging posture comprises the following steps:

[0094] S10: After the preset swing-type motion sensing game is started, an original signal of the player's posture detected by the motion sensing device is obtained.

[0095] Among these, baton-swinging motion-sensing games refer to games that require players to make swing-like movements in real life to input game commands. For example, badminton, tennis, table tennis, baseball, rhythm and lightsaber games all fall under the category of baton-swinging motion-sensing games. Of course, baton-swinging motion-sensing games are not limited to the above-mentioned categories and can also include other games. Examples are not provided here one by one. Any game that requires players to make swing-like movements in real life to control a game character can be considered a baton-swinging motion-sensing game as referred to in the technical solution of this application.

[0096] Furthermore, the preset swing-type somatosensory game is run on a terminal, which may be a desktop computer, a laptop computer, a game console, a portable game console, a smart phone, a tablet computer, a smart watch, a smart TV, etc.

[0097] Among them, the somatosensory device refers to a device that can detect the player's posture data. Generally, the somatosensory device is configured to include a six-axis IMU sensor, which includes a three-axis accelerometer and a three-axis gyroscope. The six-axis IMU sensor detects changes in the player's three-axis acceleration and three-axis angular velocity to detect the player's posture data. Specifically, the somatosensory device is configured to be wearable, and its form factors include but are not limited to the following: wristbands, gloves, watches, headscarves, hats, vests, fitness rings, and game controllers.

[0098] Furthermore, before the game, the somatosensory device needs to establish a communication connection with the terminal, wherein a wired connection or a wireless connection can be established between the somatosensory device and the terminal. Exemplarily, when the somatosensory device establishes a wired connection with the terminal, it can be based on at least one of the USB2.0 protocol, USB3.0 protocol, Thunderbolt 3 protocol, and Thunderbolt 4 protocol; and when the somatosensory device establishes a wireless connection with the terminal, it can be based on at least one of the Bluetooth protocol, WiFi protocol, infrared protocol, 2.4G communication protocol, and NFC protocol. After the somatosensory device establishes a communication connection with the terminal and the preset swing-type somatosensory game is started, the somatosensory device will send the detected player's posture original signal to the terminal through at least one of the above-mentioned communication protocols. Among them, the original signal refers to the three-axis acceleration signal and the three-axis gyroscope signal of the somatosensory device.

[0099] S20: Filter out key signals of the swing action from the original player posture signal.

[0100] Among them, the key signal of the swing action refers to the signal that best reflects the player's swing posture. For the terminal, the player's swing posture can be identified based on only the key signal.

[0101] Specifically, the original signal of the player's posture detected by the somatosensory device includes all the signals generated by the player's movement, which causes a lot of noise to be mixed in the original signal of the player's posture. These noises will affect the terminal's recognition of the player's swinging action. For example, if the player is playing a somatosensory badminton game, during the game, the somatosensory device can not only generate a posture signal when the player swings the racket, but also generate a posture signal when the player shakes unconsciously or swings the racket by mistake, and send these posture signals to the terminal. Among the above-mentioned posture signals, the posture signal generated when the player actually swings the racket can be used by the terminal to determine what kind of swing action the player has performed (such as forehand swing, backhand swing, horizontal swing, etc.), while other signals will interfere with the terminal's recognition of the player's swing action.

[0102] Therefore, filtering out the key swing signals from the player's original posture signals can also be understood as removing posture signals that affect the terminal's ability to recognize the player's movements, thereby helping to improve the terminal's recognition accuracy of the player's swing. It is worth noting that filtering out the key signals from the player's original posture signals is performed by the terminal, reducing the computing power requirements of the motion sensing device and helping to reduce transmission latency from the motion sensing device to the terminal.

[0103] S30: Import the key signal into a preset swing posture recognition model to recognize the player's swing posture.

[0104] Among them, the preset swing posture recognition model is established and trained based on a machine learning algorithm, so as to be able to identify the player's swing posture according to key signals. Specifically, after screening out the key signals, the terminal imports the key signals into the preset swing posture recognition model to identify the player's swing posture. Exemplarily, taking the somatosensory tennis game as an example, the player's swing action includes forehand swing, backhand swing, slice and other swing actions. It can be understood that the swing posture recognition model completed through training can improve the recognition accuracy and recognition speed of the player's swing action. In addition, importing the key signals of the swing action into the model is also beneficial to improving the terminal's recognition accuracy and recognition speed of the player's swing action. Such a combination can maximize the recognition accuracy and recognition speed of the player's swing posture during the game, so that the game can more accurately map the player's swing action in a shorter time, thereby helping to improve the player's gaming experience.

[0105] It is worth noting that the preset jumping action recognition model can be deployed directly on the terminal, or on a local server or cloud server connected to the terminal.

[0106] S40: Control the player's game character to perform a matching swing action according to the recognition result of the preset swing posture recognition model.

[0107] Among them, the game character refers to the character image of the player in the game, which can adopt a first-person perspective or a third-person perspective. In addition, this application does not limit the game character, and it can be adaptively adjusted according to the game content and type.

[0108] Specifically, after recognizing the player's swing gesture based on the swing gesture, the terminal can control the player's game character to perform a game action that matches the recognized swing motion based on the swing gesture recognition model's recognition results. For example, if the player is playing a somatosensory badminton game and the swing gesture recognition model identifies the player's action as a forehand swing, the player character in the game will also perform the forehand swing accordingly.

[0109] It can be understood that the swing-based somatosensory gaming method of the present application, by filtering out the key signals of the swing action from the player's original posture signal, and then importing this key signal into a preset swing posture recognition model to identify the player's swing posture, can improve the recognition speed and accuracy of the player's swing action, thereby improving the player's operating experience. It can be seen that compared with traditional somatosensory gaming methods, the somatosensory gaming method of the present application has the advantages of high swing posture recognition accuracy and a good player gaming experience.

[0110] like Figure 3 As shown, in some embodiments, filtering out the key signal of the swing action from the original player gesture signal includes:

[0111] S21 : Determine a target signal interval for a complete swing action from the player's original posture signal.

[0112] Among them, a complete swing action includes the preparatory action before the swing, the swing action and the stopping action after the swing is completed. In general swing-type games, there is a certain time interval between two adjacent swing actions during the game. For example, in a somatosensory tennis game, the player will not swing the racket twice in a row. This is because after the tennis ball is hit by the character controlled by the player, another character (NPC character or another player character) is required to hit the tennis ball back before continuing to swing. In the time interval between two adjacent swing actions, the player will inevitably perform other actions, and the posture signals generated by these actions will interfere with the terminal's recognition of the player's swing posture.

[0113] Furthermore, the player's original gesture signal, in addition to the data from the three-axis accelerometer and the three-axis gyroscope, also includes the generation timestamp of each axis signal. The terminal can determine the target signal interval of a complete swing action based on the timestamp.

[0114] It can be understood that by first filtering out a target signal interval of a complete swing action from the player's posture original signal, the player's posture original signal can be pre-processed to reduce the amount of data required to obtain the key signal while improving the matching degree between the obtained key signal and the actual swing action.

[0115] S22: Filter out key signals of the swing action from the target signal interval.

[0116] Specifically, after determining the target signal interval of the swing action, the posture signal that can reflect the swing action can be screened out from the player posture signals defined by the target signal interval as the key signal of the swing action.

[0117] It can be understood that the above method can perform a preliminary screening of the player's posture signal, which can not only reduce the amount of data when the terminal recognizes the player's swing posture, but also help improve the terminal's recognition accuracy of the player's swing posture.

[0118] like Figure 4 and Figure 5 As shown, in some embodiments, determining a target signal interval of a complete swing action from the player's posture raw signal includes:

[0119] S110 , generating a continuous waveform according to the original player gesture signal.

[0120] Specifically, because the player's raw gesture signals include the changes and time duration of each axis signal in the three-axis accelerometer and three-axis gyroscope, the terminal can generate a continuous waveform graph with time as the horizontal axis and amplitude as the vertical axis after receiving these raw gesture signals. This continuous waveform graph, with time as the horizontal axis, shows the changes in the player's raw gesture signals received by the terminal.

[0121] It is worth noting that the terminal generates a continuous waveform for each axis of the three-axis accelerometer and the three-axis gyroscope.

[0122] S120 , controlling a first window with a length of k1 and a step size of s1 to slide on the continuous waveform graph.

[0123] Here, the length k1 refers to the range that the first window can cover. Since in this embodiment, the horizontal axis of the continuous waveform graph is time, the length of the first window is in time units, such as 40 milliseconds, 50 milliseconds, 60 milliseconds, 70 milliseconds, 80 milliseconds, 90 milliseconds, 100 milliseconds, etc.

[0124] Furthermore, the step length s1 refers to the sliding distance of the first window on the continuous waveform each time. In this embodiment, the unit of the step length s1 is also time, such as 10 milliseconds, 20 milliseconds, 30 milliseconds, etc.

[0125] It is worth noting that the length of the step s1 can be greater than the length k1 of the first window, or equal to or less than the length k1 of the first window. The length k1 of the first window and the step s1 can be adaptively designed according to actual conditions, and this application does not impose any specific restrictions on this.

[0126] Specifically, after the continuous waveform graph is generated, the first window can be controlled to slide along the continuous waveform graph in the positive direction of the horizontal axis (i.e., along the direction of increasing time) with a step size s1. Specifically, if the player's gesture original signal is received for the first time, the sliding starting point of the first window is the starting point of the continuous waveform graph; if it is not the first time the player's gesture original signal is received, the starting point of the first window is the end point of the target signal interval of the previous swing action.

[0127] S130. Calculate a first amplitude average value of the waveform within the first window while sliding the first window.

[0128] Specifically, while the first window is sliding, a first amplitude average value can be calculated based on the total value of all peaks and troughs in the area covered by the first window and the length of the first window. It is worth noting that when calculating the first full-width average value, the entire waveform is calculated as an absolute value to prevent waves on the upper side of the time axis from canceling out each other with waves on the lower side.

[0129] S140: If the first amplitude average value is less than a first amplitude threshold, mark the current interval as a static interval.

[0130] S150: If the first amplitude average value is greater than a second amplitude threshold, mark the current interval as a swing interval.

[0131] S160: When the window passes through the static interval, the swing interval, and the static interval in sequence, and the sliding time of the window in two adjacent static intervals is greater than a preset time threshold, the static interval, the swing interval, and the static interval connected in sequence are marked as target signal intervals.

[0132] Specifically, after calculating the first amplitude value within the first window, the first amplitude value can be compared with a first amplitude threshold and a second amplitude threshold to determine whether the current interval is a static interval or a swing interval, wherein the second amplitude threshold is greater than the first amplitude threshold.

[0133] Specifically, based on actual data and observations, we can determine that players remain relatively still for a certain period of time before and after a swing. Therefore, we can use a first amplitude threshold to distinguish signals generated during the static phase before a swing from signals generated by other movements between the two swings. Furthermore, because the signal intensity generated during a swing is much greater than other movement signals, a second amplitude threshold can be set to determine the interval within which the swing signal falls. The purpose of setting a preset duration threshold is to ensure accurate recognition of the target signal interval, preventing other player errors from interfering with the identification of the target signal interval.

[0134] It is worth noting that the first amplitude threshold, the second amplitude threshold and the preset duration threshold can be adaptively designed according to actual conditions, and this application does not impose any specific restrictions on this.

[0135] It can be understood that the target signal interval representing a complete swing action can be quickly and accurately identified from the player's original posture signal through the above method.

[0136] In some embodiments, filtering out key signals of a bat swing action from the target signal interval includes:

[0137] S210 , control a second window with a length of k2 and a step size of s2 to slide in the target signal interval.

[0138] Here, the length k2 refers to the range that the second window can cover. Since in this embodiment, the horizontal axis of the continuous waveform graph is time, the length of the second window is in time units, such as 10 milliseconds, 20 milliseconds, 30 milliseconds, etc.

[0139] Furthermore, the step length s2 refers to the sliding distance of the second window each time it slides on the continuous waveform graph. In this embodiment, the unit of the step length s2 is also time, such as 1 millisecond, 3 milliseconds, 5 milliseconds, etc.

[0140] It is worth noting that the length of step s2 can be greater than the length k2 of the second window, or equal to or less than the length k2 of the second window. The length k2 of the second window and the step s2 can be adaptively designed according to actual conditions, and this application does not impose any specific restrictions on this.

[0141] Specifically, after obtaining the target signal interval, the second window can be controlled to slide along the positive direction of the horizontal axis of the continuous waveform graph (i.e., along the direction of increasing time) on the continuous waveform graph of the target signal interval with a step size s2. The sliding starting point of the second window is the starting point of the target signal interval.

[0142] S220: Acquire a maximum peak value in the second window while sliding the second window.

[0143] Specifically, while the second window is sliding, the maximum peak value of the waveform in the second window may be determined according to the amplitude of the waveform in the area covered by the second window.

[0144] S230, set T as the maximum peak value in the second window, t as the time of the second window, ΔT as the preset peak threshold, then

[0145] If T n -T n-1 ≥ΔT, then mark T n The front end of the second window at is the starting moment of the swing interval;

[0146] If T n+1 -T n ≤-ΔT, then mark T n+1 The rear end of the second window is the end time of the swing interval.

[0147] S240: Acquire a signal within the swing interval as a key signal of the swing action.

[0148] Specifically, T n -T n-1 It represents the difference between the maximum peak values of two adjacent second windows. If the difference is greater than or equal to the preset peak threshold, it means that a signal that can only be generated when a swing occurs appears in the nth second window. In this case, the front end of the second window (that is, the end close to the starting point of the target signal interval) can be marked as the starting point of the swing interval. Similarly, if T n+1 -T nIf the difference between the two values is less than or equal to the negative preset peak threshold, it indicates that the signal generated at the end of the swing action appears in the second window. In this case, the end of the second window (i.e., the end closest to the end of the target signal interval) can be marked as the end of the swing interval. This way, the desired start and end times of the swing interval can be obtained. Furthermore, the signal within the swing interval can be used as the key signal.

[0149] It can be understood that the swing interval determined in the above manner can, on the basis of ensuring the matching degree between the swing interval and the actual swing action, provide the swing interval with a certain front and back tolerance, thereby ensuring that the key signal obtained has sufficient signal volume, thereby allowing a certain degree of error on the basis of ensuring the accuracy of the player's swing posture recognition.

[0150] like Figure 5 As shown, in some embodiments, before filtering out the key signal of the swing action from the original player gesture signal, the method further includes:

[0151] 310. Obtain an original data set based on a bat swing action.

[0152] Specifically, data from a number of players actually playing a swing-type game may be collected to obtain an original data set.

[0153] 320. Perform noise reduction processing on the original data set to obtain a target data set.

[0154] Specifically, the original data set can be screened through digital signal processing techniques such as filtering and normalization to perform noise reduction on the original data set.

[0155] 330. Determine k1, s1, a first amplitude threshold, a second amplitude threshold, a preset duration threshold, k2, s2, and a peak threshold according to the target data set.

[0156] Specifically, a continuous waveform graph can be generated based on the data in the target dataset, and then the required k1, s1, first amplitude threshold, second amplitude threshold, preset duration threshold, k2, s2, and peak threshold can be statistically obtained. This can make the obtained values more closely match the actual player data, thereby helping to improve the accuracy of obtaining key signals of the player's swing action and further helping to improve the accuracy of identifying the player's swing action.

[0157] In some embodiments, before filtering out the key signal of the swing action from the raw player gesture signal, swing gesture recognition includes:

[0158] The player's original posture signal of one axis with the most intense signal fluctuation among the three-axis acceleration signal and the three-axis angular velocity signal is selected to filter out the key signal of the swing action for swing posture recognition.

[0159] It is worth noting that although the three-axis accelerometer and the three-axis gyroscope can generate signals on a total of six axes when the player swings the bat, there will inevitably be one axis where the signal intensity fluctuation is relatively drastic. The signal associated with the axis with the largest signal intensity fluctuation can best reflect the player's swing posture among the signals of the six axes, and the player's swing posture can then be identified based on the signal of this axis.

[0160] Specifically, methods such as the variance method can be used to identify the axis with the most dramatic signal fluctuations in the three-axis acceleration and three-axis gyroscope. Once this axis has been identified, the key signals for the desired swing motion can be filtered from this axis. This significantly reduces the computational effort required by the terminal to identify the player's swing gesture, thereby increasing the speed of swing gesture recognition and reducing the computing power required by the terminal.

[0161] In some embodiments, before obtaining the original signal of the player's gesture detected by the somatosensory device, the method further includes:

[0162] S410: Establish a swing posture recognition model based on a neural network.

[0163] Specifically, a swing posture recognition model can be established based on a neural network such as a convolutional neural network or a deep residual neural network. Specifically, using a neural network to establish a swing posture recognition model can improve the accuracy and speed of recognizing a player's swing posture.

[0164] S420: Use the data associated with the swing action in the target data set to train the swing posture recognition model.

[0165] Specifically, since the target dataset is a dataset generated by collecting player gesture signals while playing a bat-swinging game, a corresponding continuous waveform graph can also be generated. Since the target dataset is a known, labeled dataset, key data associated with the bat-swinging motion can be directly selected from the target dataset to train the model, thereby improving the model's training speed and accuracy. Of course, the design of this application is not limited to this. In other embodiments, our method for identifying key signals from the player's original gesture signals can be combined to separate the target dataset into multiple complete bat-swinging motions. Key data associated with the bat-swinging motion can then be filtered from these complete bat-swinging motions to train the established model.

[0166] In some embodiments, the swing posture recognition model is based on a deep residual neural network, wherein the residual learning unit of the deep residual neural network has no less than 4 layers and no more than 20 layers.

[0167] Specifically, building a swing posture recognition model based on a deep residual network reduces the difficulty of model convergence and adapts to the high similarity between different swings, thereby enhancing the trained model's swing posture recognition accuracy. The deep residual network's residual units are set to no fewer than 4 layers and no more than 20 layers to reduce model training time and development costs while ensuring model recognition accuracy.

[0168] In addition, an embodiment of the present invention further provides a computer-readable storage medium. The computer-readable storage medium can be any one of a hard disk, a multimedia card, an SD card, a flash memory card, an SMC, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disc read-only memory (CD-ROM), a USB memory, or any combination thereof. The computer-readable storage medium includes a swing-based motion sensing game program 10. The specific implementation of the computer-readable storage medium of the present invention is substantially the same as the specific implementation of the swing-based motion sensing game method and server 1 described above, and will not be further described here.

[0169] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0170] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0171] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0172] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0173] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0174] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A somatosensory game method based on a swinging posture, characterized in that: include: After the preset swing-type motion sensing game is started, the original signal of the player's posture detected by the motion sensing device is obtained; Filtering key signals of the swing action from the original player posture signal; Importing the key signal into a preset swing posture recognition model to recognize the player's swing posture; According to the recognition result of the preset swing posture recognition model, the game character of the player is controlled to perform a matching swing action; wherein, The key signals of the swing action are filtered out from the original player posture signal, including: determining a target signal interval for a complete swing action from the player's posture raw signal; Filtering key signals of the swing action from the target signal interval; A target signal interval of a complete swing action is filtered out from the player's original posture signal, including: generating a continuous waveform graph according to the player's posture original signal; A first window with a control length of k1 and a step size of s1 slides on the continuous waveform graph; Calculating a first amplitude average value of the waveform within the first window while sliding the first window; If the first amplitude average value is less than a first amplitude threshold and greater than a second amplitude threshold, marking the current interval as a static interval; If the first amplitude average value is greater than a first amplitude threshold, marking the current interval as a swing interval; When the window passes through the static interval, the swing interval, and the static interval in sequence, and the sliding time of the window in two adjacent static intervals is greater than a preset time threshold, the static interval, the swing interval, and the static interval connected in sequence are marked as target signal intervals; The key signals of the swing action are screened out from the target signal interval, including: A second window with a control length of k2 and a step size of s2 slides within the target signal interval; while sliding the second window, obtaining a maximum peak value within the second window; Assume T is the maximum peak value in the second window, ΔT is the preset peak threshold, and Δt is the second preset duration threshold, then If T n -T n-1 ≥ΔT, then mark T n The front end of the second window at is the starting moment of the swing interval; If T n+1 -T n ≤-ΔT, then mark T n+1 The rear end of the second window is the end of the swing interval; A signal within the swing interval is obtained as a key signal of the swing action.

2. The somatosensory gaming method based on the swinging posture according to claim 1, wherein: Before filtering out the key signal of the swing action from the original player gesture signal, the method further includes: Obtain a raw data set based on bat swing motion; Performing noise reduction processing on the original data set to obtain a target data set; The k1, s1, first amplitude threshold, preset duration threshold, k2, s2, second amplitude threshold and second preset duration threshold are determined according to the target data set.

3. The somatosensory gaming method based on the swinging posture according to claim 1, wherein: The player's posture original signal includes a three-axis acceleration signal and a three-axis angular velocity signal; Before filtering out the key signal of the swing action from the original player posture signal, the following steps are included: The player's posture original signal of one axis with the most intense signal fluctuation among the three-axis acceleration signal and the three-axis angular velocity signal is selected to filter out the key signal of the swing action.

4. The somatosensory gaming method based on the swinging posture according to claim 2, wherein: Before obtaining the original signal of the player's posture detected by the somatosensory device, the method further includes: Establish a swing posture recognition model based on neural network; The swing gesture recognition model is trained using data associated with the swing action in the target dataset.

5. The somatosensory game method based on the swinging posture according to claim 4, characterized in that: The swing posture recognition model is based on a deep residual neural network, wherein the residual learning unit of the deep residual neural network is not less than 4 layers and not more than 20 layers.

6. A somatosensory gaming device based on a bat swinging posture, characterized in that: The invention comprises a memory, a processor and a somatosensory game program based on a swinging posture stored in the memory and executable on the processor. When the processor executes the somatosensory game program based on a swinging posture, the somatosensory game method based on a swinging posture as described in any one of claims 1 to 5 is implemented.

7. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a somatosensory game program based on a swinging posture. When the somatosensory game program based on a swinging posture is executed by a processor, the somatosensory game method based on a swinging posture as claimed in any one of claims 1 to 5 is implemented.

Images