External hanging type motion sensing control system and motion sensing control method for shooting games and training

Abstract

The invention discloses an external hanging type motion sensing control system for shooting games and training. The external hanging type motion sensing control system comprises a data acquisition module, a data processing module, a power management module, an output display module, a personal area network interaction module and a cloud service module. The data acquisition module is used for acquiring data required for resolving the spatial attitude of a user through a sensor; the data processing module processes the collected sensor data, maps the sensor data into the actual state and operation of a user, and stores and manages user-defined control parameters; the power supply management module is used for supplying power to the somatosensory control equipment and managing the power supply of the somatosensory control equipment; the output display module is used for displaying the state information of the somatosensory control equipment; the personal area network interaction module is used for communication and information interaction; and the cloud service module is used for a user to customize sensor parameter mapping and equipment information management through a mobile phone. The invention further provides a somatosensory control method which has a wide application scene.

Description

Technical Field

[0001] The present invention relates to the field of somatosensory control technology, and in particular to a somatosensory control system and a somatosensory control method for plug-in shooting games and training, which realizes somatosensory game operations on multiple platforms based on low-power Bluetooth communication and somatosensory control technology. Background Art

[0002] Somatosensory games are interactive games that are operated and felt through human body movements. Studies have shown that based on the dimensions of spatial and kinesthetic participation, somatosensory gaming devices can enhance players' participation and immersion. Compared with traditional interaction methods such as mouse and keyboard, natural human-computer interaction is based on sensor technology, which converts body motion sensing signals into game control signals.

[0003] As the gaming market continues to expand, many gaming companies are developing motion-sensitive games, such as sports games, shooting games, and fitness games, which can encourage families to play and exercise together. Playing motion-sensitive games requires specific equipment, such as Wii Remote, Kinect, PlayStation Eyetoy and other game controllers.

[0004] Among them, Wii uses the error between inertial technology and human perception to achieve the purpose of body sensing; Kinect is based on optics and realizes body sensing operation by installing identification points on human joints. These devices are not only expensive and require specific game platforms, but are also easily affected by ambient light and camera vision.

[0005] On the PC platform, some manufacturers have also launched somatosensory peripherals that can map body postures to mouse operations. However, the current prices of commercial somatosensory peripherals for PC platforms are relatively high (mostly more than 1,000 yuan), and most of them have fixed appearances and functions, and the algorithms used are relatively complex and have low customization. Currently, existing commercial products lack somatosensory mapping for keyboards, that is, operations of visual orientation movement, or use joysticks for operation, which limits the types of games that can be applied to peripherals, and the immersion is not significantly improved compared to keyboard and mouse operations or handle operations. Most existing devices on the market use USB wireless receivers to connect to devices, which is quite inconvenient when connecting to mobile devices; at the same time, they only map mouse actions based on acceleration sensors, and only map traditional mouse operations. With the trend of popularization of virtual reality, they cannot provide posture mapping for corresponding objects in virtual space. Summary of the invention

[0006] In order to solve the following technical problems existing in the somatosensory game control devices on the market:

[0007] 1. There are device platform barriers and it cannot be used on multiple platforms;

[0008] 2. Unable to customize the appearance, weak immersion;

[0009] 3. Unable to customize input mapping parameters, low flexibility;

[0010] 4. Vision-based somatosensory devices involve cameras, screens, high-computing-power CPUs and other devices, and are relatively expensive.

[0011] The present invention proposes an external customizable somatosensory control system and a somatosensory control method based on low-power Bluetooth communication and somatosensory control technology to solve the above-mentioned technical problems.

[0012] The external shooting game and training somatosensory control system provided by the present invention comprises: a data acquisition module, a data processing module, a power management module, an output display module, a personal area network interaction module, and a cloud service module;

[0013] Wherein, the data acquisition module acquires the data required for solving the user's spatial posture through sensors;

[0014] The data processing module processes the collected sensor data, maps the sensor data to the actual status and operation of the user, and saves and manages the user-defined control parameters;

[0015] The power management module supplies power to and manages the somatosensory control device;

[0016] The output display module displays the status information of the somatosensory control device;

[0017] The personal area network interaction module communicates with the data processing module, the external game running terminal and the cloud service module to realize information interaction between the somatosensory control device and the external game running terminal and the cloud service module;

[0018] The cloud service module is used for users to customize sensor parameter mapping and device information management through mobile phones.

[0019] The data processing module runs on the single-chip microcomputer, and the data processing module is respectively connected to the data acquisition module, the power management module, the output display module, and the personal area network interaction module, and the personal area network interaction module is connected to the game running terminal and the cloud service module.

[0020] The single-chip microcomputer hardware device includes a data processing unit, a storage unit, a communication peripheral interface, a wireless radio frequency antenna, and an embedded real-time operating system. The data processing unit is responsible for performing various calculations and processing tasks, solving posture data in real time and converting it into user operation instructions in the game. The storage unit is used to store the user's custom control parameters, sensor calibration data, and temporary data during the game. The communication peripheral interface includes I2C, SPI, UART, GPIO and other interfaces, which are used to connect peripherals including various sensors, display devices, etc. to ensure high-speed and stable data transmission. The wireless radio frequency antenna is used to send data packets to the outside world via low-power Bluetooth or wifi.

[0021] The sensors in the data acquisition module include a nine-axis inertial sensor, a touch capacitor, a Hall sensor, a button, a three-axis joystick, etc.;

[0022] The nine-axis inertial sensor, i.e., a three-axis accelerometer-angular velocity meter-magnetometer measurement unit, is used to collect the three-axis acceleration, three-axis angular velocity and three-axis geomagnetic readings of the somatosensory control device in space;

[0023] The touch capacitor is used to detect the discharge status of the current sensor contact object;

[0024] The Hall sensor can detect the change and direction of the magnetic field, that is, it can obtain the motion state of a magnetic component without contact;

[0025] The button is used to detect the on / off status of the circuit and determine the user's operation on the button;

[0026] The three-axis joystick is used to detect the user's shaking operation in the X and Y directions, and the pressing operation in the Z direction.

[0027] The above data is transmitted to the MCU through the external interface, and is first pre-processed in the data processing module, and then further processed by parsing to parse the user's spatial posture and operation behavior and map the game operation. The nine-axis inertial sensor is connected to the MCU through the IIC bus, and other sensors are connected to the MCU through the GPIO pins.

[0028] The data processing module includes three functions: first, preprocessing the raw sensor readings obtained in the data acquisition module, then converting the preprocessed data to obtain the user's spatial posture and operation behavior, and finally mapping the solved spatial posture and operation behavior into specific game operations;

[0029] The data preprocessing module relies on the storage unit and data processing unit of the single-chip microcomputer hardware; the data processing unit and the storage unit are the basic hardware components of the single-chip microcomputer. The former is also called the data calculation unit, which is used to complete all numerical calculations on the single-chip microcomputer. The latter is used to store program instructions and data. All methods of the data preprocessing module need to be implemented based on the data processing unit and the storage unit.

[0030] Taking into account the real-time response of the system, the preprocessing includes first-order low-pass filtering and threshold filtering;

[0031] The first-order low-pass filter is used to reduce the interference of high-frequency noise and obtain more stable spatial posture and motion information by smoothing the sensor data. In the present invention, the implementation of the first-order low-pass filter includes setting the cutoff frequency of the filter, attenuating the signal above the frequency, so as to retain the low-frequency effective signal required by the device and avoid the user being affected by too much noise during operation.

[0032] The threshold filter sets a threshold to filter out small invalid data fluctuations and retain only valid motion signals that exceed the threshold. This method can effectively reduce the impact of small errors or jitters on device operation and ensure that the system can respond correctly when receiving large motion or operation inputs.

[0033] The processing and conversion of the user posture parameter data includes:

[0034] The nine-axis inertial sensor is responsible for collecting the three-axis acceleration, three-axis angular velocity and geomagnetic data of the device in space. Through the Kalman filter algorithm, the acceleration and angular velocity data are dynamically fused and processed to eliminate noise and transient errors, and the pitch angle, roll angle and yaw angle of the device in three-dimensional space are obtained. These attitude data determine the tilt and rotation angle of the user's device. The magnetometer and accelerometer are combined to correct the azimuth angle of the device to ensure that the attitude information of the device is stable and accurate when rotating or moving rapidly. In addition, after multiple samplings, the acceleration information is processed by time integration to obtain the user's motion trajectory information, such as movement direction and speed. Through this processing flow, the user's position and attitude in space can be accurately calculated.

[0035] The Hall sensor can obtain the relative motion state of the magnetic component by detecting the changes in the external magnetic field and the direction of the magnetic field. After data processing, it is converted into magnetic flux information, and then the user's operation of the component is determined.

[0036] The button sensor detects the on / off state of the circuit to determine whether the user presses or releases the button and transmits the change of the switch state.

[0037] The touch capacitive sensor determines the user's contact with the device surface by monitoring the discharge behavior of the capacitor, including whether the touch is made and the duration of the touch, and can further analyze the user's interactive actions.

[0038] The three-axis joystick obtains the user's joystick operation information by detecting the user's displacement in the X-axis and Y-axis directions, as well as the pressing operation in the Z-axis direction. The displacement changes of the X-axis and Y-axis represent the user's in-plane shaking operation, while the pressing action of the Z-axis detects the user's vertical pressure on the joystick. By collecting and processing these data, the user's control input to the device can be accurately determined, including the tilt direction, tilt angle and pressing intensity of the joystick.

[0039] The spatial posture and operation behavior mapping is a specific game operation function. The data processed and transformed by the data processing module is subjected to posture-operation mapping, and the control operation of the game object corresponding to the posture is obtained through a multi-layer decision tree algorithm, including but not limited to the following:

[0040] Movement control: Based on the spatial posture data obtained by the nine-axis inertial sensor, the system can map the user's body postures such as leaning forward, leaning back, and tilting left and right, that is, the calculated pitch angle and yaw angle, into the character movement instructions in the game. The forward leaning operation can be mapped to moving forward, the backward leaning operation can be mapped to moving backward, and the left and right tilts correspond to the character's movement direction to the left or right respectively. Users can accurately control the movement of the game character through changes in body posture.

[0041] Viewing angle adjustment: The system can use the angular velocity data of the nine-axis inertial sensor to adjust the viewing angle in the game in real time. The change in yaw angle is mapped to the left and right rotation of the camera in the game. When the user turns his head, the horizontal rotation of the viewing angle is controlled. The change in pitch angle controls the up and down viewing angle adjustment of the camera in the game. The operation of looking up or down is mapped to the action of the game character observing up and down.

[0042] Aiming and shooting: Through the posture information obtained by the nine-axis sensor and the movement of the components detected by the Hall sensor, the system can accurately determine the user's aiming behavior. Combined with the detection function of the Hall sensor, the system can determine whether the trigger of the simulated gun held by the user is pulled, thereby determining whether a shooting operation has occurred.

[0043] Switching weapons or functions: The operation information of the three-axis joystick can be used to switch functions in the game. Users can switch between different weapons or functions by shaking the joystick. The displacement of the X-axis and Y-axis of the joystick determines the specific function or weapon selected, and the pressing action of the Z-axis can trigger the confirmation of the selection.

[0044] Interaction: Capacitive touch sensors can be used to determine the user's interaction behavior in the game. By touching the surface of the device, the user can interact with objects in the virtual world, such as picking up items, opening doors, or activating devices. The duration of the touch can also be mapped to different interaction effects, such as a short press for a light touch and a long press for a continuous operation.

[0045] Jumping and dodging: The user's up and down movements and acceleration changes can be mapped to actions such as jumping, dodging, or sliding. For example, when the accelerometer detects that the user moves upward quickly, the system recognizes it as a jumping action and triggers the character's jumping behavior in the game; if the user is detected to lean to one side quickly, it will be mapped as a dodging action.

[0046] The output display module includes a display device to display usage instructions and current game and / or operation information to the user;

[0047] The power management module includes a battery power supply unit and a power management unit, which converts the source voltage and current into the voltage and current available to the load in the somatosensory control system, manages the power supply of the load, and detects and displays the current power;

[0048] The personal area network interaction module is connected to the data processing module, the game running terminal and the cloud service module to perform information interaction between the somatosensory control system and the outside; the personal area network interaction module relies on the data processing unit in the single-chip microcomputer, and together with the wireless radio frequency antenna, realizes the wireless communication interaction function between the somatosensory control system and the external device. The data processing unit is responsible for receiving the game operation data corresponding to the user posture output by the data processing module, and converting it into corresponding control instructions. For example, at this time, the game operation corresponding to the user posture output by the data processing module is shooting, and the data processing unit generates an instruction to inform the game running terminal (such as a computer) connected at this time to trigger a mouse click operation. In addition, the data processing unit also undertakes the custom instruction mapping function, and the user can set or adjust the control operation of a specific posture through the cloud service module, and the system will correspond the posture to the game operation according to the user's needs. The radio frequency antenna is responsible for sending and receiving wireless signals, including pairing connection with the game running terminal, transmitting control instructions, and interacting with the cloud service module. Through the radio frequency antenna, the somatosensory control device can realize real-time data synchronization and communication with the game terminal and the cloud, ensuring that each posture operation of the user can be quickly and accurately mapped to the corresponding action in the game.

[0049] The cloud service module includes mobile terminal software and cloud service software; users interact with the personal area network interaction module through the cloud service mobile terminal, and can customize and modify the correspondence between sensor data and control instructions in the mapping and solution module and their triggering methods. Between the cloud service mobile terminal and the personal area network interaction module, data is transmitted in a standardized data packet encapsulation format in the form of a string. The format of the string generally includes the following fields: sensor identification, sensor value, operation mapping type, control instruction type, parameter setting and checksum. For example, the string format in a specific scenario can be<SENSOR_ID:0001,VALUE:0.85,MAP_TYPE:ATTACK,CMD_TYPE:CLICK,PARAM:DELAY_100MS,CRC:AB12> , where SENSOR_ID represents the unique identification code of the sensor, VALUE represents the current data value of the sensor, MAP_TYPE specifies the user-defined operation mapping type (such as the "attack" action), CMD_TYPE defines the corresponding control instruction type (such as simulated mouse click operation), PARAM field is used to pass related parameter settings (such as trigger delay), CRC is the checksum of the data packet, which is used to verify the integrity and accuracy of the transmission. Through this standardized data packet format, the cloud service mobile terminal can flexibly configure and adjust the operation response of the somatosensory control system, making the user experience more personalized and efficient.

[0050] The interaction and parameter modification between the cloud service mobile terminal and the personal area network interaction module are transmitted via wireless communication, and the user's operations, settings and custom parameter modifications on the mobile terminal will be synchronized to the cloud server in real time. The cloud software is responsible for storing and managing these modifications to ensure that the user's custom mapping rules, sensor parameters and control command settings are consistent across different devices. Through the cloud synchronization mechanism, users can access and adjust the configuration of their somatosensory control system on different terminal devices at any time, and ensure that parameter updates are immediately effective between the game terminal and the control device, achieving a seamless operation experience across devices.

[0051] The present invention also provides a somatosensory control method for shooting games and training implemented by using the somatosensory control system, the method comprising the following steps:

[0052] Step 1: Use sensors to collect one or more user posture parameters in three-dimensional physical space, including human body movement direction, trajectory, device curvature, touch position, holding posture, and device acceleration, angular velocity, and geomagnetic data;

[0053] Step 2: Filter and preprocess the collected posture parameters;

[0054] The filtering preprocessing includes first-order low-pass filtering and / or threshold filtering;

[0055] The signal with a higher than preset cut-off frequency is filtered out by using a preset filter cut-off frequency, and a valid signal is retained;

[0056] By using a preset motion amplitude threshold, motion amplitude fluctuation signals that do not exceed the threshold are filtered out, and valid motion signals that exceed the threshold are retained; and / or,

[0057] Multiple sensor data are fused through Kalman filtering.

[0058] Step 3: Use the multi-layer decision tree algorithm to map the pre-processed attitude parameters to the pitch angle, roll angle, and yaw angle into the forward and backward movement, left and right rotation, and perspective adjustment operations in the game;

[0059] Step 4: Transmit the mapped game control instructions to the game terminal via Bluetooth or WiFi to control the characters or objects in the game to achieve cross-platform compatibility.

[0060] In a specific embodiment, the motion control method further includes: the user customizes the mapping relationship between sensor parameters and game operations through the cloud service module, encapsulates the customized data in a string format, and sends it to the motion control device to update the mapping table.

[0061] In step three, the multi-layer decision tree analyzes the data collected by the sensor layer by layer, determines the user's operation from different dimensions and maps it into specific instructions; and / or,

[0062] Each layer of the decision tree filters and refines the data according to the sensor input conditions, and finally outputs the corresponding action.

[0063] The beneficial effects of the present invention include: the present invention is based on multi-sensors, realizes highly customizable functions through wireless personal area networks, and realizes miniaturization of the device through self-designed PCB boards and shells, while having a high degree of freedom of models based on plug-in interfaces, thereby providing a stronger sense of immersion. Through the scheduling mechanism of the embedded real-time operating system FreeRTOS, the present invention adopts a priority-based multi-task scheduling method to optimize processing efficiency and resource utilization. FreeRTOS decomposes the application into multiple tasks with different priorities, and adopts time slicing and preemptive scheduling strategies to ensure that high-priority tasks can be processed in a timely manner, thereby ensuring real-time performance and system response speed.

[0064] In FreeRTOS, each task is managed by a task control block (TCB), and synchronization mechanisms such as queues, semaphores, and mutexes are used to achieve communication and coordination between tasks. For example, when the sensor data acquisition task (low priority) completes data acquisition, it passes the data to the data processing task (high priority) through a message queue. The data processing task calculates based on the sensor data and notifies the control instruction task (higher priority) through a semaphore to trigger the corresponding control instruction. In this way, high-priority tasks can respond in a shorter time, thereby ensuring the real-time nature of critical tasks and effectively reducing the requirements for hardware processing capabilities. This scheduling method not only improves the operating efficiency of the system, but also reduces the overall system cost.

[0065] Compared with the existing somatosensory game control devices on the market, the present invention has the following advantages:

[0066] 1. The invention is low-priced, compact, easy to carry, simple to use, does not require additional auxiliary equipment, and has no site restrictions.

[0067] 2. Compared with the fixed-shaped somatosensory game control devices on the market (such as fixed gun-shaped or block-shaped devices), the present invention can be quickly mounted on the Picatinny rail commonly used for firearms through the quick-release fixing slot on the shell, thereby achieving high customizability of the appearance. Users can use control devices that are closer to real objects in the game, greatly enhancing the sense of immersion and experience. This design is also applicable to other types of games. For example, by replacing the shell interface, the present invention can also be mounted on sports tools such as table tennis rackets, and then used for somatosensory control of ball sports, thereby further enhancing the versatility of the device.

[0068] 3. There are usually terminal barriers for somatosensory gaming devices on the market, and a certain device is usually only applicable to a specific gaming terminal. However, the present invention achieves cross-platform versatility through wireless personal area network and low-power Bluetooth analog input device. Based on BLE connection and the adaptation mechanism of custom controller terminals (such as keyboards, mice, handles, etc.), the present invention breaks the hardware barriers between peripherals on different platforms and achieves compatibility of devices on different terminals such as computers, game consoles, mobile phones, and tablets. Users do not need to frequently switch communication protocols or devices between different platforms, which greatly simplifies the operation process and improves the user experience.

[0069] 4. The present invention supports users to customize button-level mapping operations. Users can easily set and adjust various parameters of the device through cloud services to obtain a more personalized somatosensory interaction experience, which not only improves the intelligence level of the device, but also further improves its versatility.

[0070] 5. In the experiment, the present invention performed excellently in the multi-platform compatibility test, the device response delay was controlled within 30ms, and the setting time of user-defined mapping was shortened by 50% on average, which significantly improved the user's interactive experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative work.

[0072] Figure 1 System architecture diagram of the present invention.

[0073] Figure 2 It is a flow chart of the initialization phase of the present invention.

[0074] Figure 3 It is a flow chart of the normal use phase of the present invention.

[0075] Figure 4 It is a flow chart of the parameter customization stage of the present invention.

[0076] Figure 5 This is a left, middle and right view of a somatosensory control system according to an embodiment of the present invention installed on a somatosensory support hardware model.

[0077] Figure 6 It is a hardware device interaction framework diagram of the present invention.

[0078] Figure 7 This is a schematic diagram of the PCB board.

[0079] Figure 8-1 This is a schematic diagram of the 3D printed shell.

[0080] Figure 8-2 This is a schematic diagram showing the installation of the display expansion component.

[0081] Fig. 9 This is the cloud service module architecture diagram. DETAILED DESCRIPTION

[0082] The present invention is further described in detail with reference to the following specific examples and drawings. The process, conditions, experimental methods, etc. for implementing the present invention, except for the contents specifically mentioned below, are all common knowledge and common common sense in the art and are not particularly limited by the present invention.

[0083] The present invention provides a somatosensory control system for external shooting games and training, including: a data acquisition module, a data processing module, a power management module, an output display module, a personal area network interaction module, and a cloud service module; the data acquisition module obtains data required for solving the user's spatial posture through sensors; the data processing module processes the collected sensor data, maps the sensor data to the user's actual state and operation, and saves and manages the user-defined control parameters; the power management module supplies power and manages the somatosensory control device; the output display module displays the state information of the somatosensory control device; the personal area network interaction module communicates and exchanges information with the data processing module, the external game terminal and the cloud service module; the cloud service module is used for users to customize sensor parameter mapping and device information management through mobile phones. The present invention also provides a somatosensory control method, which has a wide range of application scenarios.

[0084] The present invention provides a somatosensory control system for external shooting games and training. The somatosensory control system of the present invention mainly includes the following modules: Figure 1 Shown: data acquisition module, data processing module, personal area network interaction module, output display module, power management module, cloud service module.

[0085] in,

[0086] The data acquisition module is used to obtain the user's posture data including human body movement direction, trajectory, touch device position, holding posture, device bending angle, etc. The data acquisition module contains sensors such as nine-axis inertial sensors, touch capacitors, Hall sensors, buttons, and three-axis joysticks. It can detect the parameters of multiple intelligent sensors for processing and fusion to obtain the user's posture parameters in three-dimensional physical space. It has high human-computer interactivity. The data acquisition module is connected to the communication peripheral interface. The electrical property data of the voltage generated by the sensor in the module is converted into analog data or digital data through the communication peripheral interface for the next step of processing.

[0087] The data processing module further includes a data preprocessing module, a posture data conversion processing module, and a posture-operation mapping module, which are used to preprocess the data obtained by the data acquisition module, and further process and convert the preprocessed data, converting the original physical data of the sensor (such as acceleration, angular velocity, geomagnetic field strength, etc.) into the user's spatial posture information (such as pitch angle, roll angle, yaw angle) and then mapping it to specific game operation behavior data (such as direction movement, attack, etc.). The data processing after preprocessing specifically includes operations such as data fusion and posture solution. The role of preprocessing is to suppress sensor noise, correct errors, and improve data stability; the role of conversion is to convert physical data into intuitive posture and action parameters, in preparation for further mapping to game operations.

[0088] The nine-axis inertial sensor in the data acquisition module includes a three-axis accelerometer, a three-axis gyroscope, and a three-axis magnetometer. When the device is started, in order to reduce the error accumulation of the inertial sensor, it is necessary to correct the reading of the three-axis magnetometer, read the initial readings of the three-axis accelerometer and gyroscope multiple times, and calculate the average as the initial bias value. At the same time, factors such as electromagnetic environment interference, sensor accuracy limitations, and serial port level jumps may cause noise and affect the stability of sensor readings. Specific impacts include excessive data fluctuations and reduced reading accuracy. To solve this problem, the present invention uses first-order low-pass filtering and threshold filtering. The first-order low-pass filter is used to remove high-frequency noise and maintain low-frequency valid signals, thereby smoothing the sensor data; the threshold filter filters out small invalid fluctuations to ensure the accuracy and stability of the system response. In addition, in addition to the above-mentioned filtering method, the preprocessing also includes Kalman filtering for multi-sensor data fusion to improve the accuracy and dynamic performance of attitude solution.

[0089] After obtaining the processed posture data, the posture-operation mapping module passes it as an input parameter into the posture-operation mapping algorithm. This algorithm is the core innovation of the present invention, and a multi-layer decision tree structure is used to realize the mapping of posture data and game operations. The posture-operation mapping is mainly divided into two parts: one is the correspondence between the perspective conversion in the physical space and the perspective conversion in the cyber game space, and the other is the correspondence between the user's other physical input (such as gestures, actions) and the character operation in the game. The correspondence between these two parts is stored and implemented through a mapping table, which records the control content (such as perspective rotation, shooting, etc.) and trigger mode (such as long press or short press operation trigger) corresponding to different sensor data.

[0090] For example, in a multi-layer decision tree, a certain posture may correspond to multiple operations: if the user's pitch angle is less than a preset value and accompanied by forward movement, it is matched to the "forward" operation; and when the hand swings forward quickly, it is mapped to the "attack" operation. The implementation process of the decision tree first filters invalid data through branch conditions according to the posture information provided by the sensor, and then accurately matches and maps within the specific range of the data corresponding to the preset different operations, and finally outputs the game operation. The mapping table can be adjusted dynamically so that the system supports user-defined operation modes to adapt to different game requirements and personal operation habits. In addition, the posture-operation mapping module can not only process the input of the current posture data, but also adjust the mapping relationship between posture and operation in real time according to the user's customized settings. For example, the user can modify the mapping relationship between sensor data and control content and the triggering method through the cloud service mobile terminal, and the modification in the cloud will be synchronized to the mapping table in the storage unit. The mapping table saves the specific mapping rules for each posture and operation, and supports dynamic updates according to different application scenarios.

[0091] In the multi-layer decision tree structure, gesture data is parsed layer by layer. For example, the first layer of decision is to determine the direction of the user's gesture, whether it is "horizontal movement" or "vertical movement"; the second layer determines the specific operation strength based on the gesture amplitude, such as "light push" or "heavy hit"; finally, based on these decision results, it is mapped to specific operation instructions in the game, such as moving the character, attacking, etc.

[0092] Through this multi-layer decision tree structure, the system can cope with complex posture changes and ensure the diversity and flexibility of input. In addition, the posture-mapping algorithm also combines the relationship between different sensor data to make the operation more precise. For example, the system combines the data of the accelerometer and gyroscope to determine the user's rotation and tilt movements, which are then mapped to the perspective rotation or the character's movement direction in the game.

[0093] The personal area network interaction module relies on the data processing unit in the single-chip microcomputer and cooperates with the wireless radio frequency antenna to interact with external terminals. It specifically includes two parts. The first part is to package the corresponding operations of the game characters calculated by the data processing module and send them to the corresponding game terminals through BLE (low-power Bluetooth). The game terminals include computers, game consoles, TVs, mobile phones or tablets. Since all devices are based on BLE for data transmission, BLE connection is required in advance when connecting different game terminals, and corresponding control instructions are generated according to the type of connected devices and the user-defined controller type (such as keyboard, mouse, handle, etc.), so as to ensure cross-platform compatibility and operation consistency. This design solves the compatibility problem of traditional somatosensory game control devices when used across terminals. Traditional devices usually face technical difficulties such as inconsistent protocols and differences in control logic when switching between different terminal platforms. The present invention realizes multi-platform seamless control through standardized BLE communication protocols and flexible instruction adaptation mechanisms. The second part is that users can interact with the device through the cloud service software on their mobile phones. Users can modify the mapping relationship and triggering method between the sensor data and the control instructions in the storage unit of the single-chip microcomputer, and then customize the operation mapping content of each data, which solves the problem that traditional somatosensory game control devices cannot customize mapping parameters and lack flexibility. In order to facilitate the communication between the cloud service software and the device and the encoding and decoding of data transmission, the present invention designs a data packet encapsulation format in the form of a string, which generally includes the sensor identifier, value, mapping type, control instruction type, parameter setting and checksum. For example, the data packet format can be<SENSOR_ID:0001,VALUE:0.85,MAP_TYPE:MOVE,CMD_TYPE:JOYSTICK,PARAM:SPEED_50,CRC:AB12> , where SENSOR_ID is the unique identifier of the sensor, VALUE is the current value of the sensor, MAP_TYPE specifies the action mapping type (such as "move"), CMD_TYPE defines the corresponding control instruction type (such as "analog joystick" operation), the PARAM field passes the relevant operation parameters (such as "speed 50%"), and CRC is a data check code used to ensure the integrity and accuracy of data transmission.

[0094] The display output module includes different display devices, including an OLED display screen, which is used to display usage instructions and current information to the user, including: remaining bullets, power level, current operation logo, etc.

[0095] The power module includes a battery power supply unit and a power management unit. It is mainly responsible for converting the source voltage and current into power that can be used by loads such as microprocessors and sensors, managing the load power supply, and being able to detect and display the current power.

[0096] The data processing unit, storage unit, and communication peripheral interface used in the above modules are all implemented using a single-chip microcomputer, and the real-time operating system installed in the single-chip microcomputer can perform multi-threaded data processing, improve throughput and response speed, reduce the quality requirements for hardware such as single-chip microcomputers, and reduce equipment costs to solve the problem of high prices of products on the market.

[0097] All of the above modules are integrated on a stacked PCB and stored in a casing to reduce its size. Thanks to the external interface designed in the casing, the device can be flexibly mounted on various firearm models, solving the problem that current products on the market cannot customize their appearance and have a weak sense of immersion.

[0098] like Figure 8-1 The figure shows a schematic diagram of the 3D printed shell. The middle part of the working plane is the shell body, the top of which is the Picatinny rail fixing slot, the bottom is the storage space for the PCB board integrating the single-chip microcomputer and electronic components, the rear has a window for wiring, and the left side of the shell has a fixing slot for fixing the extended display screen. On the left side of the shell body is a quick disassembly and assembly component for external connection, from front to back are: nut slot, latch, wrench and outer edge of the fixing slot. When in use, insert the latch into the hole of the wrench, install the nut in the hole on the right side of the fixing slot on the upper part of the shell body, fix the outer edge of the fixing slot to the corresponding position on the top of the shell, align the hole of the latch in the wrench with the hole of the fixing slot and the hole of the nut, and connect it with screws.

[0099] The right side of the housing is the display expansion assembly. Install it by inserting the two-page card slot into the three-page card slot and fixing it with screws. Figure 8-2 shown.

[0100] During external installation, the required electronic components and the microcontroller are soldered to the PCB board, inserted into the storage area of ​​the 3D printed shell together with the lithium battery, and the electronic components are connected by flying wires from the wiring window.

[0101] The cloud service module is used to allow users to customize the content of sensor parameter mapping and display the device status to users. The cloud service module consists of two parts. One is the front-end program applied to the mobile terminal, which is responsible for displaying device information to users and accepting user instructions to interact with the device. The other part is the background program running in the cloud, including database, service layer, authentication layer and interface layer, which is used to store device information such as model, preset parameters, mapping parameters, etc., and process data requests from the mobile terminal front-end software and interact with the front-end program.

[0102] The method of using the somatosensory control system in the present invention includes three stages: initialization, normal use, and parameter customization.

[0103] like Figure 2As shown, the user enters the initialization phase when turning on the power. At this time, the output display module displays the initialization guidance information, and the data acquisition module begins to collect the user's body movement direction, trajectory, touch position, device curvature, holding posture and other information. The information collected by the data acquisition module is filtered by the data preprocessing module, and the initial data is calculated by taking the average value, and stored as a parameter in the storage unit. After that, the personal area network interaction module starts broadcasting signals until a stable interactive network is established with the game running terminal.

[0104] After initialization, it enters the normal use phase, such as Figure 3 The data acquisition module continuously collects information such as the user's body movement direction, trajectory, touch position, device curvature, etc. The data preprocessing module filters the information collected by the data acquisition module, and the mapping solution module passes the processed body posture parameters as parameters into the posture-operation mapping algorithm and finally obtains the intelligent control operation of the corresponding game object under the current posture. The control operation obtained by the settlement is packaged into a control input that can be recognized by the game terminal through the personal area network interaction module to control the behavior of the game object.

[0105] A special input posture is preset in the storage unit. When the posture parameters obtained by the data processing module match the preset special posture, normal use is interrupted and the parameter customization stage is entered, such as Figure 4 As shown. At this time, the personal area network interaction module disconnects the link with the game running terminal and rebroadcasts the connection information until a stable link is established with the mobile software. After that, the mobile front-end software receives the user's input and packages the user input into a data packet in a specific format and sends it to the device. After receiving the data packet, the personal area network interaction module decodes it and stores the decoding result in the storage unit as a new preset parameter. When using the front-end software, the front-end software needs to upload the updated operation to the cloud software and store the data.

[0106] A preferred embodiment of the present invention is a firearm model external somatosensory control device based on the method of the present invention. Its appearance is as follows: Figure 5 As shown, the present invention does not include the yellow pistol model and the top sight of the model in the figure, but only includes the lower hanging part at the front of the model and the sensors attached to the model body, that is, the data acquisition module, including: the touch capacitance sensor on the grip side, the touch capacitance sensor at the bottom of the trigger guard, the touch capacitance sensor on the side of the gun body, the Hall sensor in the trigger guard, the three-axis joystick on the side of the gun body, the nine-axis inertial sensor integrated in the 3D printed shell, and the micro switch on the reload button. In addition, it also includes an output display module, that is, the OLED display on the model sight and the RGB indicator light integrated in the printed shell.

[0107] In a specific embodiment, the sensor provided on the model also includes a magnet provided at the trigger position. Since the Hall sensor cannot detect non-magnetic materials, a magnet structure is added to the trigger so that the movement of the trigger can be detected by the Hall sensor as a change in magnetic flux.

[0108] In this embodiment, the device collects the somatosensory data of the user in all directions and other posture data, such as the holding position, through the data acquisition module, and performs filtering processing through the data preprocessing module, and performs posture solution based on the preset parameters, and then converts the posture data into control instructions that the game terminal can control and recognize through the mapping solution module, and sends the instructions to the terminal through the personal area network interaction module to realize the visual aiming, shooting, walking and other instructions of the up and down left and right rotation direction and amplitude of the somatosensory device. Compared with the traditional game controller, the present invention uses somatosensory methods to aim and control the game perspective, which is easy to operate and has high authenticity. Users can aim at the target and operate the perspective more accurately and quickly. At the same time, the personal area network interaction module can also receive and decode the instructions sent by the user through the front-end software of the mobile phone to realize the customization of the mapping content at the button level.

[0109] like Figure 6 As shown, the data acquisition module in the present invention includes 7 sensors, namely: 1 three-axis joystick, 1 accelerometer-gyroscope-magnetometer integrated nine-axis inertial sensor, 3 capacitive touch sensors, and 2 switch buttons. The main control board microcontroller includes a data processing unit, a storage unit, a communication peripheral interface, a wireless radio frequency antenna, etc.;

[0110] In the specific implementation process, the output display module used is an OLED display screen with a screen size of 0.96 inches, a pixel size of 128*64, and is driven by the SSD1306 library.

[0111] Sensors other than the nine-axis inertial sensor are connected to the input GPIO pins of the main control board. These sensors input information to the control software in the main control unit by changing the high and low levels at the GPIO pins. The nine-axis inertial sensor and OLED display are connected to the SDA and SCL pins of the main control unit, and input information to the control software in the main control unit using the I2C communication protocol through the two bidirectional IO buses in the main control unit.

[0112] In this implementation, low-power Bluetooth is used to build a wireless personal area network. The protocol uses 802.11b / g / n and the frequency is 2.4GHz. When the device is powered on, the Bluetooth radio module continuously sends link broadcast signals on the three radio frequency channels 37 / 38 / 39 at intervals of 0.5s. The interactive data packet used by this solution for the device and the mobile front-end software includes sensor identification, value, mapping type, control instruction type, parameter settings and checksum. For example, the data packet format can be<SENSOR_ID:0001,VALUE:0.85,MAP_TYPE:MOVE,CMD_TYPE:JOYSTICK,PARAM:SPEED_50,CRC:AB12> , where SENSOR_ID is the unique identifier of the sensor, VALUE is the current value of the sensor, MAP_TYPE specifies the action mapping type (such as "move"), CMD_TYPE defines the corresponding control instruction type (such as "analog joystick" operation), the PARAM field passes the relevant operation parameters (such as "speed 50%"), and CRC is a data check code used to ensure the integrity and accuracy of data transmission.

[0113] The 3D printed housing and stacked PCB board used in this embodiment are implemented as follows Figure 7 、8.

[0114] like Figure 7 The hardware structure design of the PCB board of the external shooting game and the somatosensory control system for training is shown, which mainly includes the following components:

[0115] Attitude sensor (IIC): Located in the motion sensing area in the figure. The attitude sensor (such as the nine-axis inertial sensor) is responsible for collecting the user's attitude data, including three-axis acceleration, three-axis angular velocity and three-axis geomagnetic data, which is used to determine the motion state of the device in three-dimensional space.

[0116] RGB indicator light: There are multiple "RGB" positions marked in the figure, indicating the status of the device. When the device is turned on or operated, different colors of lights can be used to prompt the user of the current status or function of the device.

[0117] Hall trigger: used to detect changes in magnetic field, and cooperate with Hall sensor to determine the user's operation of trigger or joystick, etc., for shooting or other physical operations in the game.

[0118] Micro-reload button: This button is designed to capture the user's physical input, such as pressing operations. It can provide precise trigger operation feedback and is suitable for fast response operations required in games, such as firing, switching weapons, etc.

[0119] Grip sensor bar: There are two grip sensor bars, namely "Grip sensor bar #1" and "Grip sensor bar #2", which are used to detect whether the user is holding the device and perceive the user's holding status through tactile feedback, thereby improving the accuracy and immersion of the interaction.

[0120] OLED screen: Located in the external device area, it is used to display the device's operating status, in-game information, etc. It can display real-time feedback, such as the remaining bullets, battery power, and other information, allowing users to more intuitively obtain the device status in the game.

[0121] The entire device design realizes module expansion through the board expansion area, which not only enables the hardware device to be lightweight in design, but also enables the expansion of different functional modules through different external interfaces to adapt to more game scenarios and needs.

[0122] The cloud service module in this implementation adopts a C / S-like architecture and develops a front-end and back-end separated cloud service system based on WeChat applet and SpringBoot. Fig. 9 .

[0123] Fig. 9 A multi-layer system design is presented, which is mainly used for the cloud service module of the somatosensory control device. The overall architecture is divided into multiple layers from top to bottom, including the interaction layer, interface layer, authentication layer, service layer and data layer.

[0124] Interaction layer: includes WebView and JsCore, which are responsible for rendering the front-end interface of mini-programs or mobile applications. Users interact with devices through the WeChat client. This layer provides users with an intuitive operation interface and feedback on device status.

[0125] Interface layer: WebSocket is used for two-way communication between the device and the client, ensuring real-time interaction between the motion control device and the mobile terminal or game terminal. This layer supports users to adjust device configuration and sensor mapping in real time through cloud services.

[0126] Authentication layer: Sa-Token is a framework for user identity authentication and permission management to ensure that user operations are legal and protect data security. In the configuration of somatosensory control devices, users need to pass this layer for identity authentication to ensure that only authorized users can modify device parameters.

[0127] Service layer: It is composed of the Spring framework and its extensions (such as IOC / AOP), responsible for the implementation of business logic and dependency injection. This layer handles requests from clients, such as user-defined sensor data mapping or device parameter adjustment. At the same time, Spring Boot simplifies the construction and deployment of applications, and improves the development efficiency and maintainability of the service layer.

[0128] Data layer: This layer includes MySQL and Redis, which are used for persistent storage and caching of data. MySQL stores long-term data, such as user-defined device parameters and mapping rules, while Redis is used to store temporary or frequently accessed data, ensuring that the system can quickly respond to user requests and improve overall performance.

[0129] The cloud service system achieves efficient communication and operation between devices, clients and the cloud through a layered architecture. Each layer collaborates with each other through standardized interfaces, ensuring the security, real-time and high scalability of the system.

[0130] The protection content of the present invention is not limited to the above embodiments. Without departing from the spirit and scope of the present invention, changes and advantages that can be thought of by those skilled in the art are included in the present invention and are protected by the attached claims.

Claims

1. A somatosensory control system for external shooting games and training, characterized in that: The somatosensory control system includes: a data acquisition module, a data processing module, a power management module, an output display module, a personal area network interaction module, and a cloud service module; The data acquisition module acquires data required for calculating the user's spatial posture through sensors; The data processing module processes the collected sensor data, maps the sensor data to the actual status and operation of the user, and saves and manages the user-defined control parameters; The power management module supplies power to and manages the somatosensory control device; The output display module displays the status information of the somatosensory control device; The personal area network interaction module communicates with the data processing module, the external game running terminal and the cloud service module to achieve information interaction; The cloud service module is used for users to customize sensor parameter mapping and device information management through mobile phones; The data processing module is connected to the data acquisition module, the power management module, the output display module, and the personal area network interaction module respectively; the personal area network interaction module is connected to the game running terminal and the cloud service module.

2. The somatosensory control system according to claim 1, characterized in that: The sensor data collected by the data acquisition module includes data of a nine-axis inertial sensor, a touch capacitor, a Hall sensor, a button, and a three-axis remote sensing; The nine-axis inertial sensor collects three-axis acceleration, three-axis angular velocity and three-axis geomagnetic readings of the somatosensory control device in space; The touch capacitor detects the discharge status of the current sensor contact object; The Hall sensor detects the change and direction of the magnetic field to obtain the motion state of the component; The on-off condition of the button detection circuit is used to determine the user's operation on the button; The three-axis joystick is used to detect the user's shaking operation in the X and Y directions and the pressing operation in the Z direction.

3. The somatosensory control system according to claim 1, characterized in that: The data processing module runs on a single-chip microcomputer, and further includes a data preprocessing module, a posture data conversion processing module, and a posture-operation mapping module; the hardware of the single-chip microcomputer includes a data processing unit, a storage unit, a communication peripheral interface, a wireless radio frequency antenna, and an embedded real-time operating system is deployed; The data processing unit is responsible for performing various calculation and processing tasks, solving posture data in real time and converting it into operation instructions for the user in the game; The storage unit is used to store user-defined control parameters, sensor calibration data, and temporary data during the game; The communication peripheral interfaces include I2C, SPI, UART, and GPIO interfaces, which are used to connect sensors, display peripherals, and transmit communication data; The wireless radio frequency antenna is used to send data packets to the outside world via Bluetooth or WiFi.

4. The somatosensory control system according to claim 1, characterized in that: The power management module includes a battery power supply unit and a power management unit, which converts the source voltage and current into a voltage and current available to the load in the somatosensory control system; and / or, The output display module includes a display device to display usage instructions and current game and / or operation information to the user.

5. The somatosensory control system according to claim 1, characterized in that: The personal area network interaction module relies on the data processing unit in the single-chip microcomputer and works together with the wireless radio frequency antenna to realize the wireless communication interaction function between the body sensing control system and the external device; The data processing unit receives game operation data corresponding to the user's posture mapped and output by the data processing module, and converts the game operation data into corresponding control instructions; and / or, the data processing unit implements custom instruction mapping, sets or adjusts the control operation of a specific posture through the cloud service module, and the somatosensory control system matches the posture with the game operation according to the user's needs; The wireless radio frequency antenna is responsible for sending and receiving wireless signals, achieving real-time data synchronization and communication with the game terminal and the cloud, achieving pairing connection with the game running terminal, transmitting control instructions, and interacting with the cloud service module.

6. The somatosensory control system according to claim 1, characterized in that: The cloud service module includes mobile terminal software and cloud service software, interacts with the personal area network interaction module, customizes and modifies the correspondence between sensor data and control instructions in the data processing module and their triggering methods; The cloud service module and the personal area network interaction module transmit data via a standardized data packet encapsulation format in the form of a string; through the cloud storage and management mechanism, the user's customized settings remain consistent between different devices, supporting real-time synchronization across devices, ensuring that users can seamlessly switch between different terminal devices and maintain the personalized operation configuration of the somatosensory control system.

7. A somatosensory control method for shooting games and training, characterized in that: The method is implemented by a somatosensory control system according to any one of claims 1 to 6, comprising the following steps: Step 1: Use sensors to collect one or more user posture parameters in three-dimensional physical space, including human body movement direction, trajectory, device curvature, touch position, holding posture, and device acceleration, angular velocity, and geomagnetic data; Step 2: Filter and preprocess the collected posture parameters; Step 3: Use the multi-layer decision tree algorithm to map the pre-processed attitude parameters to the pitch angle, roll angle, and yaw angle into the forward and backward movement, left and right rotation, and perspective adjustment operations in the game; Step 4: Transmit the mapped game control instructions to the game terminal via Bluetooth or WiFi to control the characters or objects in the game to achieve cross-platform compatibility.

8. The somatosensory control method according to claim 7, characterized in that: The somatosensory control method further includes: the user customizes the mapping relationship between sensor parameters and game operations through the cloud service module, encapsulates the customized data in a string format, and sends it to the somatosensory control device to update the mapping table.

9. The somatosensory control method according to claim 7, characterized in that: In step 2, the collected posture parameters are subjected to first-order low-pass filtering and / or threshold filtering; The signal with a higher than preset cut-off frequency is filtered out by using a preset filter cut-off frequency, and a valid signal is retained; By using a preset motion amplitude threshold, motion amplitude fluctuation signals that do not exceed the threshold are filtered out, and valid motion signals that exceed the threshold are retained; and / or, Multiple sensor data are fused through Kalman filtering.

10. The somatosensory control method according to claim 7, characterized in that: In step 3, the multi-layer decision tree analyzes the data collected by the sensor layer by layer, determines the user's operation from different dimensions and maps it into specific instructions; and / or, Each layer of the decision tree filters and refines the data according to the sensor input conditions, and finally outputs the corresponding action.

Images