A hand key firing training device and training method

Abstract

The application discloses a hand key shooting training device and a training method, wherein the hand key shooting training device comprises a multi-modal sensing unit, which is used for non-contact acquisition of operator action information and hand key signal information in a hand key shooting process; a calculation processing unit, which is connected with the multi-modal sensing unit and is used for data processing and analysis; the calculation processing unit comprises a data acquisition and synchronization layer, which drives the multi-modal sensing unit to perform multi-modal data acquisition, time-aligns the acquired data, and outputs original physical signals; a multi-modal sensing processing layer, which converts the original physical signals output by the data acquisition and synchronization layer into structured sensing data with semantics; an expert rule diagnosis layer, which diagnoses the structured sensing data based on preset diagnosis rules, and outputs diagnosis results; and an interactive feedback and data management layer, which converts the diagnosis results into interactive content for different roles.

Description

Technical Field

[0001] This invention relates to a hand-keyboard tapping training device and training method. Background Technology

[0002] Existing shortwave Morse code hand-keyed transmission training technologies can be mainly divided into three categories. The first category is training devices based on electrical signal timing analysis. This type of technology only collects the on / off electrical signals of the hand-key contacts and uses a computer to analyze timing parameters such as the duration and interval of the dots and strokes to evaluate the transmission quality. However, this type of solution relies entirely on mathematical processing at the electrical signal level and does not involve any motion perception or analysis capabilities.

[0003] The second type is training devices assisted by contact sensors. These devices acquire some motion parameters by attaching contact sensors to the keys or the student's hands. However, their perception dimensions are limited. Pressure sensors only acquire a single resultant force value and cannot distinguish between wrist force, finger force, and upper arm force. They cannot acquire key motion information such as finger joint angles, lateral wrist swing, and key grip shape. Furthermore, the diagnostic conclusions still require manual interpretation of images and do not have automatic diagnostic capabilities.

[0004] The third category is assessment methods based on EEG or other physiological signals. However, these methods require wearing EEG electrodes and are invasive / contact physiological measurements, which cannot provide real-time diagnostic feedback at the motor level during keystroke training.

[0005] The existing training devices have the following problems: the diagnostic mode is "device collects data → terminal plots curves → human interpretation of the graphs", the same abnormal electrical signal (such as "shorter stroke duration") may be caused by completely different movement reasons (insufficient wrist movement range, loose middle finger support causing premature button rebound, premature rebound, etc.), which cannot be distinguished by the curves alone; the diagnostic quality depends entirely on the personal experience of the person interpreting the graphs, and the evaluation standards are inconsistent among different instructors; and it is impossible to provide real-time diagnostic feedback during training, all of which are post-analysis.

[0006] Currently, all existing training devices can only acquire electrical signal timing and limited single-point force / displacement parameters. They cannot acquire complete three-dimensional motion information of the operator's hand, including the spatial coordinates of 21 hand joints, the bending angle of each finger joint, the wrist's motion trajectory in three-dimensional space, the hand posture when gripping the keys, and the coordination patterns between fingers. The core causes of "bad hand" (incorrect muscle memory) in keystrokes—incorrect finger-wrist coordination patterns, insufficient middle finger support, lateral wrist swing, and improper force application—are precisely reflected in these undetected motion details. Summary of the Invention

[0007] The purpose of this invention is to provide a hand-keyboard shooting training device and training method to solve the technical problems mentioned in the background art.

[0008] The technical solution to achieve the objective of this invention is: a hand-keyboard shooting training device, comprising: A multimodal sensing unit is used for non-contact acquisition of operator action information and key signal information during the key-pressing process; Computational processing unit: connected to the multimodal sensing unit, used to perform data processing and analysis; The computation processing unit includes: Data Acquisition and Synchronization Layer: Drives the multimodal sensing unit to acquire multimodal data, performs time alignment on the acquired data, and outputs the original physical signal; Multimodal sensing processing layer: transforms the raw physical signals output by the data acquisition and synchronization layer into semantically meaningful structured sensing data; Expert rule diagnostic layer: Diagnoses structured perception data based on preset diagnostic rules and outputs diagnostic results; Interactive Feedback and Data Management Layer: Transform diagnostic results into interactive content for different roles.

[0009] Furthermore, the multimodal perception unit includes a visual perception module, an electrical signal perception module, and an audio perception module; the visual perception module acquires the three-dimensional motion information of the operator's hand using image acquisition; the electrical signal perception module is used to acquire the on / off state information of the key touch points and serve as a time synchronization reference for the multimodal data; the audio perception module acquires the audio of the training environment.

[0010] Furthermore, the expert rule diagnostic layer includes a four-level rule engine, which transforms qualitative diagnostic knowledge from professional knowledge into quantifiable physical condition judgment rules.

[0011] Furthermore, the four-level rule engine includes, in sequence: code timing rules, force application rules, force exertion mode rules, and rhythm psychology rules.

[0012] Furthermore, the multimodal perception processing layer includes a hand 3D skeleton extraction submodule and an electrical signal timing parsing submodule. The hand 3D skeleton extraction submodule is used to extract the 3D spatial coordinates of the operator's hand joints from visual data, and the electrical signal timing parsing submodule is used to parse operation timing information from electrical signals.

[0013] Furthermore, the computational processing unit also includes a deep learning evaluation layer, which trains a deep learning model using machine-labeled samples. The machine-labeled samples are derived from diagnostic labels and raw perceptual data packaged by the expert rule diagnostic layer during diagnosis.

[0014] A manual keystroke training method includes the following steps: The operator's action information and key signal information during the key-pressing process are collected non-contactly using multimodal sensing. The collected multimodal data is time-synchronized and aligned and features are extracted to generate structured perception data; Based on a pre-defined multi-level rule base, structured perception data is analyzed from multiple dimensions to generate diagnostic results; The diagnostic results are then translated into feedback information tailored to different roles.

[0015] Furthermore, it also includes distinguishing personal style and skill biases based on statistical analysis methods; the statistical analysis methods include at least one of statistical feature analysis, character association analysis, and time trend analysis.

[0016] Furthermore, the process of converting diagnostic results into feedback information for different roles includes: outputting speech correction information using a graded prompting strategy, wherein the graded prompting strategy determines the prompting level and timing based on the severity and frequency of the problem.

[0017] By adopting the above technical solution, the present invention has the following beneficial effects: (1) This invention acquires the three-dimensional spatial coordinate sequence of 21 joints of the hand for the first time through a binocular stereo vision scheme, covering complete motion information such as the three-dimensional motion trajectory of the wrist, the bending angle of each finger joint, the hand posture of holding the key, and the speed / acceleration of the fingertips. All sensors do not have physical contact with the operator's hand or the key, do not change any operating habits, and extend the diagnostic ability of key tapping training from "whether the result is right or wrong" (at the level of electrical signals) to "why it is wrong" (at the level of action reasons).

[0018] (2) This invention transforms the highly structured diagnostic knowledge from telecom administration textbooks and expert experience into 54 quantifiable physical condition judgment rules, which run independently on the edge in the form of a four-level modular rule engine. It can provide complete real-time diagnostic and speech correction capabilities when deployed in the first pilot classroom without requiring any historical training data or labeled samples.

[0019] (3) This invention uses a three-dimensional stripping method based on statistical features to distinguish between the operator’s reasonable “personal style” and the “skill deviation” that needs to be corrected. This avoids forcibly intervening in the reasonable personal style that should be protected and solves the fundamental problem that the one-size-fits-all standardized evaluation in traditional training may stifle the personalized development of trainees. Attached Figure Description

[0020] To make the content of this invention easier to understand, the invention will be further described in detail below with reference to specific embodiments and accompanying drawings, wherein... Figure 1 This is a system structure block diagram of the present invention.

[0021] Figure 2 This is a flowchart of the manual key-tapping training method of the present invention. Detailed Implementation

[0022] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0023] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0024] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0025] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0026] In the description of the embodiments of the present invention, it should be understood that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship commonly used when the product of the invention is in use, or the orientation or positional relationship commonly understood by those skilled in the art. They are only used to facilitate the description of the present invention and to simplify the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the present invention.

[0027] In the description of the embodiments of the present invention, it should also be noted that, unless otherwise explicitly specified and limited, the terms "set," "install," "connect," and "link" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal communication between two components. Those skilled in the art can understand the specific meaning of the above terms in the present invention based on the specific circumstances. The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and should not be used to limit the scope of protection of the present invention.

[0028] Example 1, see Figure 1 The hand-keyboard shooting training device of this embodiment includes: A multimodal sensing unit is used for non-contact acquisition of operator action information and key signal information during the key-pressing process; Computational processing unit: connected to the multimodal sensing unit, used to perform data processing and analysis; The computation processing unit includes: Data Acquisition and Synchronization Layer: Drives the multimodal sensing unit to acquire multimodal data, performs time alignment on the acquired data, and outputs the original physical signal; Multimodal sensing processing layer: transforms the raw physical signals output by the data acquisition and synchronization layer into semantically meaningful structured sensing data; Expert rule diagnostic layer: Diagnoses structured perception data based on preset diagnostic rules and outputs diagnostic results; Interactive Feedback and Data Management Layer: Transform diagnostic results into interactive content for different roles.

[0029] Specifically, the multimodal sensing unit includes a visual sensing module, an electrical signal sensing module, and an audio sensing module.

[0030] The visual perception module is a binocular high-speed vision module that acquires complete motion information of the operator's hand in three-dimensional space in a non-contact manner. It employs two identical global shutter CMOS image sensors (each with a resolution of at least 640×480 and a frame rate of at least 120FPS), mounted parallel to each other on the front panel of the sensor at a preset baseline distance (55-65mm), paired with a fixed focal length wide-angle lens (approximately 90° field of view). Both cameras are triggered for exposure by the same PWM clock source, ensuring frame-level synchronization of the left and right eye images. The reason for choosing a global shutter instead of a rolling shutter is to eliminate the rolling shutter distortion caused by high-speed downward wrist strikes (speeds up to 0.5-1.0m / s); the frame rate of at least 120FPS is chosen to capture at least 6 images within the approximately 50ms duration of a single point movement at a firing speed of 20WPM (words per minute), reliably capturing the peak vertical velocity of the downward wrist strike and the inflection point of the contact point closure.

[0031] The binocular high-speed vision module is connected to the edge computing unit through a dual-channel MIPI CSI interface, outputting synchronized left and right eye image frames to the data acquisition and synchronization layer; after passing through the multimodal perception processing layer, it outputs a three-dimensional spatial coordinate sequence of 21 hand joints.

[0032] The electrical signal sensing module is the electrical signal acquisition module, serving as the system's "absolute time reference" to accurately record the physical opening and closing moments of the key contacts. The on / off states of the key contacts are unaffected by optical factors such as illumination, obstruction, or motion blur; the edges of the electrical signals precisely mark the moments of contact state changes. The module uses an ARM Cortex-M series microcontroller as the acquisition master, and is optically isolated from the key contacts (with a withstand voltage of not less than 2500Vrms). An RC filter circuit (R=1kΩ, C=0.1μF) is connected in parallel at the input to eliminate mechanical chatter. The microcontroller polls the optocoupler output status at a frequency of not less than 1000Hz. When a level transition is detected, it records the event type (closed / open) and the local timer count value (resolution ≤1μs), reporting this to the end-side computing unit via the UART interface. Optical isolation also provides electrical safety protection, eliminates signal ground loops, and ensures electrical compatibility with different key contact models.

[0033] The electrical signal event sequence output by the electrical signal acquisition module serves as the "absolute time anchor" for the fusion of the three-channel data. The visual and audio data are interpolated and aligned to the most recent electrical signal event according to the timestamp. The edge computing unit sends a synchronization pulse to the microcontroller once per second to correct the clock deviation between the two sides. After correction, the alignment accuracy is controlled within 1ms.

[0034] The audio perception module is an audio acquisition module that synchronously acquires training environment audio through an array of no less than two MEMS microphones at a sampling rate of no less than 48kHz. In a classroom environment where multiple students are simultaneously recording audio, a beamforming algorithm is used to form a pickup beam pointing directly in front of the instructors, providing high-quality audio input for semantic extraction of instructor guidance speech and analysis of training environment noise.

[0035] The audio acquisition module is connected to the on-chip audio codec chip of the edge computing unit via the I2S interface. The audio frames are written with a hardware timestamp at the beginning of each frame by the ALSA driver layer and then transferred to memory via DMA. The timestamp is aligned with the visual and electrical signal data in the data acquisition and synchronization layer.

[0036] The computing unit is an edge-side computing unit, serving as the sole carrier for all sensor computing tasks. It employs a multi-core heterogeneous SoC (integrating a high-performance CPU core group, a high-efficiency CPU core group, and an NPU with at least 6 TOPS INT8 computing power), configured with at least 8GB of system memory and at least 64GB of storage. In Phase One (cold start), this unit simultaneously handles three-way data acquisition driving, stereo vision hand 3D keypoint inference (NPU acceleration), electrical signal event reception and timestamp correction, conditional matching and diagnostic output of the expert rule engine, TTS synthesis and playback of voice correction prompts, and reporting of structured training data. This level of computing power was chosen for the SoC to ensure that all the above workloads are carried out simultaneously on a single chip, achieving independent, complete, and deliverable Phase One products without relying on any external servers or networks.

[0037] The edge computing unit is the physical carrier connecting all hardware sensors and all software layers. In Phase 1, it plays the role of "all-around central hub". In Phase 3, its role evolves into "front-end data preprocessing and feature extraction computing unit".

[0038] The data acquisition and synchronization layer drives the concurrent acquisition of three sensors. The hardware timer of the edge computing unit serves as the global unified clock source. In the driver layer, each frame of visual data, each electrical signal event, and each frame of audio data is stamped with a hardware timestamp with nanosecond / microsecond precision, achieving precise time alignment of the three heterogeneous data streams.

[0039] The data acquisition and synchronization layer is the upstream data source for the multimodal sensing and processing layer. When the three data streams are fused downstream, electrical signal events are used as the absolute time anchor.

[0040] The multimodal sensing processing layer transforms the raw physical signals output from the data acquisition and synchronization layer into semantically structured data. Specifically, it includes two sub-modules: The 3D hand skeleton extraction submodule: Simultaneous images from the left and right eyes are fed into the hand keypoint detection model (inferenced on the NPU with INT8 precision), outputting the 2D pixel coordinates of 21 keypoints per frame. Using pre-calibrated dual-camera calibration parameters (intrinsic matrix, distortion coefficients, left and right camera poses), epipolar correction simplifies the stereo matching problem from a 2D search to a 1D epipolar search. Normalized cross-correlation criteria are used to confirm the correspondence between left and right eye keypoint pairs, and linear triangulation is used to calculate the 3D spatial coordinates of the point. For unilateral keypoint loss caused by finger self-occlusion and button overlap, a two-level completion strategy is adopted: within a single frame, the position of the occluded point is inferred from the visible point based on the hand skeleton kinematic model; across frames, Kalman filtering is applied to smooth the 3D trajectory of each keypoint.

[0041] Electrical signal timing analysis submodule: Performs contact jitter filtering on the original electrical signal event sequence (continuous flipping events with an interval less than a preset threshold are considered as mechanical jitter merging), and adopts an adaptive threshold strategy for dot and stroke recognition: the median of the event duration within the sliding window is used as the estimated unit dot length at the current speed, and the event type is determined accordingly.

[0042] The multimodal perception processing layer outputs a 21-point 3D coordinate sequence and a standardized dot-dotted time-series object to the expert rule diagnosis layer, which is a key transformation link connecting "physical signals" and "diagnostic semantics".

[0043] The expert rule diagnostic layer, without requiring any labeled training data, performs multi-dimensional real-time diagnosis of the operator's firing action based on structured physical condition judgment rules extracted from communication textbooks, military training standards, and expert experience. It outputs standardized diagnostic results (problem type, severity, location index, and correction suggestion text) and simultaneously outputs machine-labeled samples (AutoLabeledSample, raw perception data + diagnostic labels) for subsequent model training.

[0044] The expert rule diagnosis layer includes a four-level rule engine, which consists of: code timing rules, force application rules, force exertion pattern rules, and rhythm and psychology rules.

[0045] The expert rule-based diagnostic layer is the core diagnostic capability carrier in Phase One. It receives structured perception data from the multimodal perception processing layer, performs step-by-step matching through a four-level rule base, and sends the diagnostic results to the interactive feedback and data management layer for speech correction and training report generation. At the same time, it transmits machine-labeled samples to the training data management module.

[0046] The deep learning evaluation layer will be gradually enabled starting from Phase Two (data accumulation). Utilizing the large number of machine-labeled samples generated by the expert rule diagnostic layer's rule engine in Phase One, two types of neural network models will be trained: a "rule-learning" model (a CNN-LSTM hybrid architecture, learning the classification and judgment capabilities of the rule engine, approximately 2 million parameters) and a "feeling-learning" model (a 3D-CNN+Transformer hybrid architecture, learning the ability to continuously score comprehensive quality, approximately 15 million parameters). The "rule-learning" model, after knowledge distillation and INT8 quantization, will be deployed on the edge NPU to replace part of the rule engine's CPU inference tasks; the "feeling-learning" model will be deployed on the training server / cloud. This layer will not be enabled in Phase One.

[0047] The training data for the deep learning evaluation layer comes from the automatic annotation output of the expert rule diagnostic layer rule engine. After training, the edge model is updated through the model OTA delivery mechanism, and the edge expert rule diagnostic layer rule engine is always retained as an offline backup when the model confidence is insufficient.

[0048] The interactive feedback and data management layer transforms diagnostic results into interactive content for different roles (students / instructors). This includes: real-time voice correction (following a tiered prompting strategy that "does not interrupt the training flow"; occasional minor deviations are recorded without prompting; a brief reminder is triggered when the same problem occurs three or more times consecutively; serious problems are immediately prompted; the minimum interval between two voice prompts is preset to 15 seconds); automatic generation of structured evaluation summaries after training (including overall score, dimensional scores, comparison with historical best, list of main problems, and targeted improvement suggestions); and an instructor monitoring panel (a matrix view of all students in the classroom, real-time rendering of 3D hand skeleton animation for a single student, playback of electrical signal waveforms, and an interface for reviewing, correcting, and annotating AI diagnostic results).

[0049] The speech correction text for interactive feedback and the data management layer originates from the standardized correction suggestions output by the expert rule diagnostic layer's rule engine, which is then converted into natural speech output in real time by the TTS speech synthesis engine. The instructor's review and correction annotations are written back to the data center as "gold standard data" for rule engine threshold calibration and model training.

[0050] Example 2, see Figure 2 The manual key-press training method of this embodiment includes: The operator's action information and key signal information during the key-pressing process are collected non-contactly using multimodal sensing. The collected multimodal data is time-synchronized and aligned and features are extracted to generate structured perception data; Based on a pre-defined multi-level rule base, structured perception data is analyzed from multiple dimensions to generate diagnostic results; The diagnostic results are then translated into feedback information tailored to different roles.

[0051] Specifically, the three sensors collect data concurrently using their respective sampling mechanisms. In the data acquisition and synchronization layer, a hardware timer is used as a unified clock source to timestamp the data and align the three data streams. The multimodal perception processing layer transforms the raw physical signals into structured perception data through stereo vision reasoning and electrical signal analysis.

[0052] Structured sensing data is simultaneously fed into four rule sub-modules, each independently performing condition matching within its jurisdiction. The first-level code timing rule is a pure electrical signal determination, outputting diagnostic conclusions such as dot quality and interval regularity without relying on motion data. The second to fourth levels gradually increase the involvement of kinematic data, deepening the diagnostic dimensions from "correct or incorrect results" to "causes of force" and "trend warnings." The diagnostic results from the four modules are merged into a unified multi-dimensional diagnostic conclusion at the integration node.

[0053] The diagnostic results are categorized into three levels of severity for handling: severe issues (dummy points, consecutive codes, adhesion, etc.) trigger an immediate voice prompt; three or more consecutive instances of the same moderate issue trigger a brief reminder; occasional minor deviations are recorded silently. After training, a structured evaluation summary containing multi-dimensional data and historical comparisons is automatically generated.

[0054] After each effective training session, the trainee's individual movement baseline is dynamically updated using an exponential moving average strategy. A comprehensive assessment is conducted across three dimensions: statistical characteristic analysis (global mean deviation but small variance = personal style), character correlation analysis (errors concentrated on specific characters = early signs of poor performance), and time trend analysis (continuously widening deviation = requiring early warning and intervention). This distinguishes between "personal style" that needs protection and "skill deviation" that needs correction. Systematic deviations are characterized by mean deviation but small variance (the operator consistently deviates in the same way, showing good movement consistency), while random errors are characterized by large local variance (errors are unstable, unpredictable, and often related to specific characters or fatigue). The assessment is based on a comprehensive evaluation across these three dimensions. Statistical feature dimension: Calculate the global mean μ and global variance σ² for each action feature. If |μ-μ_pop| is large (deviates from the group standard) but σ² is small (consistent with itself), it is judged as a systematic personal style and is not corrected; if σ² is large and the deviation is concentrated in a specific character segment, it is judged as an unstable error and needs to be corrected.

[0055] Character association dimension: Errors evenly distributed across all characters indicate insufficient overall movement pattern formation for beginners (systematic slowdown for consolidation, rather than correction of individual characters); errors highly concentrated on a few specific characters indicate early signs of poor technique targeting those characters (requires separate slow practice and focused monitoring).

[0056] Time trend dimension: A continuous reduction in deviation is considered a normal learning process (mainly encouraged); a continuous increase in deviation is considered a need for early warning and intervention.

[0057] While performing real-time diagnosis, the rule engine packages diagnostic labels and raw perception data into AutoLabeledSample machine-labeled samples. These samples are continuously accumulated in Phase 1, and then fed into the model training server in Phase 2 to train two types of neural network models: "learning rules" and "learning senses," achieving a gradual evolution of capabilities from "knowledge-driven" to "data-driven."

[0058] The core innovation of the rule engine lies in "translating" qualitative diagnostic knowledge described in natural language from communications textbooks, military training standards, and the experience of senior instructors into computer-executable, physically-measure-based quantitative conditional judgment logic. The following three representative diagnostic rules illustrate this "translation" method: Example 1: Quantitative Diagnosis of "Virtual Points" The textbook describes the issue as "the touch is light and weak... the downward force of the wrist is not concentrated, the finger support is insufficient, and the force is not effectively transmitted to the key." An experienced instructor adds: the electrical signal characteristics of a false touch are "the pulse width is significantly lower than the standard," and the action characteristics are "the speed does not reach its peak at the moment of contact closure during the downward stroke, but decays prematurely."

[0059] This translates to three quantifiable physical conditions (the "virtual point" diagnosis is triggered when all three conditions are met simultaneously): Condition 1 (Electrical signal dimension): The pulse width at the current point is less than 70% of the standard point width at the student's current speed; Condition 2 (Kinematic Dimension): At the moment of contact closure, the absolute value of the vertical velocity of the wrist joint is less than 60% of the student's peak hitting speed below baseline; Condition 3 (kinematic dimension): From 10ms before the contact point closes to the moment of closure, the slope of the vertical velocity of the wrist is positive (i.e. the velocity is already decaying, which is the physical manifestation of "the force is not concentrated at the moment of contact point closure").

[0060] Example 2: Quantitative diagnosis of "insufficient support leading to shortened strokes" The instruction manual describes the stroke as "too short a duration, too long near the point... After the wrist strikes, the fingers can't support the button, causing the button to rebound prematurely." An experienced instructor adds: The middle finger's support is crucial for transferring wrist power to the button; insufficient support leads to unstable strokes.

[0061] Translated into three quantifiable physical conditions: Condition 1 (Electrical signal dimension): The pulse width of the current stroke is less than 70% of the standard stroke width at the current speed; Condition 2 (kinematic dimension): During the contact closure, the Z-coordinate of the proximal interphalangeal joint of the middle finger shows a continuous upward displacement trend (displacement > 2mm) – indicating that the middle finger relaxes during the "holding" phase, causing the button to rebound prematurely. This condition is also the key to distinguishing between "short strokes caused by insufficient support" and "intentional short strokes by the student" – the latter has the same electrical signal characteristics, but the middle finger joint will not show an upward relaxation displacement; Condition 3 (Comprehensive Dimension): The contact closure time is less than 85% of the normal stroke length of the student at the same speed, confirming that it is not an accidental fluctuation.

[0062] Example 3: Quantitative Diagnosis of "Lateral Wrist Swing" The experienced instructor's "three horizontal lines" theory emphasizes that "wrist movements are very vertical, without any issues of pushing forward or pulling backward, or spreading outward or tilting inward."

[0063] Translated into quantitative conditions: In a series of preset number (e.g., 10) or more movement cycles, the average displacement component of the wrist joint in the horizontal plane (XY plane) exceeds 25% of the vertical (Z axis) displacement component, and this deviation has a large statistical variance (unstable) rather than a systematic stable offset (the latter is a personal style and will not be corrected). The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A hand-keyboard shooting training device, characterized in that, include: A multimodal sensing unit is used for non-contact acquisition of operator action information and key signal information during the key-pressing process; Computational processing unit: connected to the multimodal sensing unit, used to perform data processing and analysis; The computation processing unit includes: Data Acquisition and Synchronization Layer: Drives the multimodal sensing unit to acquire multimodal data, performs time alignment on the acquired data, and outputs the original physical signal; Multimodal sensing processing layer: transforms the raw physical signals output by the data acquisition and synchronization layer into semantically meaningful structured sensing data; Expert rule diagnostic layer: Diagnoses structured perception data based on preset diagnostic rules and outputs diagnostic results; Interactive Feedback and Data Management Layer: Transform diagnostic results into interactive content for different roles.

2. The hand-keyboard shooting training device according to claim 1, characterized in that: The multimodal perception unit includes a visual perception module, an electrical signal perception module, and an audio perception module. The visual perception module acquires three-dimensional motion information of the operator's hand using image acquisition. The electrical signal perception module is used to acquire the on / off state information of the key touch points and serves as a time synchronization reference for the multimodal data. The audio perception module acquires audio from the training environment.

3. The hand-keyboard shooting training device according to claim 1, characterized in that: The expert rule diagnostic layer includes a four-level rule engine, which transforms qualitative diagnostic knowledge from professional knowledge into quantifiable physical condition judgment rules.

4. The hand-keyboard shooting training device according to claim 3, characterized in that: The four-level rule engine includes, in order: code timing rules, force application rules, force exertion mode rules, and rhythm psychology rules.

5. The hand-keyboard shooting training device according to claim 1, characterized in that: The multimodal perception processing layer includes a hand 3D skeleton extraction submodule and an electrical signal timing parsing submodule. The hand 3D skeleton extraction submodule is used to extract the 3D spatial coordinates of the operator's hand joints from visual data, and the electrical signal timing parsing submodule is used to parse operation timing information from electrical signals.

6. The hand-keyboard shooting training device according to claim 1, characterized in that: The computational processing unit also includes a deep learning evaluation layer, which trains a deep learning model using machine-labeled samples. The machine-labeled samples are derived from diagnostic labels and raw perceptual data packaged by the expert rule diagnostic layer during diagnosis.

7. A manual key-tapping training method, used in the manual key-tapping training device as described in any one of claims 1-6, characterized in that, Includes the following steps: The operator's action information and key signal information during the key-pressing process are collected non-contactly using multimodal sensing. The collected multimodal data is time-synchronized and aligned and features are extracted to generate structured perception data; Based on a pre-defined multi-level rule base, structured perception data is analyzed from multiple dimensions to generate diagnostic results; The diagnostic results are then translated into feedback information tailored to different roles.

8. The manual key-tapping training method according to claim 7, characterized in that, It also includes distinguishing personal style and skill bias based on statistical analysis methods; the statistical analysis methods include at least one of statistical feature analysis, character association analysis and time trend analysis.

9. The manual key-tapping training method according to claim 7, characterized in that, The process of converting diagnostic results into feedback information for different roles includes: using a graded prompting strategy to output speech correction information, wherein the graded prompting strategy determines the prompting level and timing based on the severity and frequency of the problem.

Images