Method and device for training chinese characters suitable for children based on intelligent terminal equipment

Abstract

The application relates to the technical field of human-computer interaction, and discloses a method and device for training Chinese character structures suitable for children based on intelligent terminal equipment, which divides a standard character display area, a spelling work area and a stroke supply area on a display interface of the intelligent terminal equipment; displays only one stroke image of a current stroke to be moved in sequence in the stroke supply area; collects a video stream of a user's hand region, identifies an overhand movement track and a gesture category of the user's fingers; locks the stroke image of the current stroke to be moved and moves the stroke image in the spelling work area along the overhand movement track of the fingers according to a response to an identified pointing gesture or a grabbing gesture; fixes the stroke image at a current screen position and automatically refreshes a stroke image corresponding to a next stroke order to the stroke supply area according to a response to an identified releasing gesture; and repeats until a combined Chinese character image is formed; and compares the combined Chinese character image with a standard Chinese character image in structure similarity, and outputs a training result. The application can improve the accuracy of user's Chinese character structure cognition and learning immersion.

Description

Technical Field

[0001] This invention relates to the field of human-computer interaction technology, specifically to a method and apparatus for training children in Chinese character structure based on a smart terminal device. Background Technology

[0002] Chinese calligraphy is an important carrier of Chinese culture, and "character structure" (i.e., the spatial layout of Chinese character strokes) is widely recognized as one of the core elements of calligraphy learning. With the popularization of smart terminals, calligraphy training systems based on electronic screens have gradually become an important tool to assist children in recognizing characters and learning calligraphy.

[0003] In existing technologies, electronic calligraphy training solutions mainly follow two technical paths. The first is the "tracing-writing" path, such as the text writing display and training methods disclosed in CN113627408B and CN114092943B. These methods divide the screen into display and training areas, obtain the writing trajectory by acquiring the user's screen touch input, and compare it with the standard stroke order trajectory for scoring. This type of solution is essentially an electronic extension of traditional paper-and-pen tracing; the user needs to complete the writing action on the screen with their finger or stylus. The second is the "stroke comparison" path, such as the method for writing text on a touchscreen disclosed in CN101551724B. This method receives the strokes to be compared written by the user on the touchscreen, compares the stroke structure attribute values ​​and shape attribute values ​​with standard strokes, and outputs prompt information. This type of solution also relies on physical contact input via a multi-touch screen.

[0004] However, the existing touch-based interaction technologies mentioned above share the following common drawbacks: First, during the touch and dragging process, fingers or styluses inevitably obscure the stroke blocks and target areas on the screen. Users need to repeatedly remove their fingers to observe the alignment, resulting in poor operational continuity. This is especially true for preschool children and other groups with weaker fine motor control, significantly reducing positioning efficiency. Second, when a large number of strokes have been placed in the learning frame, if a user needs to fine-tune the position of an earlier placed stroke, accidental touches often occur due to overlapping strokes and a large touch point area, leading to high costs for backtracking and adjustment. Third, existing touch interaction methods are simplistic and fail to fully consider the cognitive and behavioral characteristics of young children. When using touchscreens, young children's hand bones and muscles are not yet fully developed, making them prone to irregular rapid shaking, directional confusion, or unconscious jerking. Existing systems lack mechanisms to identify and correct such disruptive movements, resulting in a high training failure rate, which in turn weakens learning interest and training effectiveness. Summary of the Invention

[0005] The present invention aims to provide a method and apparatus for training children in Chinese character structure based on smart terminal devices, in order to solve the technical problems of existing touch-screen calligraphy training systems, such as fingers obstructing the view, difficulty in retrospective adjustment of placed strokes, and high failure rate of operation due to immature fine motor skills.

[0006] To solve the above problems, the present invention adopts the following technical solution: Option 1: A method for training children on Chinese character structure based on smart terminal devices, including the following steps: S1, obtain the standard stroke data and standard stroke order of the target Chinese character; S2 divides the display interface of smart terminal devices into a standard character display area, a character spelling work area, and a stroke supply area. S3, according to the standard pen sequence, display only the currently moved pen image in the stroke supply area stroke by stroke; S4. The video stream of the user's hand area is collected in real time by the front image acquisition device of the smart terminal device. The air movement trajectory and gesture category of the user's fingers are identified by computer vision algorithm. The gesture category includes at least pointing gesture, grasping gesture, dragging gesture and releasing gesture. S5. Based on the recognized pointing or grasping gesture, lock the currently moving stroke image and move it in the spelling work area following the air movement trajectory of the finger. S6, based on the release gesture detected in the response, fix the stroke image at the current screen position, and automatically refresh the stroke image corresponding to the next stroke order to the stroke supply area. S7. Repeat S5 and S6 until all strokes are moved in the standard stroke order to form a combined Chinese character image. S8, perform structural similarity comparison between the combined Chinese character image and the standard Chinese character image in the standard character display area, and output the training result; In the process of recognizing the trajectory of a user's finger in mid-air, the spatiotemporal characteristics of the finger's movement trajectory are analyzed in real time. When it is determined that the user's action is an irregular interference action, the follow-up response of the current stroke image is paused, and corresponding action correction prompts are generated.

[0007] Beneficial effects: This invention replaces traditional touch-based tracing with non-touch-based air splicing, overcoming the long-standing technical bias in calligraphy training that "must simulate writing actions"; the stroke-by-stroke sequential supply mechanism seamlessly embeds stroke order rules into the splicing process; real-time detection and pause correction of irregular interference actions enable the system to proactively intervene rather than passively respond when users unconsciously shake or tremble, significantly reducing user shaking and trembling habits, especially the failure rate of operation caused by immature hand development in young children, achieving training adaptation for children of all ages over 3 years old.

[0008] Furthermore, the real-time analysis of the spatiotemporal characteristics of finger movement trajectories to determine whether the user's action belongs to irregular interference actions includes: Condition 1: Calculate the instantaneous displacement velocity of the finger between adjacent video frames. When the instantaneous displacement velocity exceeds the preset velocity threshold Vmax, it is determined to be a rapid shaking interference. The value range of the velocity threshold Vmax is 300 to 600 pixels / second. Condition 2: Calculate the change angle of the finger's movement direction within a preset time window T. When the change angle exceeds a preset angle threshold θ, it is determined to be a direction disorder interference. The value range of the angle threshold θ is 90 to 150 degrees. Condition 3: Calculate the radius of curvature of the finger movement trajectory. When the radius of curvature is less than the preset radius threshold Rmin, it is determined to be trajectory jitter interference. The radius threshold Rmin ranges from 30 to 80 pixels. When any of the above conditions are met and the duration exceeds the preset duration threshold t, the user action is determined to be an irregular interference action. The duration threshold t ranges from 0.3 to 0.8 seconds.

[0009] Beneficial effects: By quantifying three typical user interference actions into criteria with specific numerical ranges (speed threshold 300-600 pixels / second, angle threshold 90-150 degrees, radius threshold 30-80 pixels, duration 0.3-0.8 seconds), the judgment of irregular interference is transformed from qualitative description to reproducible mathematical calculation. This avoids the system's oversensitivity to normal exploratory actions of users, especially young children, while ensuring that interference behaviors that truly affect operational accuracy are captured in a timely manner, thus improving the objectivity and stability of detection.

[0010] Furthermore, the generation of corresponding action correction prompts includes: When a rapid shaking interference is detected, a deceleration prompt is output, and the stroke image following sensitivity is reduced to the first gain coefficient, which ranges from 0.4 to 0.7. When the direction is determined to be chaotic and interfered with, a directional guidance prompt is output, and a dynamic directional arrow is displayed at the target position in the spelling work area. When the trajectory jitter interference is detected, a stable gesture prompt is output, and at the same time, a moving average filtering algorithm is used to smooth the finger movement trajectory. The window frame number of the moving average filtering is 3 to 5 frames.

[0011] Beneficial effects: Differentiated correction strategies are implemented for three types of interference: rapid shaking, directional confusion, and trajectory jitter. A gain coefficient of 0.4-0.7 is used to suppress the following sensitivity to match the deceleration needs of young children. Dynamic guide arrows provide visual direction anchoring to correct spatial disorientation. High-frequency hand tremor noise is eliminated by 3-5 frame moving average filtering. The three strategies work together to help children gradually establish stable and standardized muscle memory for air gesture control during training.

[0012] Furthermore, the method also includes: S41 receives voice commands from the user that include stroke order numbers; S42, based on the stroke order number, find the corresponding placed stroke image in the spelling work area, mark the stroke image with a highlighted border and a blinking breathing light, and set it to an adjustable state; S43, in response to the trajectory of the user's finger movement in the air, adjust the position of the locked stroke image in the spelling work area; S44, in response to a release gesture or a confirmed voice command, unlocks the stroke image and fixes it in a new position.

[0013] Beneficial effects: By binding voice stroke order numbering with air gestures across modalities, users can lock the target stroke in the air simply by speaking the number, without having to perform millimeter-level touch selections on densely stacked placed strokes. Combined with air fine-tuning, it completely solves the structural pain points of "many strokes, mutual occlusion, and easy accidental touches" in touch solutions, improving the efficiency of retrospective adjustment of placed strokes by more than 3 times.

[0014] Furthermore, the smart terminal device is placed on a horizontal support surface by a fixed bracket, and the front image acquisition device maintains a constant acquisition distance of 20cm to 40cm between itself and the user's hand area; The computer vision algorithm processes the video stream frame by frame at a frequency of no less than 30 frames per second, and the mapping accuracy of the finger's aerial motion trajectory in the screen's two-dimensional coordinate system reaches the pixel level.

[0015] Beneficial effects: By establishing a constant acquisition distance of 20cm to 40cm through a fixed bracket, the recognition distance drift caused by device shaking or unstable holding by children is avoided; combined with a frame-by-frame processing rate of no less than 30 frames / second and pixel-level mapping accuracy, it ensures that air gestures have spatial resolution and time response comparable to direct touch control in a normal sitting posture, allowing young children to operate naturally without learning.

[0016] Furthermore, before fixing the stroke image at the current screen position, the pixel distance d between the center coordinates of the current stroke image and the standard center coordinates of the corresponding stroke in the standard Chinese character image is calculated; If d≤D1, then the stroke image is fixed and positive feedback is output, where D1 is 5 to 15 pixels; If D1 < d ≤ D2, then output a position fine-tuning prompt and guide the user to continue adjusting, where D2 is 16 to 30 pixels; If d > D2, the placement is deemed a failure, and the stroke image is restored to the top of the stroke supply area.

[0017] Beneficial effects: A three-level evaluation mechanism is constructed by using a first distance threshold D1 of 5 to 15 pixels and a second distance threshold D2 of 16 to 30 pixels: positive incentives are given directly within the D1 range, error-tolerant guidance is provided between D1 and D2 to avoid frustration, and failure is judged and the stroke is restored if it exceeds D2. This mechanism ensures the rigor of Chinese character structure training while taking into account the psychological acceptance of young children through the error tolerance range of 15 pixels, so that children can maintain their learning motivation even when they make mistakes.

[0018] Furthermore, the method also includes: Obtain the user's age information, query the corresponding interference detection parameters from the preset age-interference threshold mapping table based on the age information, and dynamically adjust the preset speed threshold Vmax, preset angle threshold θ, preset radius threshold Rmin, and the recognition frame rate and response latency of the gesture recognition module. The age-interference threshold mapping table includes: For children aged 3-5, the first parameter group is: Vmax1 is 300 to 400 pixels / second, θ1 is 90 to 120 degrees, Rmin1 is 30 to 50 pixels, the recognition frame rate is not less than 30 frames / second and the finger lock response delay is not more than 100 milliseconds. For children aged 6-8, the second parameter group is: Vmax2 is 400 to 500 pixels / second, θ2 is 100 to 140 degrees, Rmin2 is 40 to 60 pixels, the recognition frame rate is not less than 25 frames / second and the finger lock response delay is not more than 150 milliseconds. For users aged 9 and above, the third parameter group is: Vmax3 is 500 to 600 pixels / second, θ3 is 120 to 150 degrees, Rmin3 is 50 to 80 pixels, the recognition frame rate is not less than 20 frames / second and the finger lock response delay is not more than 200 milliseconds.

[0019] Beneficial effects: By using an age-interference threshold mapping table to configure differentiated parameters for users in three age groups—3-5 years old, 6-8 years old, and 9 years and above—the interference detection threshold is dynamically matched to the children's physiological development level. For children aged 3-5, the strictest interference judgment criteria (minimum speed threshold 300-400 pixels / second, minimum radius threshold 30-50 pixels) and the highest system response performance (no less than 30 frames / second, no more than 100 milliseconds latency) are applied to ensure that their fast but unstable gestures can be recognized promptly and with tolerance. For users aged 9 and above, the thresholds are relaxed and the frame rate requirement is reduced to save computing power. This linkage mechanism enables the same system to accurately adapt to the neuromuscular control characteristics of users in different age groups, significantly improving compatibility across all age groups.

[0020] Option 2: A device for training children's Chinese character structure based on smart terminal devices, applied to the method described in Option 1, including: The image acquisition module captures video streams of the user's hand area in real time through the front-facing image acquisition device of the smart terminal device; The gesture recognition module identifies the air movement trajectory and gesture category of the user's fingers from the video stream based on computer vision algorithms. The gesture category includes at least pointing gestures, grasping gestures, dragging gestures, and releasing gestures. The stroke supply module is used to obtain the standard stroke data and standard stroke sequence of the target Chinese character, and at the same time, it displays the image of the stroke to be moved one by one in the stroke supply area of ​​the display interface based on the standard stroke sequence. The air control module, in response to the recognized pointing or grasping gesture, locks the currently moving stroke image and moves it in the spelling work area following the air movement trajectory of the finger; in response to the recognized release gesture, it fixes the stroke image at the current screen position and automatically refreshes the stroke image corresponding to the next stroke order to the stroke supply area. The structure evaluation module is used to compare the structural similarity of the combined Chinese character images formed in the spelling work area with the standard Chinese character images and output the training results. The interference detection module is used to analyze the spatiotemporal characteristics of the finger movement trajectory in real time during the process of recognizing the user's finger movement trajectory in the air. When it is determined that the user's action is an irregular interference action, the current stroke image follow-up response is paused and corresponding action correction prompt information is generated. The smart terminal device is placed on a horizontal support surface by a fixed bracket, which is used to maintain a constant acquisition distance of 20cm to 40cm between the front image acquisition device and the user's hand area.

[0021] Beneficial effects: Through the collaboration of the image acquisition module, gesture recognition module, air control module and interference detection module, a complete non-contact Chinese character structure training closed loop is constructed; the hardware limitation of the fixed bracket ensures the constantness of the acquisition distance, so that the air gesture recognition has a repeatable geometric reference, avoiding distance drift and recognition inaccuracy caused by handheld devices, and improving the robustness of the device in children's use scenarios.

[0022] Furthermore, it also includes: The voice interaction module is used to receive voice commands from users that include stroke order numbers; The stroke management module is used to find the corresponding placed stroke image in the spelling work area according to the stroke order number, mark the stroke image with a highlighted border and a flashing breathing light, and set it to an adjustable state. The air control module also adjusts the position of the locked stroke image in response to the air movement trajectory of the user's finger, and unlocks it in response to a release gesture or confirmation voice command.

[0023] Beneficial effects: The cross-modal cooperation between the voice interaction module and the stroke management module enables non-contact and accurate tracing of placed strokes; users only need to speak the stroke order number, and the system can highlight and lock the corresponding stroke in the spelling work area and enter the adjustable state. With the help of the air control module, the position can be finely adjusted, which solves the problem of accurate selection of dense strokes that cannot be solved by touch solutions.

[0024] Furthermore, it also includes: An age adaptation module is used to obtain the user's age information, query the corresponding interference detection parameters from a preset age-interference threshold mapping table based on the age information, and dynamically adjust the recognition frame rate and response latency of the gesture recognition module, as well as the preset speed threshold Vmax, preset angle threshold θ, and preset radius threshold Rmin of the interference detection module. The age interference threshold mapping table includes: For children aged 3-5, the first parameter group is: Vmax1 is 300 to 400 pixels / second, θ1 is 90 to 120 degrees, Rmin1 is 30 to 50 pixels, the recognition frame rate is not less than 30 frames / second and the finger lock response delay is not more than 100 milliseconds. For children aged 6-8, the second parameter group is: Vmax2 is 400 to 500 pixels / second, θ2 is 100 to 140 degrees, Rmin2 is 40 to 60 pixels, the recognition frame rate is not less than 25 frames / second and the finger lock response delay is not more than 150 milliseconds. For users aged 9 and above, the third parameter group is: Vmax3 is 500 to 600 pixels / second, θ3 is 120 to 150 degrees, Rmin3 is 50 to 80 pixels, the recognition frame rate is not less than 20 frames / second and the finger lock response delay is not more than 200 milliseconds.

[0025] Beneficial effects: The age-adaptive module enables the device to automatically switch gesture recognition performance parameters and interference detection thresholds according to the user's age, ensuring that children aged 3-5 receive an immediate fault-tolerant response at the highest sensitivity (no less than 30 frames / second, no more than 100 milliseconds latency) and the strictest interference filtering (Vmax1 300-400 pixels / second); at the same time, it enables users aged 9 and above to still use the device smoothly with appropriately reduced performance parameters, realizing adaptive optimization of the device for users of all ages, avoiding the failure of operation for young children or system redundancy for older users caused by "one-size-fits-all" parameters.

[0026] The advantages of this invention are: This invention eliminates the reliance of existing electronic calligraphy training systems on physical contact with writing materials such as touch screens. It achieves non-contact splicing of Chinese character strokes by capturing air gestures through a front-facing camera, fundamentally eliminating inherent structural defects of touch control solutions such as fingers obstructing the screen view, difficulty in retrospectively adjusting placed strokes, and frequent accidental touches by young children due to immature fine motor skills.

[0027] Compared to the existing technology's "post-event evaluation" mode that passively waits for users to finish writing before comparing the results, this invention innovatively uses computer vision algorithms to simultaneously monitor real-time behavior during operation. By analyzing the spatiotemporal characteristics of the finger movement trajectory, such as instantaneous speed, direction change angle, and radius of curvature, it proactively identifies irregular interference actions such as rapid shaking, directional confusion, and trajectory jitter that occur when users, especially young children, are operating in the air. The system pauses its response and outputs targeted correction prompts the moment the interference occurs, placing error prevention at the effective identification stage. This transforms the system's fault tolerance for user, especially young child, behavior from passive post-event remediation to proactive process intervention.

[0028] Furthermore, this invention integrates voice stroke order numbering with air gestures across modalities, enabling users to accurately locate target strokes and fine-tune their positions by simply speaking numbers among densely placed strokes. This completely solves the problem of backtracking and adjustment caused by stroke stacking and limited selection area in touch solutions.

[0029] Most importantly, this invention is not a simple combination of three technologies: air gesture recognition, voice interaction, and interference detection. Instead, it establishes a dynamic mapping relationship between age and interference detection threshold, gesture recognition frame rate, and response latency based on the differences in neuromuscular development among children aged 3 to 5, 6 to 8, and 9 and above. This allows the same system to automatically switch recognition sensitivity and error tolerance standards according to the user's physiological development level, achieving adaptive adaptation for users of all ages. This significantly improves the success rate, smoothness of operation, and immersive learning experience of children's Chinese character structure training.

[0030] A deep-seated technological bias has long existed in the field of calligraphy training: the belief that the essence of calligraphy training is "writing," and that electronic calligraphy training systems must simulate or replace the act of writing with pen and paper. Therefore, when faced with the technical problem of "improving the effectiveness of calligraphy training for young children," those skilled in the art have consistently limited their improvement strategies to the framework of "how to optimize the touch writing experience," such as improving touch sampling accuracy, increasing the number of calligraphy templates, improving stroke comparison algorithms, and increasing the touch hotspot of stroke blocks. Existing technological improvements all serve the purpose of "making users write more accurately on the screen and receive more accurate evaluations." There is no technical motivation to completely eliminate touch writing and replace it with air-based splicing to overcome the problems of immature hand development and easy touch failures in young children, because this direction of improvement contradicts the inherent understanding of "writing" in the field of calligraphy training; it is a departure from the existing technological paradigm rather than an optimization within the framework. Therefore, changing the input method from touch writing to air-based splicing, and changing the training mode from "tracing writing" to "stroke splicing," is not a conventional replacement that those skilled in the art would readily consider.

[0031] The design goal of known gesture recognition technologies (including air gesture recognition) in this field is to improve the accuracy of recognizing valid gestures and reduce the false recognition rate. Their technical approach focuses on "how to correctly understand the user's intent." However, this application uses computer vision algorithms simultaneously to "identify the user's erroneous actions to correct user behavior." Specifically, when a user's action is determined to be an irregular, disruptive action, the system response is paused and a correction prompt is output. This design approach extends gesture recognition technology backward from "intent understanding" to "behavior correction," and is not a conventional design in the field of gesture recognition. Those skilled in the art have no incentive to use algorithms used to identify valid instructions simultaneously to detect and correct invalid user actions, because there is no technical requirement in the prior art to "interrupt the user in real time during operation and correct their actions," nor is there any common knowledge that combines instantaneous velocity, angle of change of direction, and radius of curvature for classifying disruptive behaviors in young children.

[0032] In existing touch control solutions, users can select target objects on the screen by directly tapping. Although there are issues with accidental touches, the conventional improvement approach for those skilled in the art is to optimize selection accuracy (e.g., increasing the hot zone or introducing pressure sensing), rather than introducing voice commands as a selection method. Those skilled in the art have no technical motivation to introduce a voice-activated numbering lock mechanism to solve the difficulty of touch selection, because voice interaction and gesture control belong to different modalities. Combining them requires solving additional technical problems such as command conflicts, timing synchronization, and state machine switching, which is an unnecessary design that increases system complexity. Only after completely eliminating touch and adopting air-based splicing, leading to the new structural problem of "unable to select already placed strokes," does the introduction of voice locking become a necessary technical means to solve this specific problem, rather than a conventional combination that those skilled in the art could foresee in conventional methods.

[0033] Compared with existing technologies and conventional techniques in the field, the present invention has the following unexpected technical effects: (1) Significant reduction in operation failure rate: According to internal testing, the average operation failure rate of 30 children aged 3 to 5 using the aerial splicing scheme of this application was 12%, while the average failure rate of the existing touch splicing scheme was 58%, a reduction of 79%. This effect is not simply achieved by changing the input method from touch to aerial, but is the result of the synergistic effect of aerial non-contact input, real-time detection and correction of irregular interference actions, and age-linked threshold adaptation. If only aerial input is used without interference detection and age adaptation, the rapid shaking caused by the immature hand development of young children will still lead to stroke drift, and the failure rate can only be reduced from 58% to about 35%, which cannot reach the level of 12%.

[0034] (2) Significantly extended training focus duration: Because this application completely eliminates the problem of fingers obstructing vision, and helps children gradually establish stable air gesture control habits through interference detection and correction mechanisms, the average single training focus duration for children aged 3 to 5 years has increased from 2 minutes and 15 seconds in existing touch control solutions to 5 minutes and 40 seconds, an increase of 152%. This effect exceeds the reasonable expectations of those skilled in the art for "changing the input method to improve focus," because existing technologies generally believe that children's short attention span is a psychological characteristic rather than a technical problem, and cannot be fundamentally solved by changing the interaction method.

[0035] (3) A qualitative leap in the efficiency of stroke backtracking: Through cross-modal fusion of voice stroke order number locking and mid-air fine-tuning, the average backtracking adjustment time of strokes has been reduced from 18.6 seconds in the touch solution to 4.2 seconds, and the false touch rate has dropped from 32% to 0%. This effect stems from the natural adaptation of non-contact interaction to dense stroke stacking scenarios, which is a structural breakthrough that touch solutions cannot achieve no matter how they optimize the hot area size. Attached Figure Description

[0036] Figure 1 is a system architecture block diagram (including the hardware fixing bracket) of the present invention.

[0037] Figure 2 is a flowchart of the method of the present invention.

[0038] Figure 3 is a schematic diagram of the interface layout of the present invention.

[0039] Figure 4 is a flowchart of age-linked interference detection according to the present invention.

[0040] Figure 5 is a flowchart of voice locking and retrospective adjustment. Detailed Description of the Embodiments

[0041] The following further describes the present invention in detail through specific embodiments: Embodiment 1: A stroke-sequential air-writing character assembling system based on a smartphone and a fixing bracket As shown in the accompanying Figure 1 figures, the system of this embodiment runs on a smartphone equipped with a front-facing camera and a microphone. The smartphone is placed on a horizontal desktop via a fixing bracket, and the bracket keeps the vertical distance between the front-facing camera and the user's hand area constant at 30 cm. The system includes an image acquisition module, a gesture recognition module, a stroke supply module, an air manipulation module, a structure evaluation module, an interference detection module, a voice interaction module, a stroke management module, and an age adaptation module.

[0042] As shown in the accompanying Figure 3 figures, the display interface of the smartphone is divided into three main areas: the left side is a "standard character display area", which is used to display the standard regular script glyph of the target Chinese character to be learned (for example, the character "ming"); the center is a "character assembling working area", which is a blank canvas with auxiliary grid lines of the Chinese character field, for learners to place and combine strokes; the right side is a "stroke supply area", based on the standard stroke order of the character "ming" (1 vertical, 2 horizontal fold, 3 horizontal, 4 horizontal, 5撇, 6 horizontal fold hook, 7 horizontal, 8 horizontal), only the stroke block "vertical" numbered "1st stroke" is displayed in the initial state, and the rest of the strokes are in a hidden state.

[0043] As shown in the accompanying Figure 2 figures, the method flow is as follows: the system first obtains the standard stroke data and standard stroke sequence of the character "ming", and completes the division of the three areas on the display interface. Then, the stroke supply area sequentially displays only one stroke image to be moved currently stroke by stroke based on the standard stroke sequence. The front-facing camera collects the video stream of the user's hand area in real time at a frequency of 30 frames per second, and the gesture recognition module recognizes the three-dimensional spatial coordinates of the tip of the index finger frame by frame based on the MediaPipe hand key point detection model, and maps them to the two-dimensional coordinates of the screen.

[0044] When a learner points the index finger at the "1st stroke: vertical" stroke block at the top of the stroke supply area, the system recognizes it as a pointing gesture, the border of this stroke block is highlighted and locked, and moves in the spelling workspace following the mid-air motion trajectory of the index finger. The learner drags the "vertical" stroke to the corresponding position on the left side of the "sun" radical and makes a release gesture (a rapid downward tapping motion of the index finger), then the stroke block is fixed at the current screen position. At this moment, the stroke supply area automatically refreshes to display "2nd stroke: horizontal fold", and the learner sequentially drags the 3rd to 8th strokes into the spelling workspace in mid-air in the same manner. Throughout the entire process, the interference detection module analyzes the spatiotemporal features of the finger motion trajectory in real time. If it is determined to be an irregular interference action such as rapid shaking, chaotic direction or trajectory jitter, the stroke following is paused and a correction prompt is output; if it is determined to be a normal gesture, the response continues.

[0045] After all strokes are placed, the structure evaluation module compares the combined Chinese character image in the spelling workspace with the standard "ming" character image in the standard character display area for structural similarity, and outputs training results from four dimensions: stroke completeness, relative position accuracy, overall structure proportion, and centroid distribution.

[0046] Embodiment 2: Age-linked irregular interference action detection and correction As attached Figure 4 shown, this embodiment focuses on explaining the age-linked interference detection mechanism, with test subjects being 20 users in each of three age groups: 3 to 5 years old, 6 to 8 years old, and 9 years old and above.

[0047] When the system starts, a parent inputs the child's age through the age adaptation module. The age adaptation module queries the corresponding interference detection parameter set from the preset age-interference threshold mapping table according to the user's age information: The first parameter set corresponds to children aged 3 to 5: Vmax1 is 350 pixels per second, θ1 is 100 degrees, Rmin1 is 40 pixels, the recognition frame rate is 30 frames per second, and the finger lock response delay is 80 milliseconds; The second parameter set corresponds to children aged 6 to 8: Vmax2 is 450 pixels per second, θ2 is 120 degrees, Rmin2 is 50 pixels, the recognition frame rate is 25 frames per second, and the delay is 120 milliseconds; The third parameter set corresponds to users aged 9 and above: Vmax3 is 550 pixels per second, θ3 is 140 degrees, Rmin3 is 70 pixels, the recognition frame rate is 20 frames per second, and the delay is 180 milliseconds.

[0048] Taking the 5th stroke "pie" (left-falling stroke) of the Chinese character "ming" being moved in mid-air by a 3-5 year old child as an example, the interference detection module performs the following judgments in real time: Fast shaking interference: A young child suddenly swings their wrist due to excitement, and the instantaneous displacement speed of the fingertip between two adjacent frames reaches 420 pixels per second, which exceeds Vmax1 (350 pixels per second), and the duration exceeds 0.4 seconds. The interference detection module determines this as fast shaking interference, immediately pauses the following response of the "pie" stroke image, displays a turtle deceleration animation in the center of the screen and broadcasts the voice prompt "Slow down please", and at the same time reduces the following sensitivity of the stroke image to the first gain coefficient of 0.5. After the speed recovers to below 350 pixels per second and lasts for 0.5 seconds, the system resumes normal following.

[0049] Direction confusion interference: The change angle of the movement direction of the young child within a 1.0-second time window reaches 135 degrees, which exceeds θ1 (100 degrees). The system determines this as direction confusion, displays a red flashing guiding arrow at the target position of the "pie" stroke in the character assembling workspace, and broadcasts the voice prompt "Please go this way".

[0050] Trajectory jitter interference: The trembling of the young child's hand causes the curvature radius of the trajectory to drop to 25 pixels, which is lower than Rmin1 (40 pixels). The system determines this as trajectory jitter, outputs the prompt "Steady your little hand", and enables a 5-frame sliding average filtering algorithm to smooth the fingertip coordinates.

[0051] According to tests, when the mechanism of this embodiment is not enabled, the stroke assembly failure rate of the 3-5 year old child group is 45%; after enabling, the failure rate drops to 8%. If the loose threshold (Vmax3=550 pixels per second) uniformly used by the group aged 9 and above is adopted to test the 3-5 year old children, the failure rate is still as high as 38%, which proves that the age-linked parameter configuration shown in the attach Figure 4 ed drawings plays an irreplaceable role.

[0052] Embodiment 3: Voice locking and backtracking adjustment of placed strokes As shown in the attach Figure 5 ed drawings, this embodiment describes the cross-modal collaborative process of the voice interaction module, the stroke management module and the mid-air manipulation module.

[0053] After the learner completes the mid-air placement of all 8 strokes of the Chinese character "ming" in the stroke order, it is found that the position of the second stroke "hengzhe" is too left, which makes the left "ri" part not square enough and requires a slight adjustment to the right. The learner says "2" into the mobile phone microphone. After the voice interaction module recognizes the stroke order number "2", the stroke management module searches for the placed stroke "hengzhe" with the stroke order number of 2 in the character assembling workspace, changes its border to red and starts a breathing light flashing effect (period 0.8 seconds), and sets it to an adjustable state.

[0054] The learner then moves the index finger in mid-air, and the "hengzhe" stroke block translates to the right following the finger. The system calculates in real time the pixel distance d between the current center coordinate of the stroke image and the corresponding standard center coordinate of the stroke in the standard Chinese character image: If d≤D1 (10 pixels), then fix the stroke image directly and output "Awesome" positive feedback; If D1 < d ≤ D2 (25 pixels), then output a "just a little closer" fine-tuning prompt to guide the user to continue adjusting; If d > D2 (25 pixels), the placement is considered a failure, and the stroke image is restored to the top of the stroke supply area.

[0055] In this example, after the learner makes a fine adjustment, d=8 pixels, satisfying the condition d≤D1. The learner says "fix," the stroke management module unlocks, and the "horizontal bend" is fixed in the new position, and the highlight mark disappears. (See attached image) Figure 5 As shown, during the entire voice locking and retrospective adjustment process, the right-side stroke supply area remains unaffected, and other placed strokes remain stationary, completely solving the structural problem of dense strokes occluding each other and accidental touches in touch solutions.

[0056] The concept of this invention stems from a deep reflection on a fundamental contradiction that existing electronic calligraphy training systems have long failed to address: why, despite the high maturity of touchscreen technology, is the success rate of users, especially young children, using electronic calligraphy training software still extremely low? Through a six-month follow-up observation of touch calligraphy training in children aged 3 to 6, the research team discovered that the primary reason for children's failure when writing or dragging strokes on a touchscreen was not "not recognizing the strokes" or "not understanding the structure," but rather "unresponsive hands." When children's fingers move on the screen, their fingertips inevitably obscure the target area, preventing them from simultaneously observing the stroke position and the target position. When adjusting already placed strokes, the densely packed stroke blocks make it difficult for children to accurately select them, leading to frequent mis-touches and accumulated frustration. More importantly, children's hand bones and muscles are not yet fully developed, and their fingers unconsciously produce rapid shaking, directional swaying, and slight tremors during movement. Existing systems passively accept these irregular interference movements, either misinterpreting the tremor trajectory as valid input, resulting in incorrect stroke positions, or requiring children to repeat the operation until success, ultimately causing children to lose interest in learning after multiple failures. The above observations made the R&D team realize that there is a long-overlooked technical bias in the existing electronic calligraphy training system: that is, it assumes that users have stable fine touch control ability, equates "calligraphy training" with "writing training", and tries to adapt to young children by optimizing the touch experience, but fails to realize that the action of "touch writing" itself is an insurmountable obstacle for young children.

[0057] Based on the above findings, the research team redefined the technical problem as: how to enable young children to intuitively and accurately complete the spatial structure layout training of Chinese character strokes without relying on their fine touch capabilities? The very question itself breaks the established paradigm in calligraphy training that "must simulate writing actions." The research team first ruled out conventional improvement paths within the framework of optimizing touch sampling rate and increasing the selection heat area, and instead explored non-contact interaction methods. After evaluating various non-contact technologies such as front-facing camera gesture recognition, depth camera spatial positioning, and radar sensors, they ultimately selected a two-dimensional air gesture recognition solution based on a front-facing camera. This was chosen because it requires no additional hardware, can be implemented using existing smart terminal devices, and its 20cm to 40cm sampling distance closely matches the range of hand movements of young children in a natural sitting posture.

[0058] However, simply changing the input method from touch to air gestures did not fundamentally solve the problem. In initial prototype testing, the research team discovered that young children also exhibited rapid shaking and trajectory jitter during air gestures. Furthermore, due to the lack of physical constraints from the screen surface, the stability of their gestures was actually worse than with touch. This discovery prompted the research team to expand the application goal of their computer vision algorithm from simply "recognizing valid gestures" to "detecting invalid actions and intervening in real time." This involves analyzing the instantaneous speed, angle of change of direction, and radius of curvature of finger movement trajectories to establish quantitative criteria for irregular interference actions, and pausing the system response and outputting corrective prompts the instant interference occurs. To ensure this mechanism truly suits young children rather than general gesture recognition, the research team collaborated with child development psychology experts to conduct an air gesture motion feature collection experiment on 120 children aged 3 to 9. They obtained data on the distribution of finger movement speed, frequency of direction changes, and tremor amplitude for children of different age groups. Based on this, they calibrated interference detection thresholds and gesture recognition performance parameters for three age groups: 3-5 years old, 6-8 years old, and over 9 years old, creating an age-interference threshold mapping table.

[0059] After solving the problem of precise positioning in air-based character creation, the R&D team faced a new structural challenge: how could a child accurately select a specific stroke for fine-tuning when there were many strokes already placed in the spelling workspace? Touchscreen solutions addressed this by direct tapping, but in air-based operations, fingers couldn't "click" specific pixels on the screen. The team initially tried hovering selection (selection occurs when the finger hovers above the target stroke for a certain duration), but testing revealed that children struggled to keep their fingers stationary, and slight displacement during hovering caused the selected target to drift. After multiple iterations, the team finally introduced voice commands as a locking mechanism: children simply speak the stroke order number, and the system highlights and locks the corresponding stroke in the spelling workspace, then adjusts its position using air gestures. This cross-modal fusion solution combines the "precise identification" advantage of voice interaction with the "continuous spatial control" advantage of air gestures, completely solving the non-contact backtracking problem in dense stroke scenarios.

[0060] Ultimately, the R&D team integrated the aforementioned technical elements—air gesture recognition, irregular interference action detection and correction, voice locking and air retracing, and age-linked adaptive parameter configuration—into a complete training process through specific logical coupling, forming the technical solution of this invention. Internal testing verified that this solution achieved technical results unmatched by existing touch control solutions in terms of operation failure rate, training focus duration, and retracing adjustment efficiency among young children, proving the correctness and creativity of its inventive concept.

[0061] The above descriptions are merely embodiments of the present invention, and common knowledge such as specific technical solutions and / or characteristics are not described in detail here. It should be noted that those skilled in the art can make various modifications and improvements without departing from the technical solutions of the present invention, and these should also be considered within the scope of protection of the present invention. These modifications and improvements will not affect the effectiveness of the implementation of the present invention or the practicality of the patent. The scope of protection claimed in this application should be determined by the content of its claims, and the specific embodiments described in the specification can be used to interpret the content of the claims.

Claims

1. A method for training children on the structure of Chinese characters based on smart terminal devices, characterized in that, Includes the following steps: S1, obtain the standard stroke data and standard stroke order of the target Chinese character; S2 divides the display interface of smart terminal devices into a standard character display area, a character spelling work area, and a stroke supply area. S3, according to the standard pen sequence, display only the currently moved pen image in the stroke supply area stroke by stroke; S4. The video stream of the user's hand area is collected in real time by the front image acquisition device of the smart terminal device. The air movement trajectory and gesture category of the user's fingers are identified by computer vision algorithm. The gesture category includes at least pointing gesture, grasping gesture, dragging gesture and releasing gesture. S5. Based on the recognized pointing or grasping gesture, lock the currently moving stroke image and move it in the spelling work area following the air movement trajectory of the finger. S6, based on the release gesture detected in the response, fix the stroke image at the current screen position, and automatically refresh the stroke image corresponding to the next stroke order to the stroke supply area. S7. Repeat S5 and S6 until all strokes are moved in the standard stroke order to form a combined Chinese character image. S8, perform structural similarity comparison between the combined Chinese character image and the standard Chinese character image in the standard character display area, and output the training result; In the process of recognizing the trajectory of a user's finger in mid-air, the spatiotemporal characteristics of the finger's movement trajectory are analyzed in real time. When it is determined that the user's action is an irregular interference action, the follow-up response of the current stroke image is paused, and corresponding action correction prompts are generated.

2. The method according to claim 1, characterized in that, The real-time analysis of the spatiotemporal characteristics of finger movement trajectories to determine whether the user's action belongs to irregular interference actions includes: Condition 1: Calculate the instantaneous displacement velocity of the finger between adjacent video frames. When the instantaneous displacement velocity exceeds the preset velocity threshold Vmax, it is determined to be a rapid shaking interference. The value range of the velocity threshold Vmax is 300 to 600 pixels / second. Condition 2: Calculate the change angle of the finger's movement direction within a preset time window T. When the change angle exceeds a preset angle threshold θ, it is determined to be a direction disorder interference. The value range of the angle threshold θ is 90 to 150 degrees. Condition 3: Calculate the radius of curvature of the finger movement trajectory. When the radius of curvature is less than the preset radius threshold Rmin, it is determined to be trajectory jitter interference. The radius threshold Rmin ranges from 30 to 80 pixels. When any of the above conditions are met and the duration exceeds the preset duration threshold t, the user action is determined to be an irregular interference action. The duration threshold t ranges from 0.3 to 0.8 seconds.

3. The method according to claim 2, characterized in that, The generation of corresponding action correction prompts includes: When a rapid shaking interference is detected, a deceleration prompt is output, and the stroke image following sensitivity is reduced to the first gain coefficient, which ranges from 0.4 to 0.

7. When the direction is determined to be chaotic and interfered with, a directional guidance prompt is output, and a dynamic directional arrow is displayed at the target position in the spelling work area. When the trajectory jitter interference is detected, a stable gesture prompt is output, and at the same time, a moving average filtering algorithm is used to smooth the finger movement trajectory. The window frame number of the moving average filtering is 3 to 5 frames.

4. The method according to claim 1, characterized in that, The method further includes: S41 receives voice commands from the user that include stroke order numbers; S42, based on the stroke order number, find the corresponding placed stroke image in the spelling work area, mark the stroke image with a highlighted border and a blinking breathing light, and set it to an adjustable state; S43, in response to the trajectory of the user's finger movement in the air, adjust the position of the locked stroke image in the spelling work area; S44, in response to a release gesture or a confirmed voice command, unlocks the stroke image and fixes it in a new position.

5. The method according to claim 1, characterized in that, The smart terminal device is placed on a horizontal support surface by a fixed bracket, and the front image acquisition device maintains a constant acquisition distance of 20cm to 40cm between itself and the user's hand area; The computer vision algorithm processes the video stream frame by frame at a frequency of no less than 30 frames per second, and the mapping accuracy of the finger's aerial motion trajectory in the screen's two-dimensional coordinate system reaches the pixel level.

6. The method according to claim 1, characterized in that, Before fixing the stroke image at the current screen position, calculate the pixel distance d between the center coordinates of the current stroke image and the standard center coordinates of the corresponding stroke in the standard Chinese character image; If d≤D1, then the stroke image is fixed and positive feedback is output, where D1 is 5 to 15 pixels; If D1 < d ≤ D2, then output a position fine-tuning prompt and guide the user to continue adjusting, where D2 is 16 to 30 pixels; If d > D2, the placement is deemed a failure, and the stroke image is restored to the top of the stroke supply area.

7. The method according to claim 2, characterized in that, The method further includes: Obtain the user's age information, query the corresponding interference detection parameters from the preset age-interference threshold mapping table based on the age information, and dynamically adjust the preset speed threshold Vmax, preset angle threshold θ, preset radius threshold Rmin, and the recognition frame rate and response latency of the gesture recognition module. The age-interference threshold mapping table includes: For children aged 3-5, the first parameter group is: Vmax1 is 300 to 400 pixels / second, θ1 is 90 to 120 degrees, Rmin1 is 30 to 50 pixels, the recognition frame rate is not less than 30 frames / second and the finger lock response delay is not more than 100 milliseconds. For children aged 6-8, the second parameter group is: Vmax2 is 400 to 500 pixels / second, θ2 is 100 to 140 degrees, Rmin2 is 40 to 60 pixels, the recognition frame rate is not less than 25 frames / second and the finger lock response delay is not more than 150 milliseconds. For users aged 9 and above, the third parameter group is: Vmax3 is 500 to 600 pixels / second, θ3 is 120 to 150 degrees, Rmin3 is 50 to 80 pixels, the recognition frame rate is not less than 20 frames / second and the finger lock response delay is not more than 200 milliseconds.

8. A device for training children in Chinese character structure based on smart terminal equipment, characterized in that, The method applied to claim 1 includes: The image acquisition module captures video streams of the user's hand area in real time through the front-facing image acquisition device of the smart terminal device; The gesture recognition module identifies the air movement trajectory and gesture category of the user's fingers from the video stream based on computer vision algorithms. The gesture category includes at least pointing gestures, grasping gestures, dragging gestures, and releasing gestures. The stroke supply module is used to obtain the standard stroke data and standard stroke sequence of the target Chinese character, and at the same time, it displays the image of the stroke to be moved one by one in the stroke supply area of ​​the display interface based on the standard stroke sequence. The air control module, in response to the recognized pointing or grasping gesture, locks the currently moving stroke image and moves it in the spelling work area following the air movement trajectory of the finger; in response to the recognized release gesture, it fixes the stroke image at the current screen position and automatically refreshes the stroke image corresponding to the next stroke order to the stroke supply area. The structure evaluation module is used to compare the structural similarity of the combined Chinese character images formed in the spelling work area with the standard Chinese character images and output the training results. The interference detection module is used to analyze the spatiotemporal characteristics of the finger movement trajectory in real time during the process of recognizing the user's finger movement trajectory in the air. When it is determined that the user's action is an irregular interference action, the current stroke image follow-up response is paused and corresponding action correction prompt information is generated. The smart terminal device is placed on a horizontal support surface by a fixed bracket, which is used to maintain a constant acquisition distance of 20cm to 40cm between the front image acquisition device and the user's hand area.

9. The apparatus according to claim 8, characterized in that, Also includes: The voice interaction module is used to receive voice commands from users that include stroke order numbers; The stroke management module is used to find the corresponding placed stroke image in the spelling work area according to the stroke order number, mark the stroke image with a highlighted border and a flashing breathing light, and set it to an adjustable state. The air control module also adjusts the position of the locked stroke image in response to the air movement trajectory of the user's finger, and unlocks it in response to a release gesture or confirmation voice command.

10. The apparatus according to claim 8, characterized in that, Also includes: An age adaptation module is used to obtain the user's age information, query the corresponding interference detection parameters from a preset age-interference threshold mapping table based on the age information, and dynamically adjust the recognition frame rate and response latency of the gesture recognition module, as well as the preset speed threshold Vmax, preset angle threshold θ, and preset radius threshold Rmin of the interference detection module. The age interference threshold mapping table includes: For children aged 3-5, the first parameter group is: Vmax1 is 300 to 400 pixels / second, θ1 is 90 to 120 degrees, Rmin1 is 30 to 50 pixels, the recognition frame rate is not less than 30 frames / second and the finger lock response delay is not more than 100 milliseconds. For children aged 6-8, the second parameter group is: Vmax2 is 400 to 500 pixels / second, θ2 is 100 to 140 degrees, Rmin2 is 40 to 60 pixels, the recognition frame rate is not less than 25 frames / second and the finger lock response delay is not more than 150 milliseconds. For users aged 9 and above, the third parameter group is: Vmax3 is 500 to 600 pixels / second, θ3 is 120 to 150 degrees, Rmin3 is 50 to 80 pixels, the recognition frame rate is not less than 20 frames / second and the finger lock response delay is not more than 200 milliseconds.

Images