Typewriting training system based on image recognition

Through the typing training system based on image recognition, the user's typing posture is monitored and analyzed in real time, and targeted feedback and training are provided, which solves the problem of insufficient monitoring of typing posture in the existing technology, improves typing efficiency and accuracy, and reduces the Physical strain and enhance user engagement and fun.

CN119028205APending Publication Date: 2024-11-26XIAN QIAN INFORMATION & TECH
View PDF 17 Cites 0 Cited by

Patent Information

Application Number
CN202411077935.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-07
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

The existing typing training tools and software lack real-time monitoring and guidance of users' typing postures, making it difficult for users to realize and correct wrong typing habits, affecting typing efficiency and accuracy, and may lead to strain on wrists, arms and neck.

Method used

Design a typing training system based on image recognition to capture the user's finger image of the keyboard through the camera, use image recognition technology to analyze finger position, posture and keyboard type, and provide real-time feedback and training, including level task interface, virtual reality and Augmented reality mode, video recording and sharing capabilities, timely rest reminders, multi-user management and cross-operating system compatibility.

Benefits of technology

Real-time monitoring and analysis of user typing postures is realized, typing skills is improved, correct typing habits are cultivated, physical strain caused by improper typing is reduced, and user participation is improved through diversified training modes and interactivity. and fun.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119028205A_ABST
    Figure CN119028205A_ABST
Patent Text Reader

Abstract

The invention belongs to the technical field of interaction of computer input equipment, and particularly relates to a typewriting training system based on image recognition, which captures video images of keyboard tapping by fingers of a user through a camera and monitors typewriting postures and finger actions of the user in real time by adopting an advanced image processing algorithm. Through precise software logic, the typewriting training system can effectively filter and count key pressing behaviors of a user, and it is ensured that only correct finger knocking is recognized as effective input. The system provides real-time feedback and customized training tasks, and aims to significantly improve the typing accuracy and speed of a user while cultivating and maintaining a correct typing posture, thereby reducing typing errors and potential physical strain. The invention aims to promote the comprehensive improvement of the typewriting skills of the user through a scientific and efficient training method.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the technical field of computer input device interaction, and in particular relates to a typing training system based on image recognition. Background Art

[0002] With the increasing popularity of information technology, typing has become an indispensable skill in daily life and work. Whether writing documents, sending emails or participating in online communication, fast and accurate typing is the key to improving work efficiency. However, many people tend to ignore the correct typing posture and technique during typing, which not only affects typing efficiency and accuracy, but may also cause strain and pain in the wrists, arms and neck in the long run.

[0003] Although existing typing training tools and software can provide a certain degree of typing practice, most of them lack real-time monitoring and guidance of users' typing postures, making it difficult for users to realize and correct incorrect typing habits during the training process.

[0004] In view of this, the inventor hopes to design a typing training system that can monitor the user's typing posture in real time and provide feedback accordingly. Summary of the invention

[0005] The purpose of the present invention is to overcome the above problems existing in the prior art and provide a typing training system based on image recognition, which can monitor the user's typing posture in real time and provide feedback accordingly, capture the user's typing movements through image recognition technology, and analyze whether these movements meet the correct typing specifications. In addition, the system also has the ability to filter and count the user's typing behavior, provide users with targeted training and instant feedback, and help users form correct and efficient typing habits.

[0006] In order to achieve the above technical objectives and the above technical effects, the present invention is implemented through the following technical solutions:

[0007] The present invention provides a typing training system based on image recognition, the typing training system comprising:

[0008] An image acquisition module, the image acquisition module is used to capture an image of a user's fingers tapping the keyboard through a camera;

[0009] An image recognition module, which is used to recognize the position, posture, keyboard type, and overall typing posture of the user in the finger image to ensure that the user is in the correct working position;

[0010] A keyboard event filtering logic module, which is used to filter out key events that do not match the correct posture and keyboard type according to the output of the image recognition module, to ensure that the system only responds to valid taps under the correct posture;

[0011] The error statistics and real-time display module is used to count and display the number of valid knocks and erroneous knocks in real time.

[0012] Furthermore, in the above-mentioned typing training system based on image recognition, the typing training system is provided with a level-task typing content interface, which can provide a series of typing tasks with increasing difficulty; the level-task typing content interface can provide virtual reality and augmented reality modes, thereby enhancing user participation and fun through immersive experience.

[0013] Furthermore, in the above-mentioned image recognition-based typing training system, the typing training system has a user real-time status video recording function, which is used to capture and display key video information of the user during the typing process. The key video information includes sitting posture, facial expressions, and fingers flying on the keyboard, so that the user can self-monitor, self-motivate and improve.

[0014] Furthermore, in the above-mentioned typing training system based on image recognition, the typing training system has a template function that can provide automatic generation of videos and texts, allowing users to easily record the typing process and automatically generate a video summary containing personal performance analysis and wonderful moments for easy sharing and review.

[0015] Furthermore, in the above-mentioned typing training system based on image recognition, the typing training system has a sharing function, which enables users to quickly share video summaries and results of the typing process to social media platforms or other communication channels, thereby enhancing social interactivity and incentive mechanisms.

[0016] Furthermore, in the above-mentioned typing training system based on image recognition, the typing training system has a timely rest reminder function, which reminds the user to rest in time according to the user's typing time; the timely rest reminder function includes biorhythm monitoring, combined with the user's typing frequency and duration, to more accurately predict the user's fatigue points and realize personalized rest reminders.

[0017] Furthermore, in the above-mentioned typing training system based on image recognition, the typing training system has a multi-user account management function, allowing family members, colleagues or friends to share resources, compare progress, and promote a collective learning atmosphere.

[0018] Furthermore, in the above-mentioned typing training system based on image recognition, the camera adopts a high-definition camera, and the camera is installed near the display or in an integrated desk lamp to improve the visual environment of the user during typing practice; the integrated desk lamp has a healthy spectrum selection function, and the user can adjust the color temperature and intensity of the light source according to personal preferences and biological clock settings to protect eyesight.

[0019] Furthermore, in the above-mentioned typing training system based on image recognition, the image recognition module adopts an advanced deep learning algorithm, which not only optimizes the recognition of finger posture, keyboard type and user's overall typing posture, but also introduces emotion analysis, which can identify the user's emotional changes during typing and provide emotional support and motivation.

[0020] Furthermore, in the above-mentioned image recognition-based typing training system, the typing training system has cross-operating system compatibility, ensuring that the system can run on multiple mainstream operating systems, including but not limited to Windows, macOS, and Linux.

[0021] The beneficial effects of the present invention are:

[0022] The present invention aims to improve the user's typing skills, cultivate correct typing habits, and reduce physical strain caused by improper typing by real-time monitoring and analysis of the user's typing posture and finger movements combined with software logic processing.

[0023] Of course, any product implementing the present invention does not necessarily need to achieve all of the above advantages at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for describing the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0025] Figure 1 It is a block diagram of the composition of the present invention;

[0026] 1- Image acquisition module, 2- Image recognition module, 3- Keyboard event filtering logic module, 4- Error statistics and real-time display module. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] Example

[0029] This embodiment provides a typing training system based on image recognition, including an image acquisition module 1, an image recognition module 2, a keyboard event filtering logic module 3 and an error statistics and real-time display module 4. The image acquisition module 1 is used to capture the image of the user's fingers tapping the keyboard through a camera; the image recognition module 2 is used to identify the position, posture, keyboard type of the fingers in the finger image, and the user's overall typing posture; the keyboard event filtering logic module 3 ensures that the system only responds to valid tapping under the correct posture; the error statistics and real-time display module 4 is used to count and display the number of valid tapping and erroneous tapping in real time. The system aims to improve the user's typing skills, cultivate correct typing habits, and reduce physical strain caused by improper typing by real-time monitoring and analysis of the user's typing posture and finger movements, combined with software logic processing. The following is the details of this embodiment:

[0030] Image recognition module:

[0031] The core of this embodiment is an image recognition module, which includes one or more high-definition cameras to capture images of the user's fingers while typing. The camera can be fixed above the computer monitor, integrated into an integrated desk lamp, or other suitable locations to ensure that the user's finger movements can be clearly captured. The image recognition module uses advanced deep learning algorithms to not only optimize the recognition of finger posture, keyboard type, and the user's overall typing posture, but also introduces emotional analysis to identify the user's emotional changes during typing and provide emotional support and motivation.

[0032] When it comes to the application of deep learning algorithms in image recognition modules, especially for the recognition of finger gestures, keyboard types, user's overall typing posture, and more advanced sentiment analysis, the inventors consider using convolutional neural networks (CNNs) and recurrent neural networks (RNNs), combined with attention mechanisms and sentiment analysis models. The following is a detailed explanation of this part:

[0033] Convolutional Neural Networks (CNNs)

[0034] CNNs are mainly used to process inputs with grid structures, such as images. They perform well in image recognition and classification tasks. In this example, CNNs can be used to recognize the gestures and positions of fingers, as well as the type of keyboard.

[0035] Specific functions and formulas:

[0036] Convolutional layer: The convolution operation is defined as the process of multiplying a convolution kernel with a part of the input image and then summing them up, which can be expressed as:

[0037]

[0038] Where f is the input image, g is the convolution kernel, and τ and t represent pixel locations.

[0039] Activation function: ReLU (Rectified Linear Unit) is one of the most commonly used nonlinear activation functions, and its mathematical expression is:

[0040] f(x)=max(0,x)

[0041] Pooling layer: Pooling is used to downsample and reduce spatial dimensions. The most common form is max pooling, which takes the maximum value of a local area.

[0042] Recurrent Neural Networks (RNNs)

[0043] RNNs can process sequential data and are very suitable for time series analysis, such as video frame sequences. Here, RNNs can be used to analyze the user's dynamic behavior during typing, such as finger movement trajectory.

[0044] Specific functions and formulas:

[0045] ·Recurrent unit update rule: LSTM (Long Short-Term Memory) unit is a popular RNN structure, and its state update equation is as follows:

[0046] i i =σ(W xi x t +W hi h t-1 +b i )

[0047] f t =σ(W xf x t +W hf h t-1 +b f )

[0048] o t =σ(Wxo x t +W ho h t-1 +b o )

[0049] c t =f t ⊙c t-1 +i i ⊙tanh(W xc x t +W hc h t-1 +b c )

[0050] h t =o t ⊙tanhc t

[0051] where i i ,f t ,o i They are input gate, forget gate and output gate, c t is the cell state, h t is the hidden state, σ is the sigmoid function, and ⊙ represents element-wise multiplication.

[0052] Attention Mechanism

[0053] The attention mechanism allows the model to focus on certain key information when processing sequence data, which is very useful for recognizing the user's overall typing posture.

[0054] Specific functions and formulas:

[0055] Attention weight calculation: Attention weight can be calculated by dot-product attention, and its formula is:

[0056]

[0057] where e ij is the query vector, q i and key vector k j The dot product of .

[0058] Sentiment Analysis

[0059] Sentiment analysis models can identify users’ emotional states based on facial expressions, eye contact, and micro-expressions, providing them with emotional support and motivation.

[0060] Specific functions and formulas:

[0061] Emotion classification: You can use pre-trained deep learning models such as VGG-Face or ResNet to classify face images and identify basic emotions such as happiness, sadness, anger, surprise, etc.

[0062] Emotion intensity assessment: The output of the last layer of the neural network can be converted into a probability distribution using the Softmax function, representing the likelihood of each emotion.

[0063] Combining the above technologies, the image recognition module can accurately identify and analyze the user's behavior and emotions, providing richer and deeper data for the typing training system, thereby achieving a personalized and emotional interactive experience. It should be noted that the specific implementation and tuning of these algorithms usually require a large amount of labeled data and computing resources.

[0064] Keyboard event filtering logic module:

[0065] The system's built-in keyboard event filtering logic module is a key component of this embodiment. This logic performs real-time analysis of the user's key events based on the finger state information provided by the image recognition module. Only when the algorithm confirms that the correct finger hits the correct key, the key event is considered valid. Otherwise, the system will filter out these key events that do not conform to the correct typing technique and will not count them in the typing statistics.

[0066] The following is a simplified logical framework, including specific functions and algorithm descriptions:

[0067] Keyboard Mapping and Finger Assignment

[0068] First, the system needs to build a keyboard mapping table that associates each key with the finger that is expected to hit the key. For example, for a standard QWERTY keyboard, the left index finger is usually responsible for hitting the "F" and "R" keys.

[0069] Image recognition result analysis

[0070] The image recognition module outputs information such as finger position, posture, and keyboard type. This information needs to be converted into a form that can be matched with the keyboard mapping table.

[0071] Function definition:

[0072] getFingerPosition(fingerID): Get the current position coordinates of the specified finger ID.

[0073] getFingerPose(fingerID): Get the current pose (bending degree) of the specified finger ID.

[0074] getKeyPosition(key): Gets the position coordinates of the specified key.

[0075] getKeyboardType(): Returns the currently recognized keyboard type.

[0076] Keyboard event filtering logic

[0077] Function definition:

[0078] isValidKeyPress(keyEvent, fingerPositions, fingerPoses): Checks whether a given key event was performed by the correct finger.

[0079] Algorithm description:

[0080] 1. Read key events: Read the latest key event keyEvent from the system event queue.

[0081] 2. Get key information: Use getKeyPosition(keyEvent.key) to get the actual position of the key.

[0082] 3. Find the closest finger: traverse the positions of all fingers and use the Euclidean distance formula to find the finger closestFinger closest to the key:

[0083]

[0084] 4. Check the finger posture: Make sure the closestFinger posture is suitable for tapping keys, that is, the finger must be in a nearly straight state:

[0085] if(getFingerPose(closestFinger) <threshold)

[0086] 5. Verify finger assignment: Check if closestFinger is the finger assigned to keyEvent.key according to the keyboard mapping table:

[0087] if(closestFinger==getAssignedFingerForKey(keyEvent.key))

[0088] 6. Confirm that the key is valid: If all the above conditions are met, the key event is considered valid.

[0089] Formula application:

[0090] Euclidean distance formula: used to calculate the distance between the finger and the key, and determine which finger is closest to the key.

[0091]

[0092] Through such logic, the system can ensure that key events are accepted and recorded by the system only when the user taps the correct key with the correct finger. This helps to cultivate users' correct typing habits and improve typing efficiency and accuracy.

[0093] It should be noted that the description here provides a simplified algorithm framework. In actual applications, more details may need to be considered, such as the relative angle between the finger and the key, the hand movement speed, and other factors, to improve the accuracy and robustness of the judgment. At the same time, the selection of thresholds and parameters also needs to be optimized through experiments and debugging to adapt to different users and keyboard types.

[0094] Error statistics and real-time display module:

[0095] This embodiment also includes an error statistics and real-time display module, which is responsible for collecting and analyzing the user's typing data, including the number of correct tapping, the number of incorrect tapping, and the type of error, etc. This information will be displayed in real time on the user interface to provide users with instant feedback, helping users understand their typing performance and make targeted improvements. The following is a detailed description of this module:

[0096] Data Collection

[0097] First, the module needs to collect the output from the keyboard event filtering logic and the finger status information provided by the image recognition module. Data collection should include but is not limited to:

[0098] Correct Key Press Count

[0099] Incorrect Key Press Count

[0100] Error Type

[0101] Error type classification

[0102] Error types can be divided into the following categories:

[0103] 1. Position error: The key pressed by the user does not match the target key.

[0104] 2. Finger error: Using the wrong finger to hit the correct key.

[0105] 3. Wrong posture: Although the correct keys are struck, the finger or wrist posture is incorrect.

[0106] 4. Repeated tapping: The same key is tapped multiple times in succession.

[0107] Statistics and Analysis

[0108] Function definition:

[0109] updateKeyPressStats(keyEvent, fingerStatus): Updates typing statistics.

[0110] calculateAccuracy(): Calculates typing accuracy.

[0111] · analyzeErrorTypes(): Analyzes error types and their frequencies.

[0112] Formula application:

[0113] Typing accuracy: Typing accuracy can be calculated using the following formula:

[0114]

[0115] Error type frequency: For each error type, its occurrence frequency can be calculated by the following formula:

[0116]

[0117] Among them, ErrorFrequency type is the cumulative number of specific error types, and TotalKeyPressCount is the total number of key presses.

[0118] Real-time display

[0119] The real-time display module needs to convert the above statistics into a user-friendly format and display it on the screen. This may include:

[0120] Typing accuracy percentage

[0121] Frequency chart of various types of errors

[0122] The most common error types

[0123] User interface update function:

[0124] displayStatsOnUI(Accuracy,ErrorFrequencies,CommonErrors) : Updates the UI to show the latest typing statistics.

[0125] Through the design and implementation of the above modules, users can get instant feedback, understand their typing performance, and conduct targeted exercises based on the types of errors, thereby gradually improving their typing skills and efficiency.

[0126] Timely rest reminder function

[0127] Considering that long-term typing may put a burden on the user's body, this embodiment designs a timely rest reminder function. The system reminds the user to take a break in time according to the length and intensity of the user's typing to prevent excessive fatigue and potential strain. The following is a detailed introduction to this function:

[0128] Functional Overview

[0129] The timely break reminder function monitors the user's typing time and intensity, analyzes the user's typing pattern, and then reminds the user to take a break at appropriate time intervals to maintain healthy working habits.

[0130] Data Collection

[0131] Typing time: records the time the user continuously types.

[0132] Typing intensity: measured by the number of keys typed per unit time.

[0133] Rest strategy

[0134] Fixed interval: remind you to take a break at a certain interval, such as every 45 minutes.

[0135] Based on typing intensity: If the user types a lot of keys in a short period of time, a break reminder will be triggered in advance.

[0136] Specific functions and formulas

[0137] Function definition:

[0138] startTypingSession(): Starts a new typing session, resetting the timer and intensity counter.

[0139] recordKeyPress(): records a key press event and updates the typing time and intensity.

[0140] checkForRestAlert(): Checks whether a rest reminder needs to be issued.

[0141] notifyRest(): Notify the user to take a rest.

[0142] Formula application:

[0143] Typing strength: can be calculated by the following formula:

[0144]

[0145] Among them, KeyPressCount is the number of keys pressed per unit time, and TimeDelta is the corresponding time period (seconds).

[0146] Rest intervals: can be adjusted dynamically based on the user's typing intensity, for example:

[0147]

[0148] Among them, BaseInterval is the basic rest interval, AverageIntersity is the average typing intensity, Threshold and AdjustmentFactor are adjustment factors used to control the impact of intensity changes on the rest interval.

[0149] Through the above functions and codes, the timely break reminder function can effectively monitor the user's typing activities and remind the user to take a break at the appropriate time, helping to maintain the user's health and work efficiency.

[0150] Level task-style typing content interface

[0151] In order to increase the fun and challenge of typing training, this embodiment provides a series of typing tasks with increasing difficulty, which are presented in the form of levels. Users need to complete the tasks of the current level to unlock the next level. This design can motivate users to continue training. The following is a detailed description of this part:

[0152] Design Concept

[0153] The level-based task-based typing content interface adopts a progressive difficulty design, from basic to advanced, to guide users to gradually master and become proficient in typing skills. Each level is designed around a specific goal or theme, including typing speed and accuracy, as well as typing habits and techniques.

[0154] Level Structure

[0155] Basic Levels: The first few levels focus on typing basic letters and numbers, helping users become familiar with the keyboard layout and basic finger positioning.

[0156] Advanced Levels: As the user's skills improve, the levels gradually increase in difficulty, introducing more complex vocabulary, phrases and sentences, emphasizing speed and accuracy while adding multi-finger coordination exercises.

[0157] Advanced Levels: The final stages of the levels may include specialized terms, rapidly changing text strings, and complex typing patterns, testing the user's reaction speed and the diversity of their typing skills.

[0158] Level unlocking mechanism

[0159] The user must meet certain typing accuracy and speed standards to unlock the next level. For example, the user may need to complete the typing task of the current level without errors, or achieve an accuracy rate of more than 90% within the specified time.

[0160] Task diversity

[0161] To keep the training interesting and challenging, the tasks for each level can include:

[0162] Speed ​​Challenge: Type a text as quickly as possible within the time limit.

[0163] Accuracy test: There is no time limit and the focus is on typing accuracy.

[0164] Hybrid Challenge: Combines speed and accuracy, requiring users to type as quickly as possible while still being accurate.

[0165] ·Specific skill practice: such as practicing using the numeric keypad, symbol keys, or special character input.

[0166] Feedback and Rewards

[0167] Instant feedback: After the user completes each task, the system will immediately display the typing results, including speed, accuracy and typing habit analysis.

[0168] Points and medals: You can get points by completing levels. Accumulated points can unlock special medals or titles, which increase the user's sense of accomplishment and belonging.

[0169] Leaderboards: Set up online leaderboards to encourage friendly competition between users and make typing training more social and interactive.

[0170] Personalized training path

[0171] The system can also automatically adjust the training path according to the user's performance, customizing the most suitable typing training plan for each user. For example, if the system detects that the user has difficulty in a specific area (such as the use of the left pinky finger), it will provide more training tasks for that area.

[0172] User interface design

[0173] The interface of the level-based typing content should be designed to be intuitive and easy to use, providing clear instructions and feedback to ensure that users can easily understand and enjoy the typing training process. The interface should include:

[0174] ·Show the progress bar of the current level and task.

[0175] Highlight the portion of text the user is typing to provide visual guidance.

[0176] Clearly display the typing results and the next step of the operation.

[0177] Through the above design, the level-based task-based typing content interface can effectively stimulate users' interest in learning and help users continuously improve their typing skills while enjoying the fun.

[0178] Typing training software

[0179] The typing training software is another important component of the present embodiment. It records the user's typing data and provides personalized training suggestions based on the data. The software can also analyze the user's typing habits, point out the user's common mistakes, and provide correction suggestions. The following is detailed information about the typing training software:

[0180] Data recording and analysis

[0181] The typing training software continuously monitors and records the user's typing behavior, including but not limited to:

[0182] Typing speed (characters / minute)

[0183] Typing accuracy (the percentage of characters typed correctly)

[0184] Frequency and pattern of finger use

[0185] Type and frequency of errors (e.g., wrong position, repeated tapping, etc.)

[0186] Typing intensity (number of keystrokes per unit time)

[0187] Typing sessions (active typing time of the day)

[0188] The software uses machine learning algorithms to analyze this data and identify users' typing habits and potential problems, such as:

[0189] Preferring to use some fingers over others

[0190] Frequent mistakes on certain types of words or characters

[0191] The fluctuation pattern of typing speed over time

[0192] Personalized training suggestions

[0193] Based on the analysis results, the software generates a personalized training plan, including:

[0194] Reinforcement Practice: Provide additional practice on areas where users are struggling, such as specific letter combinations or symbols.

[0195] Finger balance training: If the user is found to be overly dependent on certain fingers, the software will design exercises to balance the use of fingers to avoid fatigue and injury.

[0196] Speed ​​and accuracy balance: If users tend to sacrifice accuracy for speed, the software will provide exercises that emphasize both speed and accuracy to help users find the best balance between the two.

[0197] Regular training reminders: Based on the user's active time, the software can set regular training reminders to encourage users to practice at the best time.

[0198] Error correction and improvement suggestions

[0199] For common typing errors, the software provides detailed correction suggestions, including:

[0200] Video tutorials: Demonstrate correct finger positions and typing postures to help users understand and imitate.

[0201] · Step-by-step guidance: Break down incorrect actions and explain the correct typing process step by step.

[0202] ·Practical exercises: Provide targeted exercises to allow users to correct their mistakes in practice.

[0203] Progress tracking and achievement incentives

[0204] The software also features a progress tracking feature so users can see how their typing skills improve over time, including:

[0205] Historical curves of typing speed and accuracy

[0206] The downward trend of error rate

[0207] Unlocked levels and completed challenges

[0208] In addition, the software has an achievement system. When users reach certain milestones, such as typing continuously without errors for a certain length, or completing all levels, they can get virtual medals or titles, which increases the fun and motivation of training.

[0209] User Interface and Interaction

[0210] The user interface of typing training software should be intuitive and easy to use, providing clear instructions and feedback to ensure that users can easily understand the training content and progress. The interface should include:

[0211] Real-time typing performance dashboard showing current typing speed and accuracy

[0212] Personalized training plan overview, highlighting areas for improvement

[0213] Error analysis report, detailing common errors and how to correct them

[0214] Achievement wall and leaderboard to show personal achievements and compare with other users

[0215] Through the above functions, typing training software not only helps users improve their typing skills, but also stimulates users' learning interest and persistence through personalized training plans and positive feedback mechanisms.

[0216] Camera Configuration:

[0217] In order to improve user acceptance and user experience, the camera configuration of this embodiment takes into account the privacy and convenience of users. The camera can be combined with daily-used devices such as desk lamps to reduce user privacy concerns and provide good lighting conditions. The following is a detailed description of this module:

[0218] Integration and privacy protection

[0219] 1. Concealed design:

[0220] oCameras can be designed to be very small and even embedded in everyday objects such as the base of a lamp, the edge of a bookshelf, or the frame of a computer monitor.

[0221] oWith a physical shielding design, the lens can be automatically or manually covered when not in use, enhancing the user's sense of control over privacy.

[0222] 2. Privacy mode and permission management:

[0223] oPrivacy mode is available at the software level, allowing users to completely disable the camera or limit its access permissions.

[0224] oApps require explicit user authorization to access the camera, and user consent is required before each use.

[0225] 3. Encrypted transmission:

[0226] oEnsure the security of camera data during transmission using end-to-end encryption to prevent unauthorized access.

[0227] oThe data encryption algorithm can be AES-256 (Advanced Encryption Standard), which is a widely recognized secure encryption standard.

[0228] Smart lighting control

[0229] 1. Ambient light sensing:

[0230] oThe device where the camera is located (such as a desk lamp) is equipped with an ambient light sensor to detect the surrounding light intensity.

[0231] oFormula: I = A*log(L+B), where I is the lighting output intensity, L is the ambient light intensity, and A and B are adjustment coefficients to ensure appropriate brightness under different lighting conditions.

[0232] 2. Facial recognition and directional lighting:

[0233] o Using facial recognition technology, the camera can determine the user's location and orientation.

[0234] o When it recognizes a user, the smart desk lamp can adjust the lighting angle and intensity to provide the most comfortable light for reading or working.

[0235] o The algorithm may be based on deep learning convolutional neural network (CNN) for face detection and localization.

[0236] 3. Color temperature adaptation:

[0237] oCameras can help adjust the color temperature of lights to suit different activities or times.

[0238] o For example, providing a higher color temperature (cool white light) during the day to promote alertness, and a lower color temperature (warm yellow light) at night to reduce blue light that affects sleep.

[0239] Data processing and analysis

[0240] 1. Data anonymization:

[0241] oThe collected data is anonymized before storage and analysis to remove any personally identifiable information.

[0242] o Differential privacy algorithms can be used to add random noise to the dataset to protect the privacy of individual data.

[0243] 2. Use pattern learning:

[0244] o Predict user usage patterns through machine learning models, such as when brighter light is needed or when the camera should automatically shut down.

[0245] oUse supervised learning or reinforcement learning algorithms to train based on user behavior data to optimize user experience.

[0246] Together, these technologies and strategies ensure that camera configurations are both practical and respectful of user privacy, while also enhancing the user experience through smart lighting features.

[0247] Cross-OS Compatibility:

[0248] The design of this embodiment takes into account cross-operating system compatibility, ensuring that the system can run on different mainstream operating systems, such as Windows, macOS, and Linux, to meet the needs of a wider user group.

[0249] Through the above technical means, this embodiment can effectively help users improve typing skills, cultivate correct typing habits, and reduce the risk of physical strain during typing training. The following are some key technical points and implementation strategies:

[0250] Technology stack selection

[0251] Programming language: Choose a programming language with strong cross-platform capabilities, such as Java, C#, or Python, which have good cross-platform support and rich libraries.

[0252] Graphical interface framework: Use a cross-platform GUI framework such as Qt, Electron, or JavaFX, which can generate consistent user interfaces on different operating systems.

[0253] Platform Abstraction Layer (PAL)

[0254] Build a platform abstraction layer to encapsulate operating system-specific functions, such as file system access, window management, and system event monitoring. In this way, the core business logic can remain unchanged, while the PAL layer can be adapted for different operating systems.

[0255] Operating system detection

[0256] At boot time, the system should be able to automatically detect the platform it is running on and load the appropriate PAL layer implementation. This can be done by checking environment variables, operating system API calls, or system properties.

[0257] File path processing

[0258] Because different operating systems represent file paths differently (such as Windows uses backslashes \, while Linux and macOS use forward slashes / ), the system should be able to intelligently process and convert paths to ensure that file read and write operations are performed correctly on all platforms.

[0259] Peripheral driver compatibility

[0260] Camera and keyboard event capture: Use cross-platform multimedia frameworks and input device libraries, such as OpenCV and SDL (Simple DirectMedia Layer), which provide a unified interface to access hardware devices on different operating systems.

[0261] Event Listeners: Create abstract event listener interfaces and capture keyboard and mouse events with platform-specific implementations, ensuring accurate capture of user input on all platforms.

[0262] Network Communication

[0263] If the system needs to communicate with a server or other devices, a cross-platform network library such as Boost.Asio (C++), Socket.IO (JavaScript), or the Socket Programming Library (Java) should be used to ensure compatibility of network functions.

[0264] Testing and Verification

[0265] Automated testing: Build cross-platform automated test suites to cover functional and performance testing on different operating systems.

[0266] Emulators and virtual machines: Use emulators or virtual machines in your development environment to simulate the behavior of different operating systems for testing and debugging.

[0267] Through the above strategy, the typing training system of this embodiment can achieve true cross-operating system compatibility, providing a consistent high-quality typing training experience for users around the world, regardless of whether they are using Windows, macOS or Linux. This not only expands the potential user base, but also ensures the flexibility and maintainability of the system.

[0268] The preferred embodiments of the present invention disclosed above are only useful to help explain the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to specific implementation methods. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can understand and use the present invention well. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. A typing training system based on image recognition, characterized in that: The typing training system includes: An image acquisition module, the image acquisition module is used to capture images of the user's fingers tapping the keyboard through a camera; An image recognition module, which is used to recognize the position, posture, keyboard type, and overall typing posture of the user in the finger image to ensure that the user is in the correct working position; A keyboard event filtering logic module, which is used to filter out key events that do not match the correct posture and keyboard type according to the output of the image recognition module, to ensure that the system only responds to valid taps under the correct posture; The error statistics and real-time display module is used to count and display the number of valid knocks and erroneous knocks in real time.

2. The typing training system based on image recognition according to claim 1, characterized in that: The typing training system is provided with a level-task typing content interface, which can provide a series of typing tasks with increasing difficulty; the level-task typing content interface can provide virtual reality and augmented reality modes, and enhance user participation and fun through immersive experience.

3. The typing training system based on image recognition according to claim 1, characterized in that: The typing training system has a real-time video recording function of the user's status, which is used to capture and display key video information of the user during the typing process. The key video information includes sitting posture, facial expressions, and fingers flying on the keyboard, so that the user can monitor, motivate and improve himself.

4. The typing training system based on image recognition according to claim 1, characterized in that: The typing training system has a template function that can provide automatic generation of videos and texts, allowing users to easily record the typing process and automatically generate a video summary containing personal performance analysis and wonderful moments for easy sharing and review.

5. The typing training system based on image recognition according to claim 1, characterized in that: The typing training system has a sharing function, which enables users to quickly share video summaries and results of the typing process to social media platforms or other communication channels, thereby enhancing social interactivity and incentive mechanisms.

6. The typing training system based on image recognition according to claim 1, characterized in that: The typing training system has a timely rest reminder function, which reminds the user to rest in time according to the user's typing time; the timely rest reminder function includes biorhythm monitoring, which combines the user's typing frequency and duration to more accurately predict the user's fatigue point and realize personalized rest reminders.

7. The typing training system based on image recognition according to claim 1, characterized in that: The typing training system has a multi-user account management function, allowing family members, colleagues or friends to share resources, compare progress, and promote a collective learning atmosphere.

8. The typing training system based on image recognition according to claim 1, characterized in that: The camera uses a high-definition camera, which is installed near a monitor or in an integrated desk lamp to improve the user's visual environment during typing practice; the integrated desk lamp has a healthy spectrum selection function, and the user can adjust the color temperature and intensity of the light source according to personal preferences and biological clock settings to protect eyesight.

9. The typing training system based on image recognition according to claim 1, characterized in that: The image recognition module adopts advanced deep learning algorithms, which not only optimizes the recognition of finger posture, keyboard type and user's overall typing posture, but also introduces sentiment analysis, which can identify the user's emotional changes during typing and provide emotional support and motivation.

10. The typing training system based on image recognition according to claim 1, characterized in that: The typing training system has cross-operating system compatibility, ensuring that the system can run on a variety of mainstream operating systems.

Citation Information

Patent Citations

  • Character input method and device

    CN108519855A

  • Three-dimensional dynamic finger text input system and method based on depth image

    CN108646910A

  • AR auxiliary instrument practice method and system, and intelligent device

    CN109102784A

  • Piano practice error correction method based on image recognition processing

    CN109887375A

  • Information input system based on image recognition

    CN112183447A