A multi-channel pressure sensing-based pen-holding fatigue and sitting posture collaborative monitoring intelligent writing assistance system

Abstract

The application discloses a kind of based on multi-channel pressure perception's hold pen fatigue and sitting posture collaborative monitoring intelligent writing auxiliary system.The system includes hardware monitoring terminal and mobile terminal software application two parts.Hardware terminal is based on the "three-point support" model of motion biomechanics, and three flexible film pressure sensors are respectively deployed in the pen holding area corresponding to thumb, index finger and middle finger, to realize the independent real-time acquisition of three fingers pen holding pressure;Embedded microcontroller executes lightweight hierarchical fatigue discrimination algorithm according to pressure threshold, duration and alarm interval time three dimensions, and carries out hierarchical acoustic reminder through buzzer;Data is transmitted to mobile terminal in real time through Bluetooth.Real-time transmission of mobile terminal software receives and visualizes pressure data, while integrating computer vision model based on YOLOv8-Pose, uses mobile phone camera to detect and analyze user's sitting posture in real time, to realize the accurate discrimination and instant warning of head forward tilt angle, shoulder tilt angle and spinal lateral offset amount.The application realizes the two-dimensional health collaborative monitoring of hand pen holding pressure and whole body posture with low-cost open-source hardware architecture, fills the market gap of hundred-yuan-level writing fatigue monitoring products, and can be applied to information technology course teaching and maker education practice.

Description

Technical Field

[0001] This invention belongs to the field of intelligent health monitoring and educational information technology, specifically relating to an intelligent writing assistance system and its monitoring method based on multi-channel pressure sensing for coordinated monitoring of pen grip fatigue and sitting posture. It is particularly suitable for children and adolescents' writing health management, information technology curriculum teaching, and maker education practice. Background Technology

[0002] Numerous medical studies have confirmed that poor pen-holding posture is a common and modifiable risk factor for both myopia and scoliosis in children and adolescents.

[0003] However, the current market lacks a truly intelligent product that meets the daily handwriting health management needs of families. High-end smart pens are typically expensive and primarily focus on teaching functions such as handwriting correction and learning assistance, with monitoring of pen grip pressure and fatigue often only playing a supporting role. Low-end physical posture correction products (such as pen grippers and corrective pens), while inexpensive, have limited functionality, passively restricting finger position without quantifying or assessing grip strength, and failing to proactively remind children when they feel fatigued. A widely available, low-cost product that uses pressure sensors for real-time pen grip fatigue monitoring is currently unavailable in the market.

[0004] In addition, existing products generally have the following technical defects: (1) The monitoring dimension is singular, focusing only on finger position or overall grip strength, and cannot achieve decoupled collection of independent pressure of the thumb, index finger and middle finger; (2) Fatigue judgment lacks quantitative standards and cannot provide graded warnings; (3) The intervention method is rigid, and the forced limit design is easy to cause resistance from children; (4) The system is closed, and users cannot adjust parameters or expand functions according to individual differences; (5) It fails to coordinate the monitoring of hand grip posture with the whole body sitting posture.

[0005] The purpose of this invention is to overcome the shortcomings of the prior art and provide an intelligent writing assistance system and its monitoring method based on multi-channel pressure sensing for coordinated monitoring of pen grip fatigue and sitting posture. It achieves independent acquisition of pen grip pressure in three channels and graded fatigue discrimination with a low-cost open-source architecture, and works in conjunction with mobile terminal sitting posture visual detection to build a two-dimensional health monitoring system for hand and whole body posture. Summary of the Invention

[0006] To address the aforementioned issues, this invention proposes an intelligent writing assistance system based on multi-channel pressure sensing for coordinated monitoring of pen grip fatigue and sitting posture, comprising a hardware monitoring terminal and a mobile software application.

[0007] The hardware monitoring terminal includes:

[0008] Three flexible thin-film pressure sensors are deployed in the pen grip area and at positions corresponding to the base of the thumb pad, the radial base of the index finger, and the supporting part of the middle finger, respectively, to independently collect pen grip pressure signals from the thumb, index finger, and middle finger.

[0009] The signal conditioning circuit is connected to the three sensors and is used to perform impedance matching, filtering, amplification and level conversion on the pressure signal;

[0010] An embedded microcontroller (preferably Arduino series) with an analog-to-digital converter interface connected to a signal conditioning circuit is used to acquire pressure data and execute fatigue discrimination algorithms.

[0011] A Bluetooth communication module, connected to the microcontroller, is used to wirelessly transmit pressure data and judgment results to the mobile device;

[0012] An acoustic feedback module (buzzer), connected to the microcontroller, is used to issue graded prompts based on fatigue assessment results.

[0013] Mobile software applications (preferably WeChat mini programs) include:

[0014] Bluetooth data receiving module, used to receive and parse stress data uploaded by hardware terminal;

[0015] The data visualization module is used to display the three-channel pressure waveform and fatigue alarm status in real time;

[0016] The sitting posture detection module integrates the YOLOv8-Pose computer vision model, uses the mobile camera to collect user images in real time, extracts the coordinates of key joints of the human head, shoulders, and spine, calculates the head tilt angle, shoulder tilt angle, and lateral offset of the spine, and determines whether the sitting posture is abnormal.

[0017] The reminder module is used to issue reminders via screen pop-ups and / or voice broadcasts when abnormal sitting posture or fatigue alarms are received.

[0018] The fatigue detection algorithm is a lightweight rule-based model that comprehensively considers three dimensions: pressure threshold, duration, and alarm interval. The specific rules are as follows:

[0019] Mild fatigue: When the pressure value of any sensor exceeds 65% of the range, and this condition continues to accumulate for 2 hours, the first fatigue alarm is triggered; the buzzer sounds for 2 seconds.

[0020] Moderate fatigue: If, within 10 minutes after the mild alarm ends, the single-channel pressure exceeds 65% again and continues for 1 hour, the alarm will be upgraded to moderate fatigue; the buzzer will sound for 4 seconds.

[0021] Severe fatigue: If the above conditions are met for the third time within 10 minutes after the moderate alarm ends, it is determined to be severe fatigue; the buzzer sounds for 5 seconds.

[0022] A method for co-monitoring pen-holding fatigue and sitting posture based on the above system includes the following steps:

[0023] S1. Hardware terminal deployment: Fix three flexible thin-film pressure sensors to the pen grip area at the positions corresponding to the thumb, index finger, and middle finger, respectively, and connect them to the embedded microcontroller;

[0024] S2. Signal Acquisition and Preprocessing: The microcontroller acquires three-channel pressure analog signals at a set sampling frequency, converts them into digital quantities after analog-to-digital conversion, and maps them to pressure percentage;

[0025] S3. Fatigue Judgment: The microcontroller runs a graded fatigue judgment algorithm to comprehensively judge the current fatigue level. When the triggering conditions are met, it drives the acoustic feedback module to emit a graded prompt tone, and at the same time outputs the pressure data and fatigue level via Bluetooth.

[0026] S4. Mobile terminal reception and visualization: The mobile terminal software receives data via Bluetooth and displays the three-channel pressure waveform and alarm status in real time;

[0027] S5. Posture Detection: The mobile software accesses the camera and runs the YOLOv8-Pose model to detect key points of the human body in real time, calculates the posture angle, and determines whether the sitting posture is abnormal.

[0028] S6. Collaborative Reminder: When the hardware triggers a fatigue alarm or the software detects abnormal posture, the mobile device will issue a reminder via pop-up window and / or voice broadcast.

[0029] The beneficial effects of this invention are:

[0030] 1. Three-channel distributed sensing: Based on the "three-point support" model of sports biomechanics, it realizes independent pressure collection of the thumb, index finger and middle finger, providing a multi-dimensional data foundation for refined writing health management.

[0031] 2. Lightweight graded fatigue identification: Without relying on neural networks, fatigue identification accuracy of 89.7% is achieved on a low-cost 8-bit microcontroller by comprehensively judging through three dimensions: threshold, duration, and interval, with a single identification time of less than 50 milliseconds.

[0032] 3. Two-dimensional monitoring with hardware and software collaboration: The hardware is responsible for real-time pressure collection and edge computing, while the mobile device is responsible for visual detection of sitting posture. Data is aggregated via Bluetooth to form a correlation monitoring system of "hand force application - torso posture".

[0033] 4. Open source architecture and educational extension attributes: All firmware code is open source, allowing users to modify alarm thresholds and add or remove sensor types; the accompanying serial port drawing tool enables real-time visualization of pressure waveforms, meeting the teaching needs of the "Process and Control" module in primary and secondary school information technology.

[0034] 5. Low cost and widespread availability: The cost of hardware materials and DIY kits is controlled at the hundred-yuan level, which is only one-third to one-fifth of similar smart pen products, filling the market gap for hundred-yuan writing fatigue monitoring products. Attached Figure Description

[0035] Figure 1 This is a schematic diagram of a hardware monitoring terminal;

[0036] Figure 2 This is a flowchart of the system of the present invention;

[0037] Figure 3 A data visualization interface diagram for mobile software;

[0038] Figure 4 This is a diagram of the visualization interface for sitting posture detection in this invention.

[0039] Figure 5 This is a screenshot of an example of open-source code. Detailed Implementation

[0040] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0041] Example 1: Hardware Terminal

[0042] This embodiment provides a hardware terminal construction scheme for an intelligent writing assistance system based on multi-channel pressure sensing that coordinates pen grip fatigue and sitting posture monitoring.

[0043] (I) Sensor Selection and Layout

[0044] Three flexible thin-film pressure sensors were selected, with a rated range of 0.2 to 60 Newtons and a resolution accuracy of 0.01 Newtons. Based on the "three-point support" model of writing biomechanics, the three sensors were fixed to the pen grip area at the corresponding positions as follows:

[0045] Channel 1 (thumb): The contact area at the base of the thumb's pad;

[0046] Channel 2 (index finger): The contact area at the radial root of the index finger;

[0047] Channel 3 (middle finger): The contact area of ​​the supporting part of the distal phalanx of the middle finger.

[0048] The sensor is pre-tightened and fixed by an embedded food-grade silicone sleeve structure. The inner side of the silicone sleeve has anti-slip texture and positioning protrusions to ensure that the sensor is always aligned with the correct anatomical position of the finger.

[0049] (II) Main Control and Signal Link

[0050] The Arduino UNO development board is used as the embedded microcontroller. The output signals of the three sensors are impedance matched by a voltage divider circuit and then connected to the three analog input ports A0, A1, and A2 of the Arduino. The Arduino's built-in 10-bit analog-to-digital converter (ADC) converts the 0~5V analog voltage signal into a digital quantity of 0~1023, and then maps it to a pressure percentage of 0%~100%.

[0051] A Bluetooth communication module (HC-05 or HM-10) connects to the Arduino's serial communication interface (TX / RX) to wirelessly transmit pressure data and fatigue levels to the mobile device. A buzzer connects to a digital output port (such as D8) to emit a grading alert tone.

[0052] (III) Power Supply Method

[0053] It connects to a computer via a USB interface for power, operates at 5V DC, and consumes approximately 0.3 watts. It can also be powered by a 3.7V lithium battery, and with the help of a power management module, it can be used for portable purposes.

[0054] Example 2: Embedded Firmware and Fatigue Detection Algorithm

[0055] This embodiment describes the specific implementation of a fatigue discrimination algorithm running in an embedded microcontroller.

[0056] (I) Data Acquisition and Preprocessing

[0057] The Arduino continuously reads the three-channel ADC values ​​at a sampling frequency of 100Hz, calculating a moving average every 10 samples to suppress transient noise. The averaged ADC values ​​are then mapped to a pressure percentage using the following formula:

[0058] Pressure percentage = (ADC value - ADC no-load value) / (ADC full-load value - ADC no-load value) × 100%

[0059] The ADC no-load value is the output when the sensor is under no pressure (approximately 300), and the ADC full-load value is the output when the sensor withstands the maximum range (approximately 820).

[0060] (II) Grading Fatigue Judgment Logic

[0061] The system maintains three state variables: mild alarm flag, moderate alarm flag, and severe alarm flag; and three timers: mild duration timer, moderate duration timer, and severe duration timer.

[0062] The judgment logic is as follows:

[0063] When the pressure percentage of any channel is > 65%, the corresponding duration timer is started; when the pressure drops to ≤ 65%, the timer is reset to zero.

[0064] If the cumulative duration of continuous exceedance of any channel reaches 2 hours and the mild alarm sign is false, a mild fatigue alarm will be triggered: the buzzer will sound for 2 seconds, the mild alarm sign will be set to true, and the alarm time will be recorded.

[0065] If, within 10 minutes after a minor alarm ends, any channel continues to exceed the limit for 1 hour, and the moderate alarm flag is false, a moderate fatigue alarm will be triggered: the buzzer will sound for 4 seconds, and the moderate alarm flag will be set to true.

[0066] If, within 10 minutes after a moderate alarm ends, any channel continues to exceed the limit for 1 hour for the third time, and the severe alarm flag is false, then a severe fatigue alarm will be triggered: the buzzer will sound for 5 seconds, and the severe alarm flag will be set to true.

[0067] All alarm information is simultaneously output via serial port with "mild / moderate / severe" indicators, and then transmitted to the mobile device via Bluetooth module.

[0068] The lightweight rule model was validated through a standardized pen-holding and writing experiment (sample size N=30, each person writing for 4 hours). The model achieved an accuracy rate of 89.7% in recognizing writing fatigue, with a single discrimination operation taking approximately 35 milliseconds.

[0069] Example 3: Mobile software and posture monitoring

[0070] This embodiment describes the implementation scheme of a mobile WeChat mini program, focusing on the collaborative monitoring function of pen-holding fatigue and sitting posture.

[0071] (a) Bluetooth data reception

[0072] After the mini-program is launched, it automatically scans for and connects to preset Bluetooth devices (hardware terminals). Once connected, it receives three-channel pressure percentage data and fatigue level indicators according to the agreed protocol, and updates the numerical display and waveform curves in the UI interface in real time.

[0073] (ii) Data visualization

[0074] Use a canvas component to draw three colored curves, corresponding to the pressure change trends of the thumb (red), index finger (green), and middle finger (blue), respectively. The horizontal axis represents time (last 60 seconds), and the vertical axis represents pressure percentage (0~100%). When a fatigue alarm is received, the corresponding level icon and text prompt are displayed at the top of the interface.

[0075] (III) YOLOv8-Pose Posture Detection

[0076] The phone's front-facing camera is used to capture images at a frame rate of 15fps. A lightweight YOLOv8-Pose model is used to detect the coordinates of 17 key points on the human body in real time, with a focus on the following key points: nose, left eye, right eye, left shoulder, right shoulder, left hip, and right hip.

[0077] Calculate the following attitude angles based on the key point coordinates:

[0078] Forward head tilt angle: The angle between the vector connecting the midpoint of the shoulder and the tip of the nose and the vertical direction is greater than 15° and is considered abnormal forward tilt.

[0079] Shoulder tilt angle: The horizontal angle between the lines connecting the left and right shoulders; a tilt angle greater than 10° is considered a tendency toward scoliosis.

[0080] Distance between eyes and tabletop: estimated based on the relative position of the tip of the nose and the shoulder; if too close, it is considered abnormal near-field vision.

[0081] If any posture indicator exceeds the limit for more than 5 seconds, a pop-up window on the screen and a voice reminder will be triggered (such as "Please raise your head and straighten your chest" or "Pay attention to your posture").

[0082] (iv) The coordinated effect of pen grip fatigue and sitting posture

[0083] When the hardware sends a severe fatigue alarm and the software simultaneously detects an abnormal head tilt angle, the mini-program determines that the user is in a "high-risk writing state," triggering a composite warning strategy and suggesting immediate rest and full-body relaxation. This collaborative judgment logic correlates the health risks of hand fatigue and abnormal posture, two independent dimensions, significantly improving the accuracy and effectiveness of intervention. For example, when the system detects that the user's middle finger pressure is consistently high (≥65% for 1.5 hours) while the head tilt angle exceeds 20°, it simultaneously triggers a pen-holding posture adjustment reminder and a sitting posture correction reminder, providing a comprehensive suggestion to "stand up and move around for 5 minutes."

[0084] The intelligent writing assistance system and its monitoring method based on multi-channel pressure sensing for coordinated monitoring of pen grip fatigue and sitting posture provided by this invention can be implemented in a low-cost, easily replicable open-source hardware format. It is suitable for various scenarios such as home writing health management, school information technology curriculum teaching, maker education practice, and rehabilitation training assistance in medical institutions. All parts of the system utilize mature commercial components and open-source software frameworks, demonstrating good industrial feasibility and prospects for large-scale production.

[0085] Although the above embodiments have been shown and described, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Any changes, modifications, substitutions and variations made to the above embodiments by those skilled in the art are within the protection scope of the present invention.

Claims

1. A smart writing assistance system based on multi-channel pressure sensing for coordinated monitoring of pen-holding fatigue and sitting posture, characterized in that, include: Hardware monitoring terminals include: Three flexible thin-film pressure sensors are deployed in the pen grip area and at positions corresponding to the base of the thumb pad, the radial base of the index finger, and the supporting part of the middle finger, respectively, to independently collect pen grip pressure signals from the thumb, index finger, and middle finger. The signal conditioning circuit is connected to the three sensors and is used to perform impedance matching, filtering, amplification and level conversion on the pressure signal; An embedded microcontroller, whose analog-to-digital conversion interface is connected to a signal conditioning circuit, is used to acquire pressure data and execute a fatigue discrimination algorithm. A Bluetooth communication module, connected to the microcontroller, is used to wirelessly transmit pressure data and judgment results to the mobile device; An acoustic feedback module, connected to the microcontroller, is used to issue graded prompt sounds based on fatigue assessment results; Mobile software applications, including: Bluetooth data receiving module, used to receive and parse stress data uploaded by hardware terminal; The data visualization module is used to display the three-channel pressure waveform and fatigue alarm status in real time; The sitting posture detection module integrates the YOLOv8-Pose computer vision model, uses the mobile camera to collect user images in real time, extracts the coordinates of key joints of the human head, shoulders, and spine, calculates the head tilt angle, shoulder tilt angle, and lateral offset of the spine, and determines whether the sitting posture is abnormal. The reminder module is used to issue reminders via screen pop-ups and / or voice broadcasts when abnormal sitting posture or fatigue alarms are received.

2. The intelligent writing assistance system according to claim 1, characterized in that, The fatigue discrimination algorithm running in the embedded microcontroller is a lightweight rule model that comprehensively considers three dimensions: pressure threshold, duration, and alarm interval. Specifically, it includes: Mild fatigue detection: When the pressure value of any sensor exceeds 65% of the range, and this state continues to accumulate for a first preset time (e.g., 2 hours), a mild fatigue alarm is triggered. Moderate fatigue judgment: If, within the second preset time period (e.g., 10 minutes) after the mild alarm ends, the single channel pressure exceeds 65% again and continues for a third preset time period (e.g., 1 hour), a moderate fatigue alarm is triggered. Severe fatigue detection: If, within the second preset time period after the moderate alarm ends, the single-channel pressure exceeds 65% for the third time and continues for the third preset time, a severe fatigue alarm is triggered.

3. The intelligent writing assistance system according to claim 1, characterized in that, The embedded microcontroller is an Arduino series microcontroller. All firmware code of the system is released in open source form, allowing users to access, modify, and reprogram it through the Arduino integrated development environment.

4. The intelligent writing assistance system according to claim 1, characterized in that, The hardware monitoring terminal also includes a power supply unit, which is powered by a USB interface or a battery, with a working voltage of 5V DC and a power consumption of no more than 0.3 watts.

5. The intelligent writing assistance system according to claim 1, characterized in that, The mobile application also includes: a data storage module for recording pressure data, alarm history, and fatigue level for each writing session; a statistical analysis module for generating daily / weekly / monthly health reports, displaying pressure trend charts, pressure distribution percentages for each finger, and alarm frequency statistics; and a parameter configuration module that provides a graphical threshold setting interface, allowing users to adjust alarm parameters without writing code.

6. The intelligent writing assistance system according to claim 1, characterized in that, The flexible thin-film pressure sensor has a range of 0.2 to 60 Newtons and a resolution accuracy of 0.01 Newtons. The sensor is pre-tightly fixed to the pen barrel by an embedded silicone sleeve structure, and the silicone sleeve is made of food-grade silicone material.

7. A method for coordinating monitoring of pen grip fatigue and sitting posture based on the intelligent writing assistance system according to any one of claims 1 to 6, characterized in that, Includes the following steps: S1. Hardware terminal deployment: Fix three flexible thin-film pressure sensors to the pen grip area at the positions corresponding to the thumb, index finger, and middle finger, respectively, and connect them to the embedded microcontroller; S2. Signal Acquisition and Preprocessing: The microcontroller acquires three-channel pressure analog signals at a set sampling frequency, converts them into digital quantities after analog-to-digital conversion, and maps them to pressure percentage; S3. Fatigue Judgment: The microcontroller runs a graded fatigue judgment algorithm to comprehensively judge the current fatigue level. When the triggering conditions are met, it drives the acoustic feedback module to emit a graded prompt tone, and at the same time outputs the pressure data and fatigue level via Bluetooth. S4. Mobile terminal reception and visualization: The mobile terminal software receives data via Bluetooth and displays the three-channel pressure waveform and alarm status in real time; S5. Posture Detection: The mobile software accesses the camera and runs the YOLOv8-Pose model to detect key points of the human body in real time, calculates the posture angle, and determines whether the sitting posture is abnormal. S6. Collaborative Reminder: When the hardware triggers a fatigue alarm or the software detects abnormal sitting posture, the mobile device issues a reminder via pop-up window and / or voice broadcast, forming a two-dimensional health intervention based on hand exertion and torso posture.

8. The method according to claim 7, characterized in that, The fatigue discrimination algorithm in step S3 does not rely on a neural network model. Its computational load and memory usage are controlled within the range that an 8-bit microcontroller can handle. The time for a single discrimination operation is less than 50 milliseconds, and the fatigue recognition accuracy is not less than 89.7%.

9. The method according to claim 7, characterized in that, It also includes a semi-automatic calibration step: the user clicks the calibration button on the mobile device, keeps the pen barrel unloaded, and the microcontroller automatically records the zero-point offset value of the three-channel sensor and performs compensation to adapt to the pen-holding habits of different users.

10. The application of the intelligent writing assistance system according to any one of claims 1 to 6 or the method according to any one of claims 7 to 9 in at least one of the following scenarios: family writing health management for school-age children; teaching aids for the "process and control" module of information technology courses in primary and secondary schools; maker education practice and open-source hardware secondary development platform.

Images