Sitting posture and fatigue monitoring device and method

Abstract

The invention relates to a sitting posture and fatigue monitoring device and method.The device comprises a seat, a support, a mobile phone and a server, the seat is provided with a backrest, a cushion and a control circuit, the backrest and the cushion are provided with a plurality of pressure sensors, the control circuit is arranged at the bottom of the seat, one end of the control circuit is connected with the pressure sensors, and the other end of the control circuit is connected with the mobile phone through Bluetooth; the mobile phone is supported in front of the seat through the support, a camera of the mobile phone faces the seat, the mobile phone is connected with the server through application software, the pressure sensor collects pressure data, the mobile phone shoots video data, and the server obtains a sitting posture evaluation result and a fatigue monitoring result through the pressure data and the video data. Compared with the prior art, the method has the advantages of high accuracy, high robustness and the like.

Description

Technical Field

[0001] The present invention relates to the technical field of posture monitoring, and in particular to a sitting posture and fatigue monitoring device and method. Background Art

[0002] With the progress of society, the study burden of students and the office pressure of working people are increasing, and the sitting time of the groups has also increased significantly. In order to prevent the formation of bad sitting postures, avoid sitting for long periods of time, and reduce fatigue, it is very important to design an intelligent sitting posture evaluation system that can monitor sitting postures, monitor fatigue and remind people to sit for a long time, and record the length of time sitting.

[0003] Current sitting posture or fatigue status monitoring systems usually split and analyze the two parts of monitoring results. For sitting posture recognition, the existing technology uses: 1) peripheral devices to capture sitting posture images for image recognition analysis; 2) wearable devices to measure the degree of body bending for analysis; 3) pressure sensors for force analysis. For fatigue status recognition, the fatigue status of the user is determined by calculating the proportion of the time when the user's eyes are closed to the total time or analyzing the direction of the eye gaze.

[0004] When sitting for a long time, people often change their sitting posture and adjust the stress points to reduce the effects of fatigue. In the prior art, there is no cross-analysis between the two recognition methods. A single recognition method is overly sensitive to mild self-adjustment or overly indulgent to incorrect sitting posture adjustments. The overall accuracy of the recognition method is low, making it difficult to achieve effective supervision. Summary of the invention

[0005] The purpose of the present invention is to provide a sitting posture and fatigue monitoring device and method in order to overcome the defect of low precision in the above-mentioned prior art.

[0006] The purpose of the present invention can be achieved by the following technical solutions:

[0007] A sitting posture and fatigue monitoring device comprises a seat, a bracket, a mobile phone and a server. The seat is provided with a backrest, a seat cushion and a control circuit. The backrest and the seat cushion are equipped with a plurality of pressure sensors. The control circuit is arranged at the bottom of the seat. One end of the control circuit is connected to each pressure sensor, and the other end is connected to a mobile phone via Bluetooth. The mobile phone is supported in front of the seat by the bracket, and the camera of the mobile phone faces the seat. The mobile phone is connected to the server via application software. The pressure sensor collects pressure data, and the mobile phone shoots video data. The server obtains sitting posture evaluation results and fatigue monitoring results through the pressure data and video data.

[0008] Furthermore, the pressure sensors of the seat cushion are distributed in the seat cushion in a rectangular array, and the pressure sensors of the backrest are distributed along a straight line at the edge of the backrest.

[0009] Furthermore, the seat cushion has 13 pressure sensors, of which 3 pressure sensors are distributed on the contact surface of the user's left leg, 3 pressure sensors are distributed on the contact surface of the user's right leg, 2 pressure sensors are distributed between the user's legs, and 5 sensors are distributed on the contact surface of the user's buttocks. There are 3 pressure sensors on the backrest, which are distributed in a straight line at 5 cm from the upper edge of the backrest, and the distance between each adjacent pressure sensor is not less than 5 cm.

[0010] Furthermore, the control circuit includes a microprocessor, and Bluetooth, a multiplexer switch and a pressure sensor wire connected to the microprocessor. The microprocessor, Bluetooth, the multiplexer switch and the pressure sensor wire are all fixed to the bottom of the seat. The pressure sensor wire connects the pressure sensor and the input end of the multiplexer switch, and the output end of the multiplexer switch is connected to the microprocessor.

[0011] Furthermore, the distance between the bracket and the seat is 1 m, and the bracket has multiple sections and multiple height adjustment knobs.

[0012] Furthermore, after the mobile phone receives the pressure sensor data collected by the microprocessor, it is uploaded to the server through the application software, and the recorded video data is uploaded to the server at the same time. The server also stores the sitting posture evaluation model, the user's identity information and the user's historical monitoring data.

[0013] Furthermore, the signal acquisition process of the device is: read the values ​​of the sampling ports in sequence, perform analog-to-digital conversion on the values ​​of the sampling ports, and store the analog-to-digital conversion results until all pressure sensors complete a sampling through the sampling ports, and a set of data is obtained. After a set of data is sent, the next round of sampling is performed after a specified delay.

[0014] A second aspect of the present invention is a monitoring method based on any one of the above-mentioned sitting posture and fatigue monitoring devices, comprising the following steps:

[0015] S1: Adjust the height of the phone according to the user's height;

[0016] S2: Start the control circuit and prepare for data collection;

[0017] S3: The user sits on the seat, the microprocessor collects the pressure sensor data and sends it to the mobile phone, the mobile phone sends the pressure sensor data to the server, and records the video at the same time, and sends the video to the server;

[0018] S4: The server obtains the sitting posture evaluation result according to the pressure sensor data and the video through the sitting posture evaluation model;

[0019] S5: The server uses the expression recognition model to capture the dynamic features of the user's face through the video, and combines the coordinate attention mechanism to obtain the facial monitoring results;

[0020] S6: Combine the facial monitoring results and the sitting posture evaluation results to obtain fatigue monitoring results.

[0021] S7: Receive the sitting posture recognition and fatigue status recognition results returned by the server via the mobile phone.

[0022] Furthermore, the server segments the pressure sensor data through the sliding window method, extracts features from the video stream data through the transformer encoder to obtain semantic features of the video stream, and then models the semantic features of the video stream through spatiotemporal patch extraction. Through contrastive learning and mutual information maximization, the sitting posture recognition-related representation and the existing modality are obtained. The completed modality is obtained through a distillation module containing a lateral cross-modal conversion unit. The existing modality, the completed modality and the real modality are fused to obtain the feature representation, which is input into the sitting posture evaluation model to obtain the sitting posture evaluation result.

[0023] Furthermore, according to a sitting posture and fatigue monitoring method according to claim 8, the sitting posture evaluation model is divided into multiple data sets according to body height, and a corresponding sitting posture evaluation sub-model is trained for each data set.

[0024] Compared with the prior art, the present invention has the following beneficial effects:

[0025] 1) The present invention uses a seat equipped with a pressure sensor and a camera device to collect pressure sensor data and video data respectively, analyzes the pressure sensor data and video to obtain a sitting posture evaluation result, adopts an expression recognition model to obtain a facial monitoring result, and effectively integrates different data modalities through modal factorization and hierarchical knowledge distillation technology. The facial monitoring result and the sitting posture evaluation result are combined to obtain a fatigue monitoring result, which significantly improves the recognition accuracy and robustness of the model.

[0026] 2) The present invention uses a mobile phone as a data transmission hub, simplifies the communication process between the pressure sensor and the server, facilitates uploading and downloading data, and provides convenience for long-term sitting posture and fatigue monitoring and obtaining monitoring results.

[0027] 3) The present invention rationally distributes the positions of pressure sensors and sets sensors based on the main force-bearing positions that a person may select when the sitting posture changes, so as to collect pressure data more accurately. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 The figure is a side view of a sitting posture detection and fatigue state detection device of the present invention.

[0029] Figure 2 The present invention discloses a collection circuit diagram of a sitting posture detection and fatigue state detection device.

[0030] Figure 3The present invention is a hardware flow chart of a sitting posture detection and fatigue state detection device.

[0031] Figure 4 FIG. 4 is a distribution diagram of the pressure sensors in the seat cushion of the embodiment.

[0032] Figure 5 FIG. 4 is a distribution diagram of the pressure sensors in the backrest of the embodiment.

[0033] Explanation of the markings in the figure: 1. seat, 2. bracket, 3. mobile phone, 4. server, 5. control circuit. DETAILED DESCRIPTION

[0034] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0035] Example 1

[0036] The present invention is a sitting posture and fatigue monitoring device and method, such as Figure 1 As shown, the device includes a seat 1, a bracket 2, a mobile phone 3 and a server 4. The seat 1 is provided with a backrest, a cushion and a control circuit 5. The backrest and the cushion are equipped with multiple pressure sensors. The control circuit 5 is arranged at the bottom of the seat 1. One end of the control circuit 5 is connected to each pressure sensor, and the other end is connected to the mobile phone 3 via Bluetooth. The mobile phone 3 is supported in front of the seat 1 by the bracket 2. The camera of the mobile phone 3 faces the seat 1. The mobile phone 3 is connected to the server 4 through the application software. The pressure sensor collects pressure data, the mobile phone 3 shoots video data, and the server 4 obtains the sitting posture evaluation result and the fatigue monitoring result through the pressure data and the video data.

[0037] like Figure 4 As shown, the seat cushion is provided with pressure sensors in a rectangular array, and there are 13 pressure sensors in the seat cushion, of which 3 pressure sensors are distributed on the contact surface of the user's left leg, 3 pressure sensors are distributed on the contact surface of the user's right leg, 2 pressure sensors are distributed between the user's legs, and 5 sensors are distributed on the contact surface of the user's buttocks; Figure 5 As shown, there are three pressure sensors on the backrest, which are distributed in a straight line at 5 cm from the upper edge of the backrest. The distance between each adjacent pressure sensor is not less than 5 cm.

[0038] The control circuit 5 includes a microprocessor, and Bluetooth, a multiplex switch and a pressure sensor wire connected to the microprocessor. The microprocessor, Bluetooth, the multiplex switch and the pressure sensor wire are all fixed to the bottom of the seat 1. The pressure sensor wire is connected to the input end of the pressure sensor and the multiplex switch, and the output end of the multiplex switch is connected to the microprocessor. The above modules and wires can be fixed to the bottom of the seat 1 using straps.

[0039] like Figure 2 As shown, Y0 to Y15 are the acquisition terminals of 16 analog voltage signals, which are connected to the 16 input terminals of the multiplexer. S0, S1, S2, and S3 are the four address bits of the multiplexer, which are connected to the P2.0, P2.1, P2.2, and P2.3 ports of the MSP430F149 respectively. The output terminal Z of the multiplexer is connected to the P6.0 port of the MSP430F149.

[0040] The mobile phone 3 is supported by the bracket 2 and uses a front camera. The height of the mobile phone 3 and the distance from the seat 1 can be adjusted. In this example, the height of the mobile phone 3 is 80 cm and the distance from the seat 1 is 1 m. The height of the mobile phone 3 can be adjusted by the knob at the connection between the first section and the second section of the bracket 2.

[0041] like Figure 3 As shown, the signal acquisition process of the device is: read the values ​​of the sampling ports in turn, perform analog-to-digital conversion on the values ​​of the sampling ports, and store the analog-to-digital conversion results until all pressure sensors (a total of 16 pressure sensors in this embodiment) complete a sampling through the sampling ports, and a group of data is obtained. After a group of data is sent, the next round of sampling is performed after a delay of 6s.

[0042] The present invention also includes a server and a mobile phone 3APP user terminal connected in communication. The mobile phone is connected to the control circuit via Bluetooth, and the voltage data of the circuit is sent to the mobile phone. The mobile phone transmits the data in the form of a txt file and stores it in the server. At the same time, the mobile phone records 2s of video stream data, which is transmitted and stored in the server in the form of an mp4 file. The data stored in the server includes basic information such as the user's account and password, as well as the user's sitting posture, including date, sitting posture category, and sitting posture duration. At the same time, the server includes a standard data set, which is divided into 160cm, 170cm, and 180cm standard data sets according to height. If the user has no customized data set, the model is trained according to the data set selected by the user, and the currently received data is used as the input of the model, and the user's current sitting posture is output, and the duration of the sitting posture is increased and a voice reminder is issued in the mobile phone APP. The voice reminder function can be manually turned off by the user according to the situation. The user can view today's long sitting time and historical long sitting time on the mobile phone APP; view the long sitting time for a week and the long sitting time ranking on the web page. The web version also has an additional parent-child function, where users can bind themselves to their parents. Parents can log in to the web version and send sedentary reminders to the bound teenage users.

[0043] The detection method of the present invention is:

[0044] S1: Adjust the height of the phone according to the user's height;

[0045] When using the present invention to detect sitting posture and fatigue status, the user adjusts the height and distance of the mobile phone according to the current status. At the same time, if the user needs to place the mobile phone on the desktop, the desktop bracket can be used to support the mobile phone. It is necessary to ensure that the front camera of the mobile phone can capture the face.

[0046] S2: Start the control circuit and prepare for data collection;

[0047] The user connects the power cord of the seat (typeC port), opens the mobile phone APP, enters the login interface, enters the account and password. If you have not registered, click the "Register" button to register. Enter the main interface, click Bluetooth on the first interface, turn on Bluetooth, click Receive Data, and start data monitoring.

[0048] S3: The user sits on the seat, the microprocessor collects the pressure sensor data and sends it to the mobile phone, the mobile phone sends the pressure sensor data to the server, and records the video at the same time, and sends the video to the server;

[0049] When the user sits on the seat, each pressure sensor receives pressure, and the microcontroller transmits data to the mobile phone APP via Bluetooth. The mobile phone APP transmits the data to the server and records the video and sends it to the server.

[0050] S4: The server obtains the sitting posture evaluation result according to the pressure sensor data and the video through the sitting posture evaluation model;

[0051] S5: The server uses the expression recognition model to capture the dynamic features of the user's face through the video, and combines the coordinate attention mechanism to obtain the facial monitoring results;

[0052] S6: Combining the facial monitoring result and the sitting posture evaluation result to obtain a fatigue monitoring result.

[0053] The server preprocesses the received pressure sensor data and video stream data. For pressure sensor data, the sliding window method is used for data segmentation; for video stream data, the multi-level self-supervised learning video Transformer encoder is used for feature extraction. The encoder designs ten self-supervised tasks, including frame prediction, frame interpolation, and center expansion, to achieve efficient capture of the semantic features of the sitting video stream. At the same time, through customized spatiotemporal patch extraction, contextual semantic information is accurately modeled.

[0054] The server adopts a hierarchical knowledge distillation strategy based on modality factorization for multimodal fusion. First, the modality is decoupled into posture recognition related representation and modality specific representation through contrastive learning and mutual information maximization. Then, a distillation module (M_IID) is used to transfer and complete knowledge, which contains a horizontal cross-modal transformation unit (U_LIT). Finally, the existing modality, the completed modality and the real modality are fused to form a unified feature representation.

[0055] S7: Receive the sitting posture recognition and fatigue status recognition results returned by the server via the mobile phone.

[0056] The mobile phone APP receives the sitting posture recognition and fatigue status recognition results returned by the server, and broadcasts them by voice at the same time, and the corresponding sitting posture information is displayed in the APP.

[0057] The present invention adopts multimodal data fusion technology, which deeply fuses the data collected by the pressure sensor and the camera to achieve high-precision recognition of sitting posture, and uses video stream data to identify fatigue status. This improvement provides users with a more comprehensive health management solution, reminding users to adjust their posture or rest in time to avoid the health risks brought by long-term sitting. The present invention effectively integrates the advantages of different data modalities through modal factorization and hierarchical knowledge distillation technology, overcomes the limitations of single-modal systems, and significantly improves recognition accuracy and robustness.

[0058] The present invention has also optimized the hardware design, using the low-power, low-cost MSP430F149 single-chip microcomputer and HC08 Bluetooth module, reducing the overall cost and making it more cost-effective. The present invention allows users to customize their own sitting posture model according to their personal needs, and conduct long-term data monitoring through the server. Users can view historical data at any time, analyze their own sitting posture trends, and make adjustments according to the system's suggestions.

[0059] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A sitting posture and fatigue monitoring device, characterized in that: The invention comprises a seat (1), a bracket (2), a mobile phone (3) and a server (4); the seat (1) is provided with a backrest, a seat cushion and a control circuit (5); the backrest and the seat cushion are provided with a plurality of pressure sensors; the control circuit (5) is arranged at the bottom of the seat (1); one end of the control circuit (5) is connected to each pressure sensor, and the other end is connected to the mobile phone (3) via Bluetooth; the mobile phone (3) is supported in front of the seat (1) via the bracket (2); the camera of the mobile phone (3) faces the seat (1); the mobile phone (3) is connected to the server (4) via application software; the pressure sensor collects pressure data; the mobile phone (3) shoots video data; and the server (4) obtains a sitting posture evaluation result and a fatigue monitoring result through the pressure data and the video data.

2. A sitting posture and fatigue monitoring device according to claim 1, characterized in that: The pressure sensors of the seat cushion are distributed in the seat cushion in a rectangular array, and the pressure sensors of the backrest are distributed along a straight line on the edge of the backrest.

3. A sitting posture and fatigue monitoring device according to claim 2, characterized in that: The seat cushion has 13 pressure sensors, of which 3 are distributed on the contact surface of the user's left leg, 3 are distributed on the contact surface of the user's right leg, 2 are distributed between the user's legs, and 5 are distributed on the contact surface of the user's buttocks. The backrest has 3 pressure sensors, which are distributed in a straight line at 5 cm from the upper edge of the backrest, and the distance between adjacent pressure sensors is not less than 5 cm.

4. A sitting posture and fatigue monitoring device according to claim 1, characterized in that: The control circuit (5) comprises a microprocessor, and Bluetooth, a multiplexer switch and a pressure sensor wire connected to the microprocessor. The microprocessor, Bluetooth, the multiplexer switch and the pressure sensor wire are all fixed to the bottom of the seat (1). The pressure sensor wire connects the pressure sensor and the input end of the multiplexer switch, and the output end of the multiplexer switch is connected to the microprocessor.

5. The sitting posture and fatigue monitoring device according to claim 1, characterized in that: The distance between the support (2) and the seat is 1 m, and the support (2) has multiple sections and multiple height adjustment knobs.

6. The sitting posture and fatigue monitoring device according to claim 1, characterized in that: After the mobile phone (3) receives the pressure sensor data collected by the microprocessor, it uploads the data to the server (4) through the application software, and simultaneously records the video data and uploads it to the server (4). The server (4) also stores the sitting posture evaluation model, the user's identity information and the user's historical monitoring data.

7. A sitting posture and fatigue monitoring method according to claim 1, characterized in that: The signal acquisition process of the device is: reading the values ​​of the sampling ports in sequence, performing analog-to-digital conversion on the values ​​of the sampling ports, and storing the analog-to-digital conversion results until all pressure sensors complete a sampling through the sampling ports, and a group of data is obtained. After sending a group of data, the next round of sampling is performed after a specified delay.

8. A monitoring method based on a sitting posture and fatigue monitoring device as claimed in any one of claims 1 to 7, characterized in that: The following steps are involved: S1: Adjust the height of the phone according to the user's height; S2: Start the control circuit and prepare for data collection; S3: The user sits on the seat, the microprocessor collects the pressure sensor data and sends it to the mobile phone, the mobile phone sends the pressure sensor data to the server, and records the video at the same time, and sends the video to the server; S4: The server obtains the sitting posture evaluation result according to the pressure sensor data and the video through the sitting posture evaluation model; S5: The server uses the expression recognition model to capture the dynamic features of the user's face through the video, and combines the coordinate attention mechanism to obtain the facial monitoring results; S6: Combine the facial monitoring results and the sitting posture evaluation results to obtain fatigue monitoring results. S7: Receive the sitting posture recognition and fatigue status recognition results returned by the server via the mobile phone.

9. A sitting posture and fatigue monitoring method according to claim 8, characterized in that: The server segments the pressure sensor data through a sliding window method, extracts features from the video stream data through a transformer encoder to obtain semantic features of the video stream, and then models the semantic features of the video stream through spatiotemporal patch extraction. Through contrastive learning and mutual information maximization, the sitting posture recognition-related representation and the existing modality are obtained. The completed modality is obtained through a distillation module containing a lateral cross-modal conversion unit. The existing modality, the completed modality and the real modality are fused to obtain a feature representation, which is input into a sitting posture evaluation model to obtain a sitting posture evaluation result.

10. A sitting posture and fatigue monitoring method according to claim 8, characterized in that: The sitting posture evaluation model is divided into a plurality of data sets according to body height, and a corresponding sitting posture evaluation sub-model is trained for each data set.

Images