Method and system for eye use, sitting posture monitoring and adjustment based on multi-sensor fusion

Abstract

The application discloses a method and system for eye use, sitting posture monitoring and adjustment based on multi-sensor fusion, relates to the technical field of eye use health management, and comprises the following steps: wearing smart glasses equipment for a measured person, collecting real-time eye use data of the measured person, including eye use distance data, head posture data and sitting posture data, performing feature extraction on the real-time eye use data, obtaining a distance sequence, a posture angle set and a sitting posture set, performing time sequence evolution analysis on the distance sequence, the posture angle set and the sitting posture set, obtaining a plurality of bad eye use nodes, grouping the nodes based on bad types to obtain different types of abnormal node groups, performing early warning based on the abnormal node groups, performing lens fogging on the smart glasses equipment according to a warning result, and adjusting the sitting posture of the measured person, recording operation contents performed according to the warning result to a device terminal, and pushing the operation contents to a mobile device of the measured person, so that timely supervision and correction of eye use behavior and sitting posture are realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of eye health management, in particular to a method and system for eye use and sitting posture monitoring and adjustment based on multi-sensor fusion. BACKGROUND

[0002] Currently, in the field of eye health monitoring, some technologies have attempted to identify behaviors and remind through sensors, for example, some devices use single infrared distance measurement or camera image recognition to monitor the distance between the user and the book or screen, or issue a sound alarm when an abnormal posture is detected. However, such methods often have the following limitations:

[0003] Firstly, a single type of sensor cannot fully capture complex eye use and sitting posture behaviors, such as distance alone cannot determine whether the head is tilted or the sitting posture is forward leaning, leading to missed judgments or false alarms, secondly, most systems only make instantaneous judgments, lack analysis of behavior evolution over time, and cannot identify persistent bad habits, which is not conducive to long-term health management.

[0004] In view of the above problems, although some research attempts to introduce multi-sensor, but in the data fusion level, it is mostly limited to simple summary, and cannot realize time series correlation analysis and grouping processing, and still lags behind in real-time response structure and early warning mechanism, which cannot support effective closed-loop behavior correction. SUMMARY

[0005] The purpose of the present application is to provide a method and system for eye use and sitting posture monitoring and adjustment based on multi-sensor fusion to solve the problems in the background art.

[0006] In order to achieve the above purpose, the present application provides the following technical scheme: a method for eye use and sitting posture monitoring and adjustment based on multi-sensor fusion, comprising the following steps:

[0007] Step S1: wearing a smart glasses device for the measured person, for collecting real-time eye use data of the measured person, the real-time eye use data including eye use distance data, head posture data and sitting posture data, performing feature extraction on the real-time eye use data to obtain corresponding distance sequence, posture angle set and sitting posture set;

[0008] Step S2: performing time series evolution analysis on the distance sequence, posture angle set and sitting posture set to obtain a plurality of bad eye use nodes, grouping the nodes based on the bad types of the bad eye use nodes to obtain different types of abnormal node groups;

[0009] Step S3: performing early warning based on the abnormal node groups, performing lens fogging on the smart glasses device according to the early warning result, and reminding the measured person to adjust the sitting posture, recording the operation content performed according to the early warning result to the device terminal, and pushing to the mobile device of the measured person.

[0010] In a preferred embodiment, the smart glasses device is worn by the subject for collecting real-time eye use data of the subject, and the process of feature extraction on the real-time eye use data to obtain the distance sequence, the posture angle set, and the sitting posture set includes:

[0011] The real-time eye use data of the subject is collected by the distance sensor and the posture sensor, including eye use distance data, head posture data, and sitting posture data, which respectively include eye use distance, head image, and sitting posture image of the subject at a plurality of time nodes;

[0012] The eye use distance at the plurality of time nodes is labeled, and a distance safety threshold is set. When the eye use distance of a certain label is greater than the distance safety threshold, the corresponding eye use distance is identified as eye use safety distance, otherwise, it is identified as eye use harmful distance;

[0013] The head image and the sitting posture image at each time node are divided into image frames to obtain a plurality of head images and sitting posture images, and a private processing node and a public processing node are established, and a corresponding visual analysis architecture is established for each frame of head image and sitting posture image;

[0014] The private processing node extracts features from the real-time eye use data, and the public processing node obtains related parameters after feature extraction, and integrates to obtain the distance sequence, the posture angle set, and the sitting posture set.

[0015] In a preferred embodiment, the process of obtaining the distance sequence, the posture angle set, and the sitting posture set includes:

[0016] The private processing node does not access the local area network, and all operations in the private processing node are performed locally, and the public processing node accesses the local area network to obtain cloud computing resources to integrate all related parameters;

[0017] The eye use distance is arranged based on the timestamp of the time node to construct the distance sequence;

[0018] A plurality of head images and sitting posture images at the same time node are allocated to a private processing node, the plurality of head images and sitting posture images in the same private processing node are aligned in frame position to obtain a plurality of image pairs, and the plurality of image pairs are arranged in position in the corresponding private processing node;

[0019] Key point detection is performed on each image pair in the order of position arrangement to obtain all head feature points and sitting posture feature points of the subject, the head feature points and the sitting posture feature points in the same image pair are respectively posture calibrated to obtain different posture angles of the head and different sitting posture angles of the upper body, all posture angles are combined as the posture angle set, and all sitting posture angles are integrated as the sitting posture set.

[0020] In a preferred embodiment, the process of performing time series evolution analysis on the distance sequence, the posture angle set and the sitting posture set to obtain a plurality of bad eye use nodes includes:

[0021] Establishing time series coordinate axes for the distance sequence, the posture angle set and the sitting posture set respectively, mapping a plurality of eye use safe distances and eye use harmful distances, a plurality of posture angles and a plurality of sitting postures onto the respective time series coordinate axes as respective evolution trend characteristics;

[0022] Setting up a sliding window, setting the window size to the time length of a time node, aligning the time series coordinate axes of the distance sequence, the posture angle set and the sitting posture set based on the time nodes, and sequentially traversing the evolution trend characteristics of the different time series coordinate axes under each time node by the sliding window to determine whether it is a synchronous bad behavior;

[0023] If yes, mark the current time node as a bad eye use node, otherwise, do not perform the operation, and after the sliding window traverses all the time nodes, a plurality of bad eye use nodes are marked.

[0024] In a preferred embodiment, the process of grouping nodes based on the bad types of bad eye use nodes to obtain different types of abnormal node groups includes:

[0025] The bad types of bad eye use nodes include major abnormalities and minor abnormalities;

[0026] The major abnormalities include head posture abnormalities and sitting posture abnormalities, which are determined based on posture angles not conforming to a preset angle range or sitting postures not conforming to a preset standard sitting posture view;

[0027] The minor abnormalities include a plurality of refined abnormal types in the head posture abnormalities and the sitting posture abnormalities;

[0028] Grouping all the bad eye use nodes based on the major abnormalities to obtain a major abnormal group corresponding to the number of major abnormalities, and taking the bad types of major abnormalities as the source search credentials of the major abnormal group, grouping the bad eye use nodes in the major abnormal group that are in the same refined abnormal type based on the minor abnormalities to obtain a plurality of different types of abnormal node groups under the major abnormal group.

[0029] In a preferred embodiment, the process of performing pre-warning based on the abnormal node groups, lens fogging the intelligent glasses device according to the pre-warning result, and reminding the measured person to adjust the sitting posture includes:

[0030] Each abnormal node group is composed of a plurality of abnormal eye behaviors corresponding to different time nodes and belonging to the same refined abnormal type, and an early warning message and an early warning processing measure are established based on the first abnormal eye behavior in the abnormal node group;

[0031] For each abnormal node group, a cache container is constructed for each abnormal eye behavior other than the one for which the early warning message is established, and the cache container is used to call the early warning message of the first abnormal eye behavior in the abnormal node group;

[0032] The early warning message of the first abnormal eye behavior is mapped and stored in a pre-deployed virtual machine, and the storage address between the first abnormal eye behavior and the virtual machine is sent to a plurality of cache containers. Each cache container calls the early warning message from the virtual machine based on the storage address, and the early warning processing measure of the early warning message is cached in advance. According to the cache content, the lens of the smart glasses device is fogged, or the subject is prompted to adjust his / her sitting posture.

[0033] In a preferred embodiment, the process of recording the operation content performed according to the early warning result to the device terminal and pushing it to the mobile device of the subject includes:

[0034] The operation content is stored in the Bluetooth module on the smart glasses device, a data delivery channel is established between the Bluetooth module and the device terminal, and a plurality of data integration points are set in the data delivery channel. Each data integration point is used to integrate similar parts of the operation content, and the corresponding integrated data package is constructed;

[0035] A push address is established for each integrated data package and sent to the device terminal. Based on each push address, the device terminal allocates a data display area in its own device storage space to obtain the integrated data package. The operation content is disassembled into a plurality of sub-stage operation contents based on time nodes, and a broadcast queue and an execution queue are set for each data integration point. The broadcast queue and the execution queue push the operation content in the data delivery channel to the mobile device of the subject and display it.

[0036] In a preferred embodiment, the process of pushing all the operation content in the data delivery channel to the mobile device of the subject and displaying it in the mobile device includes:

[0037] The initial sub-stage operation content under the same push address is taken as the source target content, and the other sub-stage operation contents are taken as auxiliary push contents. Based on the stage time sequence, the plurality of sub-stage operation contents are arranged in position on the broadcast queue and the execution queue;

[0038] The execution queue is used for data analysis and data pushing of the source target content and the auxiliary push content, and the broadcast queue sets a plurality of broadcast nodes, the first broadcast node obtains the data analysis progress of the source target content arranged in the first position, and synchronously broadcasts to other broadcast nodes in the same broadcast queue, and the auxiliary push content is analyzed and buffered;

[0039] The operation contents of the sub-stages after graphing are packaged respectively, a plurality of to-be-pushed data packets are generated, when the source target content is viewed, a data pulling request is generated in the mobile device of the tested person, the corresponding to-be-pushed data packet is obtained based on the data pulling request, and the operation content corresponding to the eye monitoring and adjustment of the tested person at a certain time node after graphing is displayed on the mobile device.

[0040] The application also provides a system for eye monitoring and adjustment based on multi-sensor fusion, which comprises:

[0041] A data acquisition and processing module is used for wearing smart glasses for the tested person, for acquiring real-time eye data of the tested person, the real-time eye data comprising eye distance data, head posture data and sitting posture data, and performing feature extraction on the real-time eye data to obtain corresponding distance sequence, attitude angle set and sitting posture set;

[0042] A data analysis and grouping module is used for time sequence evolution analysis on the distance sequence, attitude angle set and sitting posture set to obtain a plurality of bad eye nodes, node grouping based on the bad types of the bad eye nodes to obtain different types of abnormal node groups;

[0043] An abnormality early warning and processing module is used for early warning based on the abnormal node groups, lens fogging of the smart glasses based on the early warning result, and reminding the tested person to adjust the sitting posture, and recording the operation content performed according to the early warning result to the device terminal and pushing to the mobile device of the tested person.

[0044] In the above technical solution, the application provides technical effects and advantages:

[0045] 1、The application simultaneously acquires eye distance, head posture and sitting posture data, and performs feature extraction and integration through the cooperation of private processing nodes and public processing nodes, constructs distance sequence, attitude angle set and sitting posture set with time stamp, provides reliable data basis for subsequent time sequence evolution analysis, realizes multi-dimensional related data monitoring, establishes corresponding time sequence coordinate axes for the distance sequence, attitude angle set and sitting posture set respectively, and aligns and traverses based on time nodes by using a sliding window, can effectively identify the "synchronous bad behavior" of simultaneous appearance of too close distance and abnormal posture, accurately mark bad eye nodes, and reduce the occurrence of false positives.

[0046] 2、The application groups the abnormal eye use nodes into different abnormal node groups according to the large category abnormalities and the small category detailed types, generates early warnings according to the abnormal node groups, and generates early warnings and measures based on the first abnormal behavior in the abnormal node groups, so that the same type of abnormal behaviors in the subsequent abnormal node groups directly call the related early warning and measure operation data of the first abnormal behavior through the cache container to complete their corresponding early warning processing, which reduces the processing delay to a certain extent, realizes fast and consistent early warning response, and improves the real-time performance of intervention and user experience. BRIEF DESCRIPTION OF DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments described in the present application, and other drawings can also be obtained by those skilled in the art based on these drawings.

[0048] Figure 1 The method flowchart of the present application.

[0049] Figure 2 The system block diagram of the present application. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical scheme and advantages of the embodiments of the present application more clear, the technical scheme in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0051] Embodiment 1, please refer to Figure 1 The method for monitoring and adjusting eye use and sitting posture based on multi-sensor fusion described in this embodiment includes the following steps:

[0052] Step S1: Wear smart glasses equipment for the measured person, which is used to collect real-time eye use data of the measured person, the real-time eye use data including eye use distance data, head posture data and sitting posture data, and the real-time eye use data is subjected to feature extraction to obtain corresponding distance sequence, posture angle set and sitting posture set;

[0053] Step S2: Time sequence evolution analysis is performed on the distance sequence, posture angle set and sitting posture set to obtain a plurality of abnormal eye use nodes, the nodes are grouped based on the abnormal types of the abnormal eye use nodes, and different types of abnormal node groups are obtained;

[0054] Step S3: warning based on the abnormal node group, lens fogging of the smart glasses device according to the warning result, and reminding the measured person to adjust the sitting posture, recording the operation content performed according to the warning result to the device terminal, and pushing to the mobile device of the measured person.

[0055] It needs to be further explained that, in the specific implementation process, the smart glasses device is worn by the measured person for collecting real-time eye data of the measured person, and the real-time eye data is subjected to feature extraction to obtain the corresponding distance sequence, attitude angle set and sitting posture set, which includes:

[0056] The smart glasses device is composed of a terminal host, a magnetic mirror clamp and a split frame;

[0057] The terminal host is composed of a micro control processor, a distance measuring sensor, an attitude sensor, a battery, a battery management module, a color changing film driving module and a on-off button; the micro control processor is used for cooperative control of the distance measuring sensor, the attitude sensor, the Bluetooth module, the battery management module, the color changing film driving module and the on-off button.

[0058] The battery management module is responsible for the charging and discharging function of the battery and the power supply to each other module;

[0059] It is judged whether the measured person wears glasses or not, if yes, the clamping piece type wearing method is selected, the terminal host is fixed to the glasses worn by the measured person through the magnetic mirror clamp, if not, the frame wearing type is selected, the terminal host, the magnetic mirror clamp and the split frame are assembled to constitute a complete smart glasses device for the measured person to wear;

[0060] The distance measuring sensor and the attitude sensor are used for collecting real-time eye data of the measured person, and the real-time eye data specifically includes eye distance data, head posture data and sitting posture data, the eye distance data includes eye distance of the measured person at a plurality of time nodes, the head posture data includes head image of the measured person at a plurality of time nodes, and the sitting posture data includes sitting posture image of the measured person at a plurality of time nodes.

[0061] The eye distance at a plurality of time nodes is labeled, the label is marked as i, i.e. i = 1, 2, 3, …, n, wherein n is a natural number greater than 0, the i-th eye distance is marked as , the distance safety threshold is set, and the distance safety threshold is marked as ; all eye distances with > are identified as eye safety distance; all eye distances with ≤ The eye distance marked as the eye-harmful distance. The eye distances marked in the plurality of times are arranged based on the time stamps corresponding to the time nodes, and a distance sequence is constructed, denoted as , then = { , , …, };

[0062] The head image and the sitting posture image under each time node are divided into image frames to obtain a plurality of frames of head images and sitting posture images under the corresponding time node, a private processing node and a public processing node are established, and a corresponding visual analysis architecture is established for each frame of head image and sitting posture image. The number of private processing nodes is the same as the number of visual analysis architectures.

[0063] The visual analysis architecture is used as a processing architecture for image analysis processing of each frame of head image or sitting posture image. The visual analysis architecture corresponding to each private processing node is different. The visual analysis architecture includes a plurality of image analysis points. The number and distribution of the image analysis points correspond to the arrangement of the pixel blocks of the head image or the sitting posture image in the private processing node.

[0064] The private processing node is used for feature extraction of real-time eye data.

[0065] The public processing node is used to obtain relevant parameters after feature extraction and integrate the relevant parameters to obtain a distance sequence, a set of posture angles, and a set of sitting postures.

[0066] The private processing node does not access the local area network. All operations in the private processing node are performed locally. All relevant parameters extracted by the plurality of private processing nodes are transmitted to the public processing node. The public processing node accesses the local area network and obtains cloud computing resources in the public processing node to integrate all relevant parameters extracted by the feature extraction. Specifically as follows:

[0067] The plurality of frames of head images and sitting posture images under the same time node are allocated to a private processing node. The plurality of frames of head images and sitting posture images in the same private processing node are aligned in frame position to obtain a plurality of image pairs. The plurality of image pairs are arranged in position in the corresponding private processing node.

[0068] Key point detection and posture calibration are performed on each image pair in the order of position arrangement. All head feature points corresponding to the head of the measured person are obtained through key point detection. The coverage of the head feature points specifically includes the corners of the mouth, the corners of the eyes, the contours of the nose, and the contours of the face. All sitting posture feature points of the upper body of the measured person are obtained. The coverage of the sitting posture feature points specifically includes the shoulders, elbows, wrists, hips, and waist.

[0069] The head feature points and the sitting posture feature points in the same image pair are respectively subjected to posture calibration, and the content of the posture calibration is: establishing a head frame model, filling the head frame model based on all head feature points, and establishing the rotation vector and the translation vector of the head formed by different head feature points relative to the camera, and converting the rotation vector into the Euler angle, the pitch angle, the yaw angle and the roll angle corresponding to the head posture;

[0070] A sitting posture frame model is established, the sitting posture frame model is filled based on all sitting posture feature points, the angle and the vector formed by different sitting posture feature points are calculated, and then the spinal curvature, the head forward inclination and the shoulder inclination are obtained;

[0071] The spinal curvature: when the included angle corresponding to the connecting line between the shoulder key point, the waist key point and the hip key point is calculated, if the angle of the included angle exceeds the preset included angle threshold, it indicates that the back is bent, otherwise, there is no problem;

[0072] The head forward inclination: the horizontal offset of the head relative to the shoulder is calculated, when the horizontal offset exceeds the preset offset threshold, it indicates that the head is forwardly inclined abnormally, the sitting posture is forwardly inclined, otherwise, it indicates that there is no abnormality;

[0073] The shoulder inclination: the inclination angle of the connecting line formed by different points on the shoulder is calculated, when the inclination angle exceeds the preset inclination threshold, it indicates that the sitting posture is skewed, otherwise, it indicates that it is normal.

[0074] Through the posture calibration, different posture angles corresponding to the head and different sitting posture angles corresponding to the upper body are obtained, all the posture angles are combined as a posture angle set, and all the sitting posture angles are integrated as a sitting posture set.

[0075] It should be further explained that, in the specific implementation process, the process of obtaining a plurality of bad eye use nodes by performing time sequence evolution analysis on the distance sequence, the posture angle set and the sitting posture set includes:

[0076] A time sequence coordinate axis corresponding to each of the distance sequence, the posture angle set and the sitting posture set is established, a plurality of eye use safe distances and eye use harmful distances in the distance sequence, a plurality of posture angles in the posture angle set and a plurality of sitting posture angles in the sitting posture set are respectively mapped to the corresponding time sequence coordinate axis, and the corresponding evolution trend characteristics are obtained;

[0077] The evolution trend characteristics corresponding to the distance sequence are decomposed into a plurality of first evolution trend nodes, and each first evolution trend node corresponds to an eye use distance at a time node.

[0078] The evolution trend feature corresponding to the posture angle set is disassembled into a plurality of two-class evolution trend nodes, and each two-class evolution trend node corresponds to a posture angle at a time node;

[0079] The evolution trend feature corresponding to the sitting posture set is disassembled into a plurality of three-class evolution trend nodes, and each three-class evolution trend node corresponds to a sitting posture at a time node.

[0080] A sliding window is set up, the window size of the sliding window is set to the time length corresponding to a time node, and the time sequence coordinate axes of the distance sequence, the posture angle set and the sitting posture set are aligned based on the time node. The evolution trend features of different time sequence coordinate axes under each time node are sequentially traversed by the sliding window, and it is determined whether it is a synchronous bad behavior;

[0081] When there is a harmful eye distance and any one of an abnormal posture angle or an abnormal sitting posture at the same time node on the time sequence coordinate axis traversed by the sliding window, it is determined that it is a synchronous bad behavior, the current time node is marked as a bad eye node, otherwise, it is not determined as a synchronous bad behavior, and no marking is performed, indicating a normal eye condition.

[0082] After the sliding window traverses all the time nodes on the time sequence coordinate axis, a plurality of bad eye nodes and a plurality of time nodes in a normal eye condition which are not marked are obtained.

[0083] It needs to be further explained that, in the specific implementation process, the process of grouping nodes based on the bad type of the bad eye node to obtain different types of abnormal node groups includes:

[0084] The bad type corresponding to the bad eye node includes a large-class abnormality and a small-class abnormality;

[0085] The large-class abnormality includes a head posture abnormality and a sitting posture abnormality, the head posture abnormality is determined based on a posture angle not conforming to a preset angle range, and the sitting posture abnormality is determined based on a sitting posture not conforming to a preset standard sitting posture view;

[0086] The small-class abnormality includes a plurality of refined abnormality types in the head posture abnormality and the sitting posture abnormality, specifically including head forward inclination, head backward inclination, head skew and head rotation in the head posture abnormality, and specifically including body lateral inclination, body forward inclination, shoulder skew and body backward inclination in the sitting posture abnormality.

[0087] Based on the large category abnormality, all the bad eye use nodes are grouped to obtain the abnormal large group corresponding to the number of large category abnormality, and the bad type of large category abnormality is taken as the source search voucher of the abnormal large group. Based on the small category abnormality, the bad eye use nodes in the same detailed abnormal type in the abnormal large group are further grouped to obtain several abnormal node groups of different types under the abnormal large group.

[0088] It needs to be further explained that, in the specific implementation process, the process of warning based on the abnormal node group, fogging the lens of the intelligent glasses device according to the warning result, and reminding the measured person to adjust the sitting posture includes:

[0089] Each abnormal node group is composed of a plurality of corresponding different time nodes of bad eye use behaviors in the same detailed abnormal type. The first bad eye use behavior in the abnormal node group is used to establish a warning message, and a corresponding warning processing measure of the warning message is also established synchronously;

[0090] For each bad eye use behavior in the abnormal node group except the first bad eye use behavior, a corresponding cache container is constructed. The cache container is used to call the data of the warning message of the first bad eye use behavior in the abnormal node group.

[0091] The warning message of the first bad eye use behavior is mapped and stored to the pre-deployed virtual machine. The storage address between the first bad eye use behavior and the virtual machine is sent to the plurality of cache containers. Each cache container calls the warning message from the virtual machine based on the storage address, and the warning processing measure corresponding to the warning message. The warning processing measure is cached in advance. According to the cache content, the lens of the intelligent glasses device is fogged, or the measured person is prompted to adjust the sitting posture.

[0092] The content of the lens fogging is that when the eye use distance between the measured person and the viewed object reaches the harmful eye use distance, a warning message is sent synchronously. The content of the warning message is "your viewing distance is too close, please adjust in time". The micro control processor on the terminal host controls the on-off key of the color film driving module on the intelligent glasses device to cover the fogging film to the whole lens area, thereby completing the lens fogging.

[0093] It needs to be further explained that, in the specific implementation process, the process of recording the operation content executed according to the warning result to the device terminal and pushing it to the mobile device of the measured person includes:

[0094] The operation content executed according to the warning result of all the warning messages is stored in the Bluetooth module on the intelligent glasses device. A data delivery channel is established between the Bluetooth module and the device terminal. A plurality of data integration points are set in the data delivery channel.

[0095] Each data integration point is used to integrate similar part data contents, construct corresponding integrated data package, obtain device permission of the device terminal, establish a push address for each integrated data package, and send all push addresses to the device terminal based on the obtained device permission.

[0096] Based on each push address, the device terminal allocates a corresponding data display area in the device storage space, obtains the corresponding integrated data package from the data display area, and decomposes the operation content corresponding to the integrated data package into several sub-stage operation contents based on time nodes in each data display area.

[0097] Based on each data integration point in the data delivery channel, a corresponding broadcast queue and an execution queue are set, the initial sub-stage operation content under the same push address is taken as the source target content, other sub-stage operation contents under the same push address are taken as auxiliary push contents, and several sub-stage operation contents are arranged in position on the broadcast queue and the execution queue based on stage timing.

[0098] The execution queue is used for data analysis or data push of the source target content and the auxiliary push content, and the target object of data push is the terminal device. Any sub-stage operation content is graphed through data analysis, and the graphed sub-stage operation content is convenient for the subject to view and browse on the mobile device terminal.

[0099] The broadcast queue is correspondingly provided with several broadcast nodes, wherein the first broadcast node is used to obtain the data analysis progress of the source target content arranged in the first position, and the data analysis progress of the current source target content is broadcast to other broadcast nodes in the same broadcast queue. Other broadcast nodes obtain the broadcast content, and the auxiliary push content stored in the broadcast node is analyzed and cached in advance.

[0100] All graphed sub-stage operation contents under the same data integration point are packaged respectively, and several to-be-pushed data packages are generated. When the source target content to be viewed is needed, a data pull request is generated in the mobile device corresponding to the subject, the corresponding to-be-pushed data package is obtained based on the data pull request, and the operation content corresponding to the eye and sitting posture monitoring and adjustment of the subject at a certain time node after graphing is displayed on the mobile device.

[0101] Other to-be-pushed data packages under the same data integration point are pulled in the mobile device corresponding to the data pull request in linkage, and whether to view the operation content of the subject for eye and sitting posture monitoring and adjustment at other several time nodes corresponding to the linkage pull is determined by the user of the mobile device.

[0102] The operation content is viewed on a pre-installed sitting posture APP on the mobile device, and the graphed operation content is used to record the eye use, sitting posture and all related operations of the subject in the monitored period, specifically including the total eye use time of the subject, the time length of normal eye use / sitting posture and abnormal eye use / sitting posture respectively occupied in the total eye use time, and the specific abnormal eye use related data and abnormal sitting posture related data in each abnormal period, and after comprehensively counting all abnormal data, the degree of health impact on the subject in the current total eye use time is judged, and a corresponding health report is generated and stored on the mobile device for viewing.

[0103] Embodiment 2, please refer to Figure 2 As shown in the figure, the application also provides a system for eye use and sitting posture monitoring and adjustment based on multi-sensor fusion, which comprises:

[0104] A data acquisition and processing module is used to wear a smart glasses device for the subject, to collect real-time eye use data of the subject, the real-time eye use data including eye use distance data, head posture data and sitting posture data, to perform feature extraction on the real-time eye use data, to obtain corresponding distance sequence, posture angle set and sitting posture set;

[0105] A data analysis and grouping module is used to perform time series evolution analysis on the distance sequence, posture angle set and sitting posture set, to obtain a plurality of bad eye use nodes, to perform node grouping based on the bad types of the bad eye use nodes, and to obtain different types of abnormal node groups;

[0106] An abnormal warning and processing module is used to perform warning based on the abnormal node groups, to perform lens fogging on the smart glasses device according to the warning result, and to remind the subject to adjust the sitting posture, and to record the operation content performed according to the warning result to the device terminal and push to the mobile device of the subject.

[0107] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto, any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for monitoring and adjusting eye use and sitting posture based on multi-sensor fusion, characterized in that, Includes the following steps: Step S1: The subject wears smart glasses to collect real-time eye data, including eye distance data, head posture data, and sitting posture data. Features are extracted from the real-time eye data to obtain the corresponding distance sequence, posture angle set, and sitting posture set. Step S2: Perform time-series evolution analysis on the distance sequence, attitude angle set, and sitting posture set to obtain several poor eye use nodes. Group the nodes based on the poor eye use node type to obtain different types of abnormal node groups. Step S3: Issue an early warning based on the abnormal node group, fog up the lenses of the smart glasses device according to the warning result, and remind the subject to adjust their sitting posture. Record the operation performed according to the warning result to the device terminal and push it to the subject's mobile device. The process of performing time-series evolution analysis on distance sequences, attitude angle sets, and sitting posture sets to obtain several nodes of poor eye use includes: Establish time-series coordinate axes for distance sequences, attitude angle sets, and sitting posture sets respectively, and map several safe and harmful distances for eye use, several attitude angles, and several sitting postures onto their respective time-series coordinate axes as their respective evolution trend characteristics. A sliding window is set up, with the window size set to the time length of a time node. The temporal coordinate axes of the distance sequence, attitude angle set, and sitting posture set are aligned based on the time node. The sliding window iterates through the evolution trend characteristics of different temporal coordinate axes under each time node to determine whether it is a synchronization problem. If so, mark the current time node as a bad eye use node; otherwise, do not perform any operation, and slide the window to traverse all time nodes to mark several bad eye use nodes. The process of grouping nodes based on the type of poor eye use, and then obtaining different types of abnormal node groups, includes: The types of abnormalities in poor eye use include major categories and minor categories; Major anomalies include head posture anomalies and sitting posture anomalies. Head posture anomalies and sitting posture anomalies are determined based on posture angles that do not conform to the preset angle range or sitting postures that do not conform to the preset standard sitting posture view, respectively. Subcategories of abnormalities include several refined abnormality types within head posture abnormalities and sitting posture abnormalities; Based on major anomalies, all poor eye use nodes are grouped into nodes to obtain a number of major anomaly groups corresponding to the major anomalies. The type of poor eye use in the major anomalies is used as the source retrieval credential for the major anomalies. Based on minor anomalies, poor eye use nodes in the same detailed anomaly type in the major anomalies are grouped into several different types of anomaly node groups under the major anomalies.

2. The method for monitoring and adjusting eye use and sitting posture based on multi-sensor fusion according to claim 1, characterized in that, The process of having the test subject wear smart glasses to collect real-time eye use data, extracting features from the real-time eye use data, and obtaining the corresponding distance sequence, posture angle set, and sitting posture set includes: Real-time eye data of the subject is collected by ranging and posture sensors, including eye distance data, head posture data and sitting posture data, including eye distance, head image and sitting posture image of the subject at several time points. The viewing distance at several time points is labeled, and a safe viewing threshold is set. When the viewing distance at a certain label is greater than the safe viewing threshold, the corresponding viewing distance is marked as a safe viewing distance; otherwise, it is marked as a harmful viewing distance. The head image and sitting posture image at each time point are divided into image frames to obtain several frames of head image and sitting posture image. Private processing nodes and public processing nodes are established to establish a corresponding visual analysis architecture for each frame of head image and sitting posture image. Private processing nodes extract features from real-time eye use data, while public processing nodes obtain the relevant parameters after feature extraction and integrate them to obtain distance sequences, pose angle sets, and sitting posture sets.

3. The method for monitoring and adjusting eye use and sitting posture based on multi-sensor fusion according to claim 2, characterized in that, The process of obtaining the distance sequence, the set of pose angles, and the set of sitting postures includes: Private processing nodes are not connected to the local area network, and all operations within the private processing node are performed locally. Public processing nodes are connected to the local area network to obtain all relevant parameters for cloud computing resource integration. The distance sequence is constructed by arranging the eye-use distances based on the timestamps of time nodes; Several frames of head images and posture images at the same time node are assigned to a private processing node. The frame positions of several frames of head images and posture images in the same private processing node are aligned to obtain several corresponding image pairs. The positions of several image pairs are arranged in the corresponding private processing nodes. Keypoint detection is performed on each image pair according to the positional arrangement order to obtain all head feature points and sitting posture feature points of the subject. The head feature points and sitting posture feature points in the same image pair are calibrated separately to obtain different head posture angles and different upper body sitting postures. All posture angles are merged into a posture angle set, and all sitting postures are integrated into a sitting posture set.

4. The method for monitoring and adjusting eye use and sitting posture based on multi-sensor fusion according to claim 3, characterized in that, The process of issuing early warnings based on abnormal node groups, fogging the lenses of the smart glasses device according to the early warning results, and reminding the test subject to adjust their sitting posture includes: Each abnormal node group consists of several unhealthy eye behaviors corresponding to different time nodes within the same refined abnormality type. An early warning message and early warning handling measures are established based on the first unhealthy eye behavior in the abnormal node group. For each abnormal node group, in addition to establishing warning messages, other undesirable eye-use behaviors are configured to create their own cache containers for data retrieval of the warning message for the first undesirable eye-use behavior within the abnormal node group. The warning message of the first poor eye use behavior is mapped and stored in a pre-deployed virtual machine. The storage address that maps the first poor eye use behavior to the virtual machine is sent to several cache containers. Each cache container retrieves the warning message from the virtual machine based on the storage address, and the warning processing measures of the warning message are cached in advance. Based on the cached content, the lenses of the smart glasses device are fogged up, or the subject is prompted to adjust their sitting posture.

5. The method for monitoring and adjusting eye use and sitting posture based on multi-sensor fusion according to claim 4, characterized in that, The process of recording the actions performed based on the warning results to the device terminal and pushing them to the test subject's mobile device includes: The operation content is stored in the Bluetooth module of the smart glasses device, a data transmission channel is established between the Bluetooth module and the device terminal, and several data integration points are set up in the data transmission channel. Each data integration point is used to integrate the data content with similar operation content to build the corresponding integrated data package. A push address is established for each integrated data packet and sent to the device terminal. The device terminal allocates a data display area in its own device storage space based on each push address to obtain the integrated data packet. The operation content is broken down into several sub-stage operation content based on time nodes. A broadcast queue and execution queue are set up for each data integration point. The broadcast queue and execution queue push the operation content in the data delivery channel to the mobile device of the test subject and display it.

6. The method for monitoring and adjusting eye use and sitting posture based on multi-sensor fusion according to claim 5, characterized in that, The process of pushing all operations within the data delivery channel to the test subject's mobile device and displaying them on the mobile device includes: The initial sub-stage operation content under the same push address is used as the source target content, and other sub-stage operation content is used as auxiliary push content. Based on the stage sequence, the positions of several sub-stage operation contents are arranged in the broadcast queue and the execution queue. The execution queue is used for data parsing and pushing of source target content and auxiliary push content. The broadcast queue is set with several broadcast nodes. The first broadcast node obtains the data parsing progress of the source target content that is at the top of the position and broadcasts it synchronously to other broadcast nodes in the same broadcast queue. The auxiliary push content is parsed and cached. The sub-stage operations at the same data integration point are packaged into several data packages to be pushed. When the required source target content is viewed, a data pull request is generated on the test subject's mobile device. Based on the data pull request, the corresponding data package to be pushed is obtained, and the corresponding operation content of the test subject's eye use and sitting posture monitoring and adjustment at a certain time point after charting is displayed on the mobile device.

7. A system for monitoring and adjusting eye use and sitting posture based on multi-sensor fusion, used to implement the method for monitoring and adjusting eye use and sitting posture as described in any one of claims 1 to 6, characterized in that, The system includes: The data acquisition and processing module is used to have the smart glasses device worn by the test subject to collect the test subject's real-time eye use data, which includes eye distance data, head posture data, and sitting posture data. The module extracts features from the real-time eye use data to obtain the corresponding distance sequence, posture angle set, and sitting posture set. The data analysis and grouping module is used to perform time-series evolution analysis on distance sequences, attitude angle sets, and sitting posture sets to obtain several poor eye use nodes. Based on the poor eye use node's type of poor eye use, the nodes are grouped to obtain different types of abnormal node groups. The anomaly warning and handling module issues warnings based on anomaly node groups, fogs up the lenses of the smart glasses device according to the warning results, and reminds the test subject to adjust their sitting posture. The operation content performed according to the warning results is recorded to the device terminal and pushed to the test subject's mobile device.

Images