Method and device for realizing automatic switching of books in point reading

Abstract

The present invention relates to the field of image processing technology, and in particular to a method and device for realizing automatic switching of books in point reading. The method comprises the following steps: using a camera device to track hand movements and capture eye states to generate dynamic gesture data and eye state data. Predicting the action intention of the dynamic gesture data to generate gesture intention data; evaluating the switching intention of the eye state data to generate eye intention data; stacking the predicted intention using the gesture intention data and the eye intention data to generate switching intention analysis data; automatically switching books according to the switching intention analysis data to generate switching book data; analyzing the switching habit based on the switching book data to generate the switching habit data; identifying the book switching based on the switching habit data to generate the book re-switching data. The present invention improves the accuracy of identifying the intention of book switching by analyzing the dynamic data of the hands and eyes.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method and device for realizing automatic switching of books in point reading. Background Art

[0002] With the continuous development of technology, the use of gesture recognition and eye movement analysis methods combined with advanced technologies such as deep learning has provided a more intelligent and efficient automatic switching method. Automatic switching allows readers to turn pages without manual work, providing a more convenient and interference-free reading experience. It can also be customized according to the user's gestures, eye movement habits, and reading habits to provide a personalized reading experience to meet the needs of different readers. However, the existing methods and devices for realizing automatic switching of books in point-and-reading have a poor automatic switching reading experience for users due to the single recognition method or the failure to analyze the user's reading habits. Summary of the invention

[0003] Based on this, it is necessary to provide a method for realizing automatic switching of books during point reading to solve at least one of the above technical problems.

[0004] To achieve the above object, a method for realizing automatic switching of books in point reading is provided, the method comprising the following steps:

[0005] Step S1: using a camera device to perform hand motion capture and recognition, and generate recognition success signal data; based on the recognition success signal data, performing hand motion tracking and eye state capture, and generating dynamic gesture data and eye state data;

[0006] Step S2: Analyze the motion characteristics of the dynamic gesture data to generate gesture motion characteristic data; predict the action intention based on the gesture motion characteristic data to generate gesture intention data;

[0007] Step S3: performing eye movement trajectory analysis on the eye state data to generate fused trajectory data; performing gaze area duration analysis based on the fused trajectory data to generate focus duration distribution data; performing switching intention evaluation based on the focus duration distribution data to generate eye intention data;

[0008] Step S4: using the gesture intention data and the eye intention data to predict the intention stacking, and generating switching intention analysis data; performing automatic book switching processing according to the switching intention analysis data, and generating switching book data;

[0009] Step S5: Perform switching habit analysis based on the book switching data to generate switching habit data; simplify the identification method based on the switching habit data to generate simplified identification strategy data; perform book switching identification based on the simplified identification strategy data to generate book re-switching data.

[0010] The present invention captures hand movements through a camera device, provides signal data for successful recognition, generates dynamic gesture data and eye state data, and is used to identify important information such as user behavior patterns, gesture habits, and attention changes. It helps to understand the habits and interests of users during reading, and provides data support for personalized services and understanding of reading habits. The motion feature data provides an accurate description of gesture movements, including information such as speed, direction, and acceleration. It helps to accurately capture the user's gesture behavior. For example, it determines whether the user wants to turn pages or perform other operations. Such predictions provide an intelligent reading experience, allowing the device to better respond to user expectations and improve reading efficiency. By analyzing the eye movement trajectory and generating fusion trajectory data, it helps to understand the user's gaze point and visual attention when reading. To provide a more comprehensive understanding of reading habits and user attention. By analyzing the duration of the gaze area, focusing duration distribution data is generated, which helps to evaluate the user's concentration. It can be used to infer whether the user is ready to switch books. Gesture intention data and eye intention data are predicted and stacked to generate switching intention analysis data, which helps to understand the user's intention more comprehensively and improve the accuracy and intelligence of book switching. Automatic switching processing is performed according to the switching intention analysis data to generate switching book data. Such automated processing provides an intelligent reading experience, allowing the device to better respond to the user's intention. Therefore, the method and device for realizing automatic switching of books in point-and-reading of the present invention simultaneously analyzes the user's dynamic gesture data and eye state data to predict the automatic switching intention, thereby improving the accuracy of automatic switching recognition, and simplifies the recognition strategy by analyzing the user's switching habits, thereby improving the speed of automatic switching recognition.

[0011] Preferably, step S1 comprises the following steps:

[0012] Step S11: Acquire side palm image data; perform feature point trajectory marking on the side palm image data to generate image data to be trained;

[0013] Step S12: training a preset neural network model based on the image data to be trained to generate a side palm recognition model;

[0014] Step S13: using a camera device to capture the hand and generate hand image data;

[0015] Step S14: screening the side palm contour of the hand image data to generate side palm contour data;

[0016] Step S15: using the side palm recognition model to perform side palm recognition screening on the side palm contour data to generate recognition success signal data;

[0017] Step S16: Tracking hand movements and capturing eye states based on the recognition success signal data to generate dynamic gesture data and eye state data.

[0018] The present invention provides visual information of hand posture and form by acquiring side palm image data, which is helpful to understand the position, shape and dynamic characteristics of the hand. The side palm image data is marked with feature point trajectories to generate image data to be trained, which will serve as the basis for training a neural network model and help the model to better understand and identify hand features. The preset neural network model is trained based on the image data to be trained, so that the model can learn to identify side palm features and provide more accurate recognition and analysis capabilities. The hand is captured by a camera device to obtain real-time hand image data, which provides real-time input data for identifying and analyzing hand postures. By screening the side palm contour of the hand image data, it is helpful to extract the side palm features of the hand and reduce redundant information. The side palm contour data is screened by the side palm recognition model to generate recognition success signal data, which provides more accurate recognition of the side palm. Hand movement tracking and eye state capture are performed based on the recognition success signal data to generate dynamic gesture data and eye state data, which are used to understand important information of user behavior and visual focus and help infer the user's intention and behavior.

[0019] Preferably, step S2 comprises the following steps:

[0020] Step S21: extracting feature points from the dynamic gesture data to generate feature point motion data;

[0021] Step S22: performing feature point fitting processing according to the feature point motion data to generate gesture motion trajectory data;

[0022] Step S23: performing motion feature analysis based on the gesture motion trajectory data to generate gesture motion feature data;

[0023] Step S24: performing feature encoding processing on the gesture motion feature data to generate gesture feature vector data;

[0024] Step S25: Use the pre-trained gesture classifier to predict the action intention of the gesture feature vector data to generate gesture intention data.

[0025] The present invention extracts feature points from dynamic gesture data, provides the position and motion information of key points in the gesture, and helps to capture the key features of gesture motion. Fitting processing is performed based on the feature point motion data to generate gesture motion trajectory data. It helps to convert feature point motion into a more accurate and understandable gesture motion trajectory, and provides overall path information of gesture motion. Motion feature analysis is performed based on the gesture motion trajectory data to generate gesture motion feature data. These feature data can provide important information such as the speed, direction, acceleration, etc. of the gesture motion, which helps to understand the motion characteristics and patterns of the gesture. Feature encoding processing is performed on the gesture motion feature data to generate gesture feature vector data. This step helps to convert complex gesture motion features into more concise and easier to process feature vectors, thereby improving the operability and effectiveness of the data. The gesture feature vector data is predicted by using a pre-trained gesture classifier to generate gesture intention data. The intention of the gesture can be inferred and classified, providing effective prediction and recognition capabilities for understanding the user's gesture behavior.

[0026] Preferably, step S23 includes the following steps:

[0027] Step S231: using the boundary point calculation formula to mark the state transition boundary of the gesture motion trajectory data to generate gesture change point data;

[0028] Step S232: discretizing the feature point motion data using the gesture change point data to generate trajectory segment data;

[0029] Step S233: performing curvature calculation on the trajectory segment data to generate trajectory segment curvature data;

[0030] Step S234: performing differential direction evaluation on the trajectory segment data to generate angle change rate data; performing curvature correction on the trajectory segment curvature data using the angle change rate data to generate curvature correction data;

[0031] Step S235: using the curvature correction data to perform trajectory change correction on the motion trajectory data to generate trajectory correction data;

[0032] Step S236: performing speed change evaluation on the trajectory correction data to generate speed change data; performing energy change analysis based on the speed change data to generate energy change data;

[0033] Step S237: marking the trajectory correction data with energy points based on the energy change data to generate gesture motion feature data.

[0034] The present invention marks the state transition boundary of gesture motion trajectory data through a boundary point calculation formula to generate gesture change point data. These change points represent the state transition in the gesture motion trajectory, such as the start, end or key point position of a gesture action, which helps to understand the change process of the gesture action. The feature point motion data is discretized using the gesture change point data to generate trajectory segment data. It is helpful to divide the continuous gesture motion data into discrete trajectory segments. The curvature of the trajectory segment data is calculated to generate trajectory segment curvature data. These curvature data provide curvature information of the gesture motion trajectory, which helps to understand the curvature degree and shape characteristics of the gesture path. The curvature of the trajectory segment curvature data is corrected using the angle change rate data to generate curvature correction data. So that the curvature more accurately reflects the motion characteristics of the gesture, further improving the understanding and analysis of the gesture motion path. The motion trajectory data is corrected for trajectory changes using the curvature correction data to generate trajectory correction data. It is helpful to adjust the original trajectory data according to the curvature correction so that the trajectory is more consistent with the actual gesture motion path, thereby improving the accuracy and reliability of the data. The trajectory correction data is evaluated for speed change to generate speed change data. Provides information about the change in gesture movement speed, which helps analyze the speed change characteristics during gesture movement. Performs energy change analysis based on speed change data to generate energy change data. Energy change data represents the energy consumption of gesture movement, which helps understand the strength and degree of change of gesture movements. Based on the energy change data, energy points are marked on the trajectory correction data to generate gesture movement feature data. Describes the energy change points of gesture movement, which helps extract key features of gesture movement.

[0035] Preferably, the boundary point calculation formula in step S231 is as follows:

[0036]

[0037] In the formula, S i is the state value of trajectory point i, x i is the horizontal coordinate of trajectory point i, x i-1 is the horizontal coordinate of trajectory point i-1, y i is the ordinate of trajectory point i, y i-1 is the ordinate of trajectory point i-1, α is the weight value of angle transformation, v i-1 is the vector value between trajectory point i-1 and trajectory point i, v i is the vector value between trajectory point i and trajectory point i+1.

[0038] The present invention constructs a boundary point calculation formula for marking the state transition boundary of gesture motion trajectory data and generating gesture change point data. The formula fully considers the horizontal coordinate x of trajectory point i. i , the horizontal coordinate x of trajectory point i-1 i-1, the ordinate y of trajectory point i i , the ordinate y of trajectory point i-1 i-1 , the vector value v between trajectory point i-1 and trajectory point i i-1 , the vector value v between trajectory point i and trajectory point i+1 i And the interaction relationship between variables, forming the following functional relationship:

[0039]

[0040] pass Indicates the distance between the current trajectory point i and the previous trajectory point i-1 to capture the continuity of the gesture motion trajectory. The vector value v between the trajectory point i-1 and the trajectory point i in the formula is i-1 =(x i -x i-1 ,y i -y i-1 ), the vector value v between trajectory point i and trajectory point i+1 i =(x i+1 -x i ,y i+1 -y i ), used to represent the vector between adjacent trajectory points, The inner product of the vectors is used to calculate the cosine of the angle between adjacent trajectory vectors to measure the angle change, that is, the angle change between the current trajectory point i and the two previous and next trajectory points i-1 and i+1, where v i-1 ·v i Calculate the direction of the vector. The change in the direction of the vector can represent different gesture actions. If the direction of the two vectors changes greatly, it means that the gesture has changed a lot in space, and the state value will increase accordingly, ‖v i-1 ‖·‖v i ‖The size of the calculated vector represents the distance or movement between the trajectory points. If the distance between the two points is small, it means that the gesture movement amplitude is small, and the state value increases accordingly, which means that the gesture has changed direction within a short distance, and it is more likely to be a state transition point. α is used to adjust the degree of influence of the angle between vectors on the state value. Its adjustment can change the sensitivity of the state value to trajectory changes. For example, the switching gesture of the bent palm should be more sensitive to angle changes, while the switching gesture of the straight palm is the opposite. In this way, the state transition boundary points of different switching gestures can be quickly and accurately marked, providing a basis for subsequent analysis, thereby effectively improving the efficiency of gesture recognition. At the same time, the weight value in the formula can be adjusted according to the actual situation and applied to different gesture recognitions, thereby improving the flexibility and applicability of the algorithm.

[0041] Preferably, step S3 comprises the following steps:

[0042] Step S31: marking the eyelid center point of the eye state data to generate eye change trajectory data;

[0043] Step S32: evaluating the blink interval based on the eye change trajectory data to generate blink frequency data;

[0044] Step S33: marking the eyeball mass center of the eye state data to generate mass center motion trajectory data;

[0045] Step S34: supplementing the center of mass motion trajectory data with occlusion trajectory points according to the blink frequency data to generate center of mass motion correction data;

[0046] Step S35: performing spatiotemporal correlation trajectory fusion according to the eye change trajectory data and the center of mass motion correction data to generate fused trajectory data;

[0047] Step S36: performing a fixation time analysis based on the fused trajectory data to generate focus time distribution data;

[0048] Step S37: Predict intention based on the focus duration distribution data to generate eye intention data.

[0049] The present invention can provide eye movement trajectory information by marking the center point of the eye and tracking its movement over time. It is helpful to understand the movement pattern of the user's eyes when reading, such as scanning, skipping or concentrated reading. By evaluating the time interval between blinks, the frequency of the user's blinking can be understood. It is helpful for judging the user's concentration, fatigue level and visual fatigue state. Marking the eye center of mass and tracking its movement provides the movement path of the eye on the visual plane. It is helpful to understand the positioning, movement and jumping pattern of the user's gaze during reading. By supplementing the occlusion trajectory point, the center of mass movement trajectory data is corrected. The continuity of the eye movement trajectory can be improved and more accurate eye movement path information can be provided. Combined with the eye change trajectory data and the center of mass movement correction data, spatiotemporal correlation trajectory fusion is performed. It can integrate multiple eye movement information, provide more comprehensive and accurate eye movement trajectory data, and help understand the user's visual attention and eye movement pattern in reading. Based on the fused trajectory data, the fixation area duration analysis is performed to generate focus duration distribution data. It can reveal the length of time the user stays in different areas, which helps to understand the user's focus of attention and the degree of attention to specific content. Intention prediction is performed based on the focus duration distribution data to generate eye intention data. This can be used to infer the user's intention, such as whether the user is focusing on a certain text or picture, thereby providing more accurate predictions for automatic book switching.

[0050] Preferably, step S4 comprises the following steps:

[0051] Step S41: performing trajectory similarity evaluation on the fused trajectory data based on the gesture motion trajectory data to generate trajectory similarity value data;

[0052] Step S42: performing feature correlation analysis based on the gesture motion feature data and the focus duration distribution data to generate feature correlation data;

[0053] Step S43: stacking the predicted intention according to the gesture intention data and the eye intention data to generate comprehensive intention prediction data;

[0054] Step S44: weighting the trajectory similarity value data and the feature correlation data, and correcting the prediction result of the comprehensive intention prediction data to generate switching intention analysis data;

[0055] Step S45: Perform automatic book switching processing according to the switching intention analysis data to generate switching book data.

[0056] The present invention evaluates the fusion trajectory data through gesture motion trajectory data to generate trajectory similarity value data. It is helpful to measure the similarity between gesture and eye movement, and help identify whether the gesture movement is consistent with the eye movement. If the trajectory movement is consistent, the probability that the user expects the book to switch is greater. The feature correlation data is generated by combining the gesture motion feature data and the focus duration distribution data. It is helpful to determine the correlation between gesture and attention distribution, and can reveal the connection between gesture features and the user's focus area. By integrating gesture intention data and eye intention data, comprehensive intention prediction data is generated. A more comprehensive prediction can be provided, combining gesture and eye movement information, which helps to understand the user's intention and motivation more accurately. : By weighting the trajectory similarity value data and feature correlation data, combined with the comprehensive intention prediction data, switching intention analysis data is generated. It can comprehensively consider the trajectory similarity and feature correlation when judging the switching intention, and correct the prediction results by assigning different weights to improve the precision and accuracy of automatic switching. Automatic switching processing is performed according to the switching intention analysis data to generate switching book data. It is the core of the whole process. It uses the switching intention analysis data obtained from the previous comprehensive analysis to make specific automatic switching decisions, realize automatic switching of books, and improve user experience and reading fluency.

[0057] Preferably, step S5 comprises the following steps:

[0058] Step S51: extracting the switching time of the switching book data to generate the switching time data;

[0059] Step S52: Perform switching interval analysis based on the switching time data to generate switching frequency data; perform smoothing on the switching frequency data to generate switching habit data;

[0060] Step S53: using a habit stability evaluation formula and based on the switching frequency data, performing a habit stability evaluation on the switching habit data to generate habit stability data;

[0061] Step S54: switching the intention recognition method according to the habit stability data to generate simplified recognition strategy data;

[0062] Step S55: Soliciting user opinions based on the simplified recognition strategy data to generate recognition strategy correction data;

[0063] Step S56: Recognize the intention of switching books according to the identification strategy correction data, and automatically switch books to generate re-switching book data.

[0064] The present invention can help analyze the time required for users to switch between reading different contents by obtaining the time data of book switching. By analyzing the switching time data, the switching frequency data is obtained. Smoothing helps to eliminate possible noise or sudden changes, making the switching frequency more readable and predictable. Using the habit stability evaluation formula, the user's switching habits are evaluated based on the switching frequency data. It helps to understand whether the user's reading behavior is stable and whether there is a certain regular switching pattern. Through the habit stability data, a simplified recognition strategy is determined, which helps to simplify the decision-making process of automatic switching and improve the switching accuracy. Consulting users can understand the recognition method expected by users, so that the switching is more in line with the actual needs of users, thereby simplifying the process of book switching recognition and reducing computing power. Based on the revised recognition strategy, the book switching intention is identified and automatic switching is performed, and finally the re-switching book data is generated. It helps to improve the system's ability to accurately capture user intentions and the efficiency of executing switching.

[0065] Preferably, the habit stability evaluation formula in step S53 is as follows:

[0066]

[0067] Where W is the habit stability score, e is the base of the natural logarithm, γ is the weight parameter of the switching frequency, n is the total number of switching times, f is the frequency of the i-th switching, and t 0 is the start time of the evaluation, t 1 is the end time of the evaluation, f(t) is the change value of the switching frequency, δ is the weight parameter of the classification frequency, m is the total number of readings, M j is the average switching frequency of the jth reading, V j is the switching frequency variance value of the jth reading, and ω is the deviation correction value.

[0068] The present invention constructs a habit stability evaluation formula for evaluating the habit stability of switching habit data based on switching frequency data to generate habit stability data. The formula fully considers the base of the natural logarithm e, the weight parameter γ of the switching frequency, the total number of switching times n, the frequency f of the i-th switching i , the start time of the evaluation is t 0 , the end time of the evaluation is t 1 , switching frequency change value f(t), classification frequency weight parameter δ, total number of readings m, average switching frequency M of the jth reading j , the switching frequency variance value V of the jth reading j , the deviation correction value ω and the interaction relationship between variables constitute the following functional relationship:

[0069]

[0070] pass The sum of the inverse squares of the switching frequencies is calculated. This sum can become large if the switching frequencies are very high or some of the frequencies are very low. A large sum can reduce the overall score, suggesting an unstable frequency distribution. The role of the exponential function is to convert the inverse square value into a weighted impact on the total score. γ controls the weight of the switching frequency on the overall score. For example, when γ is small, the total score will not be affected much even if the frequency is high, and the switching frequency will not have a significant impact on the score. When γ is large, the switching frequency will have a greater impact on the total score, which means that a higher switching frequency will lead to a lower total score. It is an integral about time, which takes into account the inverse of the derivative of the absolute value of the frequency change over the time period. A smaller derivative change rate will lead to a larger integral result, which means that the frequency change is relatively small, corresponding to a higher overall score, indicating a stable frequency change. It is a weighted sum of the deviations between the switching frequency of each reading and its average switching frequency. The sum reflects the degree to which the overall frequency deviates from the average of each reading. If the overall frequency is closer to the average frequency of each reading, the sum will be smaller and the score will be relatively higher. The deviation sum is corrected by referring to the exponential function. The weight parameter δ of the classification frequency adjusts the influence of the exponential term on the overall score. A smaller δ will amplify the influence of the deviation sum. This part of the expression evaluates the overall frequency stability. A larger exponential term indicates that the overall stability is lower. In this way, we can gain in-depth insights into user behavior habits. To understand whether users tend to maintain a relatively stable switching frequency or whether there is a trend of frequent switching in a short period of time. Provide a basis for subsequent personalized services. For example, if the switching habit stability is high, you can choose to identify the book switching intention only through gestures or eye movements to improve the efficiency of recognition. The functional relationship is corrected by using the deviation correction value ω. ω can be used as a nonlinear factor to consider more parameters, or formulated based on the historical stability evaluation results to reduce the error impact caused by abnormal data and error data, thereby more accurately generating the habit stability score W, and improving the accuracy and reliability of habit stability evaluation of switching habit data. At the same time, the weights and deviation correction values ​​in the formula can be adjusted according to actual conditions and applied to the habit stability assessment of different users, thereby improving the flexibility and applicability of the algorithm.

[0071] A device for realizing automatic switching of books during point reading, comprising:

[0072] The switching gesture recognition module is used to use the camera device to capture and recognize hand movements and generate recognition success signal data; based on the recognition success signal data, hand movement tracking is performed, and eye state capture is performed to generate dynamic gesture data and eye state data.

[0073] The gesture intention analysis module is used to analyze the motion characteristics of dynamic gesture data to generate gesture motion characteristic data; perform action intention prediction based on the gesture motion characteristic data to generate gesture intention data;

[0074] The eye intention analysis module is used to analyze the eye movement trajectory of the eye state data to generate fused trajectory data; analyze the fixation area duration based on the fused trajectory data to generate focus duration distribution data; and evaluate the switching intention based on the focus duration distribution data to generate eye intention data;

[0075] The comprehensive intention evaluation module is used to use the gesture intention data and the eye intention data to predict the intention stacking and generate the switching intention analysis data; perform automatic book switching processing according to the switching intention analysis data and generate the switching book data;

[0076] The recognition strategy simplification module is used to analyze the switching habits based on the switching book data and generate the switching habit data; to simplify the recognition method based on the switching habit data and generate the simplified recognition strategy data; to perform book switching recognition based on the simplified recognition strategy data and generate the book re-switching data.

[0077] The beneficial effect of the present application is that the present invention timely captures the user's hand movements and eye states through a camera device to provide real-time data. Identify hand movements and generate recognition success signal data to provide trigger points and markers for subsequent processing. Generate dynamic gesture data and eye state data to provide a basis for subsequent analysis and understanding of user intentions. By analyzing dynamic gesture data, motion features such as trajectory, speed, acceleration, etc. are extracted to gain an in-depth understanding of the motion pattern and characteristics of gestures. Generating gesture motion feature data is an abstraction and generalization of gesture motion patterns, providing a basis for pattern recognition and intent analysis. Intention prediction is performed based on motion feature data to infer the user's gesture intention, thereby better understanding the user's behavior and intention. By analyzing the eye movement trajectory, the eye movement pattern and behavior data are obtained, and this information is integrated into fusion trajectory data to more comprehensively understand the user's observation behavior. Analyze the user's gaze duration in different areas, generate focus duration distribution data, and help identify the user's attention to a specific area. Evaluate the switching intention based on the focus duration distribution data, and generate eye intention data to understand whether the user has the intention to switch behavior. By stacking gesture intention data and eye intention data, more comprehensive and accurate switching intention analysis data is obtained, thereby better capturing the user's switching intention. Automatic switching of books is achieved based on the switching intention analysis data, and actual switching book data is generated, providing a more coherent experience between the user and the content. The user's switching frequency, habit patterns, etc. are analyzed to form switching habit data, which helps to understand the user's behavior pattern. Book switching is identified by simplifying the recognition strategy data, making the system more intelligent and adaptable, which helps to improve the user experience. Therefore, the method and device for realizing automatic switching of books in point reading of the present invention simultaneously analyzes the user's dynamic gesture data and eye state data to predict the automatic switching intention, thereby improving the accuracy of automatic switching recognition, and simplifies the recognition strategy by analyzing the user's switching habits, thereby improving the speed of automatic switching recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] Figure 1 A schematic flow chart of the steps of a method for realizing automatic switching of books in point reading;

[0079] Figure 2 for Figure 1 Detailed implementation steps of step S2 in the flowchart;

[0080] Figure 3 for Figure 1Detailed implementation steps of step S3 in FIG.

[0081] Figure 4 for Figure 1 Detailed implementation steps of step S4 in FIG.

[0082] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0083] The following is a clear and complete description of the technical method of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by technicians in this field without creative work are within the scope of protection of the present invention.

[0084] In addition, the accompanying drawings are only schematic illustrations of the present invention and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. The functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor methods and / or microcontroller methods.

[0085] It should be understood that, although the terms "first", "second", etc. may be used herein to describe various units, these units should not be limited by these terms. These terms are used only to distinguish one unit from another unit. For example, without departing from the scope of the exemplary embodiments, the first unit may be referred to as the second unit, and similarly the second unit may be referred to as the first unit. The term "and / or" used herein includes any and all combinations of one or more of the listed associated items.

[0086] To achieve this, please refer to Figures 1 to 4 A method for realizing automatic switching of books in point reading, the method comprising the following steps:

[0087] Step S1: using a camera device to perform hand motion capture and recognition, and generate recognition success signal data; based on the recognition success signal data, performing hand motion tracking and eye state capture, and generating dynamic gesture data and eye state data;

[0088] Step S2: Analyze the motion characteristics of the dynamic gesture data to generate gesture motion characteristic data; predict the action intention based on the gesture motion characteristic data to generate gesture intention data;

[0089] Step S3: performing eye movement trajectory analysis on the eye state data to generate fused trajectory data; performing gaze area duration analysis based on the fused trajectory data to generate focus duration distribution data; performing switching intention evaluation based on the focus duration distribution data to generate eye intention data;

[0090] Step S4: using the gesture intention data and the eye intention data to predict the intention stacking, and generating switching intention analysis data; performing automatic book switching processing according to the switching intention analysis data, and generating switching book data;

[0091] Step S5: Perform switching habit analysis based on the book switching data to generate switching habit data; simplify the identification method based on the switching habit data to generate simplified identification strategy data; perform book switching identification based on the simplified identification strategy data to generate book re-switching data.

[0092] The present invention captures hand movements through a camera device, provides signal data for successful recognition, generates dynamic gesture data and eye state data, and is used to identify important information such as user behavior patterns, gesture habits, and attention changes. It helps to understand the habits and interests of users during reading, and provides data support for personalized services and understanding of reading habits. The motion feature data provides an accurate description of gesture movements, including information such as speed, direction, and acceleration. It helps to accurately capture the user's gesture behavior. For example, it determines whether the user wants to turn pages or perform other operations. Such predictions provide an intelligent reading experience, allowing the device to better respond to user expectations and improve reading efficiency. By analyzing the eye movement trajectory and generating fusion trajectory data, it helps to understand the user's gaze point and visual attention when reading. To provide a more comprehensive understanding of reading habits and user attention. By analyzing the duration of the gaze area, focusing duration distribution data is generated, which helps to evaluate the user's concentration. It can be used to infer whether the user is ready to switch books. Gesture intention data and eye intention data are predicted and stacked to generate switching intention analysis data, which helps to understand the user's intention more comprehensively and improve the accuracy and intelligence of book switching. Automatic switching processing is performed according to the switching intention analysis data to generate switching book data. Such automated processing provides an intelligent reading experience, allowing the device to better respond to the user's intention. Therefore, the method and device for realizing automatic switching of books in point-and-reading of the present invention simultaneously analyzes the user's dynamic gesture data and eye state data to predict the automatic switching intention, thereby improving the accuracy of automatic switching recognition, and simplifies the recognition strategy by analyzing the user's switching habits, thereby improving the speed of automatic switching recognition.

[0093] In the embodiment of the present invention, reference Figure 1The above is a schematic flow chart of the steps of a method for realizing automatic switching of books in point reading of the present invention. In this example, the method for realizing automatic switching of books in point reading comprises the following steps:

[0094] Step S1: using a camera device to perform hand motion capture and recognition, and generate recognition success signal data; based on the recognition success signal data, performing hand motion tracking and eye state capture, and generating dynamic gesture data and eye state data;

[0095] In the embodiment of the present invention, the hand movements are captured by a camera device, and the hand movements are recognized by a specific algorithm or model, thereby generating recognition success signal data. The recognition success signal data is used to track the hand movements in real time and to capture the eye state data at the same time. Different models or algorithms are needed to process these two types of data.

[0096] Step S2: Analyze the motion characteristics of the dynamic gesture data to generate gesture motion characteristic data; predict the action intention based on the gesture motion characteristic data to generate gesture intention data;

[0097] In the embodiment of the present invention, the motion characteristics of the gesture are described by using kinematic feature analysis technology, such as trajectory analysis, velocity and acceleration calculation, angle change, etc., to generate corresponding gesture motion feature data. Based on the generated gesture motion feature data, a machine learning or deep learning model, such as a classifier or a neural network, is used for training and prediction, thereby identifying the action intention of the gesture and generating corresponding gesture intention data.

[0098] Step S3: performing eye movement trajectory analysis on the eye state data to generate fused trajectory data; performing gaze area duration analysis based on the fused trajectory data to generate focus duration distribution data; performing switching intention evaluation based on the focus duration distribution data to generate eye intention data;

[0099] In an embodiment of the present invention, the eye state data is processed by image processing technology or tracking algorithm, and the trajectory and changes of eye movement are analyzed to generate fused eye movement trajectory data. Based on the eye movement trajectory data, the user's gaze area duration is identified and analyzed, and described using indicators such as focus point and dwell time, thereby generating focus duration distribution data. Based on the focus duration distribution data, recognition technology or predefined rules are used to evaluate the user's eye intention, determine whether there is a switching intention, and finally generate eye intention data.

[0100] Step S4: using the gesture intention data and the eye intention data to predict the intention stacking, and generating switching intention analysis data; performing automatic book switching processing according to the switching intention analysis data, and generating switching book data;

[0101] In an embodiment of the present invention, by fusing gesture intention data and eye intention data, a model fusion technique (such as an integrated learning method) can be used to integrate information from these two sources. This may include model fusion, feature fusion, or a voting mechanism, etc., to ultimately generate switching intention analysis data. Based on the switching intention analysis data, an automated algorithm or decision system is used to switch books. Predefined rules, machine learning models, or logic algorithms need to be applied to automatically perform book switching based on the generated analysis data, and ultimately generate switching book data.

[0102] Step S5: Perform switching habit analysis based on the book switching data to generate switching habit data; simplify the identification method based on the switching habit data to generate simplified identification strategy data; perform book switching identification based on the simplified identification strategy data to generate book re-switching data.

[0103] In an embodiment of the present invention, by performing frequency and pattern analysis on book switching data, statistical methods, machine learning techniques, or time series analysis can be used to identify and understand the user's switching habits, generate switching habit data, and describe the rules and tendencies of the user's book switching. Based on the switching habit data, the switching stability of the switching habit data is evaluated, and the intention recognition model or method can be adjusted. For example, if the stability is high, it can be selected to perform switching intention recognition only by identifying gestures or eye movements, thereby improving recognition efficiency. Applying the adjusted recognition strategy, based on the user's switching habits and previous book switching data, book switching is achieved with higher accuracy, and book re-switching data is generated. The effectiveness of the recognition strategy can be verified by comparing the actual switching behavior with the model prediction.

[0104] Preferably, step S1 comprises the following steps:

[0105] Step S11: Acquire side palm image data; perform feature point trajectory marking on the side palm image data to generate image data to be trained;

[0106] Step S12: training a preset neural network model based on the image data to be trained to generate a side palm recognition model;

[0107] Step S13: using a camera device to capture the hand and generate hand image data;

[0108] Step S14: screening the side palm contour of the hand image data to generate side palm contour data;

[0109] Step S15: using the side palm recognition model to perform side palm recognition screening on the side palm contour data to generate recognition success signal data;

[0110] Step S16: Tracking hand movements and capturing eye states based on the recognition success signal data to generate dynamic gesture data and eye state data.

[0111] The present invention provides visual information of hand posture and form by acquiring side palm image data, which is helpful to understand the position, shape and dynamic characteristics of the hand. The side palm image data is marked with feature point trajectories to generate image data to be trained, which will serve as the basis for training a neural network model and help the model to better understand and identify hand features. The preset neural network model is trained based on the image data to be trained, so that the model can learn to identify side palm features and provide more accurate recognition and analysis capabilities. The hand is captured by a camera device to obtain real-time hand image data, which provides real-time input data for identifying and analyzing hand postures. By screening the side palm contour of the hand image data, it is helpful to extract the side palm features of the hand and reduce redundant information. The side palm contour data is screened by the side palm recognition model to generate recognition success signal data, which provides more accurate recognition of the side palm. Hand movement tracking and eye state capture are performed based on the recognition success signal data to generate dynamic gesture data and eye state data, which are used to understand important information of user behavior and visual focus and help infer the user's intention and behavior.

[0112] In an embodiment of the present invention, the side palm image data is crawled by crawler technology, and the key feature points on the side palm image are identified by computer vision technology (e.g., an algorithm based on feature point detection), such as using SIFT, SURF, or a deep learning model (e.g., a key point detection model based on a neural network) to mark the feature points. The marked feature points are saved as a data set, including the side palm image and its corresponding feature point coordinates. The image data set to be trained is divided into a training set and a validation set. According to the task requirements (e.g., convolutional neural network, ResNet, or other deep learning architectures), an appropriate neural network model is selected. The neural network model is trained using the training set data, the model parameters are optimized to reduce the prediction error, and the model performance is verified using the validation set. The model performance is evaluated using a test set that is not involved in the training, including indicators such as accuracy, recall rate, and F1 score. A camera device is used to capture hand image data. A hand capture program is run on the device to obtain hand image data in a continuous manner. The hand image is preprocessed, such as removing the background, adjusting the contrast and brightness, etc. The side palm contour in the hand image is detected and extracted using image processing technology (e.g., edge detection, contour extraction algorithm). For example, morphological transformation, edge detection algorithm (such as Canny operator) and other methods are used. The successfully extracted side palm contour is saved as a data set, including the feature point coordinates of the side palm contour or other data describing the shape of the side palm. The side palm contour data is identified and classified using a pre-trained side palm recognition model. According to the output of the recognition model, the successfully identified side palm data is marked as "recognition successful". Computer vision technology (such as key point detection-based algorithms or motion tracking algorithms) is used to track the identified hand contours or key points in order to track hand movements in real time. Eye states and movements are captured from images or videos, which may include gaze position, blinking frequency and other information. Based on the results of hand movement tracking and eye state capture, corresponding dynamic gesture data and eye state data are generated.

[0113] Preferably, step S2 comprises the following steps:

[0114] Step S21: extracting feature points from the dynamic gesture data to generate feature point motion data;

[0115] Step S22: performing feature point fitting processing according to the feature point motion data to generate gesture motion trajectory data;

[0116] Step S23: performing motion feature analysis based on the gesture motion trajectory data to generate gesture motion feature data;

[0117] Step S24: performing feature encoding processing on the gesture motion feature data to generate gesture feature vector data;

[0118] Step S25: Use the pre-trained gesture classifier to predict the action intention of the gesture feature vector data to generate gesture intention data.

[0119] The present invention extracts feature points from dynamic gesture data, provides the position and motion information of key points in the gesture, and helps to capture the key features of gesture motion. Fitting processing is performed based on the feature point motion data to generate gesture motion trajectory data. It helps to convert feature point motion into a more accurate and understandable gesture motion trajectory, and provides overall path information of gesture motion. Motion feature analysis is performed based on the gesture motion trajectory data to generate gesture motion feature data. These feature data can provide important information such as the speed, direction, acceleration, etc. of the gesture motion, which helps to understand the motion characteristics and patterns of the gesture. Feature encoding processing is performed on the gesture motion feature data to generate gesture feature vector data. This step helps to convert complex gesture motion features into more concise and easier to process feature vectors, thereby improving the operability and effectiveness of the data. The gesture feature vector data is predicted by using a pre-trained gesture classifier to generate gesture intention data. The intention of the gesture can be inferred and classified, providing effective prediction and recognition capabilities for understanding the user's gesture behavior.

[0120] As an example of the present invention, refer to Figure 2 As shown, in this example, step S2 includes:

[0121] Step S21: extracting feature points from the dynamic gesture data to generate feature point motion data;

[0122] In the embodiment of the present invention, a specific algorithm or method can be used to extract key feature points from dynamic gesture data through image processing technology. These feature points may represent the joints of the hand or special parts of the gesture, such as finger joints. The coordinate information of the extracted feature points is recorded to form feature point motion data. This includes the positions of the feature points at different time points.

[0123] Step S22: performing feature point fitting processing according to the feature point motion data to generate gesture motion trajectory data;

[0124] In the embodiment of the present invention, the feature point motion data is fitted by using mathematical or statistical methods to establish a gesture motion trajectory model. Curve fitting, polynomial fitting or other appropriate mathematical models may be used. According to the fitting model, gesture motion trajectory data is generated, which represents the motion path of the gesture, so as to better describe the motion characteristics of the gesture.

[0125] Step S23: performing motion feature analysis based on the gesture motion trajectory data to generate gesture motion feature data;

[0126] In an embodiment of the present invention, the collected gesture motion trajectory data is subjected to preprocessing steps such as noise removal, smoothing or outlier processing. The state transition boundary of the gesture motion trajectory data is marked, the change point of the gesture state is determined, and the gesture change point data is generated. The feature point motion data is discretized using the gesture change point data and divided into a plurality of trajectory segment data, which represent the specific motion state of the gesture. The curvature and angle change rate are calculated for each trajectory segment data. The curvature reflects the degree of curvature of the trajectory, and the angle change represents the change in direction of the gesture. The motion trajectory data is corrected based on the curvature correction data to correct the changes or anomalies in the trajectory and generate the corrected trajectory data. The speed change and energy change analysis of the corrected trajectory data is performed, which helps to understand the speed change and energy consumption of the gesture movement. Finally, the energy point of the trajectory correction data is marked based on the energy change data, and various analysis data are integrated to generate data describing the gesture motion characteristics.

[0127] Step S24: performing feature encoding processing on the gesture motion feature data to generate gesture feature vector data;

[0128] In an embodiment of the present invention, the most relevant features are extracted by a suitable method through gesture motion feature data. Various feature extraction techniques can be used, such as Fourier transform, wavelet transform, principal component analysis (PCA) or other feature selection algorithms. The extracted features are encoded to facilitate the use of subsequent classifiers. Encoding methods include discretizing continuous value features, normalizing them, encoding them into sparse vectors, etc. The encoded features are combined into gesture feature vector data. These vectors are the input data format required by the classification algorithm.

[0129] Step S25: Use the pre-trained gesture classifier to predict the action intention of the gesture feature vector data to generate gesture intention data.

[0130] In an embodiment of the present invention, a data set covering a variety of gestures or actions is collected. It covers various possibilities to cover various situations and changes that the system may encounter. The data is labeled and cleaned to ensure the quality of the data set. The data is labeled to indicate the type of gesture or action corresponding to each example. The features of hand position, speed, acceleration or other related information are selected to represent the gesture. The extracted features are preprocessed to ensure the validity and comparability of the features. A classification algorithm is selected, such as a support vector machine (SVM), a neural network (deep learning model), a decision tree or a random forest. The selected classification algorithm is trained using the prepared feature data and labeled data. The data set is split into a training set and a test set, the model is trained using the training set, and its performance is evaluated using the test set. The performance of the model is evaluated, and its accuracy, recall rate, precision and other indicators are examined, and compared with other possible models. Finally, a gesture classifier is generated. The generated gesture feature vector data is used as the input of the classifier. The feature vector is predicted using a pre-trained classifier to generate gesture intent data. The classifier maps the feature vector to a predefined gesture category or action intent.

[0131] Preferably, step S23 includes the following steps:

[0132] Step S231: using the boundary point calculation formula to mark the state transition boundary of the gesture motion trajectory data to generate gesture change point data;

[0133] Step S232: discretizing the feature point motion data using the gesture change point data to generate trajectory segment data;

[0134] Step S233: performing curvature calculation on the trajectory segment data to generate trajectory segment curvature data;

[0135] Step S234: performing differential direction evaluation on the trajectory segment data to generate angle change rate data; performing curvature correction on the trajectory segment curvature data using the angle change rate data to generate curvature correction data;

[0136] Step S235: using the curvature correction data to perform trajectory change correction on the motion trajectory data to generate trajectory correction data;

[0137] Step S236: performing speed change evaluation on the trajectory correction data to generate speed change data; performing energy change analysis based on the speed change data to generate energy change data;

[0138] Step S237: marking the trajectory correction data with energy points based on the energy change data to generate gesture motion feature data.

[0139] The present invention marks the state transition boundary of gesture motion trajectory data through a boundary point calculation formula to generate gesture change point data. These change points represent the state transition in the gesture motion trajectory, such as the start, end or key point position of a gesture action, which helps to understand the change process of the gesture action. The feature point motion data is discretized using the gesture change point data to generate trajectory segment data. It is helpful to divide the continuous gesture motion data into discrete trajectory segments. The curvature of the trajectory segment data is calculated to generate trajectory segment curvature data. These curvature data provide curvature information of the gesture motion trajectory, which helps to understand the curvature degree and shape characteristics of the gesture path. The curvature of the trajectory segment curvature data is corrected using the angle change rate data to generate curvature correction data. So that the curvature more accurately reflects the motion characteristics of the gesture, further improving the understanding and analysis of the gesture motion path. The motion trajectory data is corrected for trajectory changes using the curvature correction data to generate trajectory correction data. It is helpful to adjust the original trajectory data according to the curvature correction so that the trajectory is more consistent with the actual gesture motion path, thereby improving the accuracy and reliability of the data. The trajectory correction data is evaluated for speed change to generate speed change data. Provides information about the change in gesture movement speed, which helps analyze the speed change characteristics during gesture movement. Performs energy change analysis based on speed change data to generate energy change data. Energy change data represents the energy consumption of gesture movement, which helps understand the strength and degree of change of gesture movements. Based on the energy change data, energy points are marked on the trajectory correction data to generate gesture movement feature data. Describes the energy change points of gesture movement, which helps extract key features of gesture movement.

[0140] In an embodiment of the present invention, by using a boundary point calculation formula, the formula fully considers the distance between state points and the angle change amplitude, so as to quickly and accurately identify the boundary points of state transition according to the gesture motion trajectory data. And mark the state transition boundary points in the gesture motion trajectory data. These boundary points may include turning points, pause points in gesture motion, or the beginning or end of a specific action. In addition to the preset formula, different state points in the data can also be identified by segmentation analysis, machine learning and other methods. The feature point motion data is discretized using boundary point or state transition point data to divide the gesture motion trajectory into different trajectory segments. The gesture motion trajectory data is segmented according to the change point, and a label or identifier is added to each segment. The curvature of each trajectory segment data is calculated. The curvature formula in differential geometry or the numerical approximation method can be used to calculate the curvature of each trajectory segment. The calculated curvature value is associated with each trajectory segment to form trajectory segment curvature data. The angle in the trajectory segment data is calculated using the difference method or the rate of change calculation method to generate the angle change rate data. The obtained angle change rate data is used to correct the curvature data of the trajectory segment, and the curvature correction data is obtained by correcting the direction or amplitude of the original curvature data. The original motion trajectory data is corrected using the curvature correction data. This may involve adjusting the direction, shape or path of the trajectory. The speed change of the corrected trajectory data can be calculated using a differential method or a kinematic method. Energy change analysis is performed based on the speed change data, and energy changes can be evaluated through indicators such as the rate of change or amplitude of the speed change. Based on the energy change data, significant energy change points are marked in the trajectory correction data. These points may represent key actions or special turning points in gestures.

[0141] Preferably, the boundary point calculation formula in step S231 is as follows:

[0142]

[0143] In the formula, S i is the state value of trajectory point i, x i is the horizontal coordinate of trajectory point i, x i-1 is the horizontal coordinate of trajectory point i-1, y i is the ordinate of trajectory point i, y i-1 is the ordinate of trajectory point i-1, α is the weight value of angle transformation, v i-1 is the vector value between trajectory point i-1 and trajectory point i, v i is the vector value between trajectory point i and trajectory point i+1.

[0144] The present invention constructs a boundary point calculation formula for marking the state transition boundary of gesture motion trajectory data and generating gesture change point data. The formula fully considers the horizontal coordinate x of trajectory point i. i, the horizontal coordinate x of trajectory point i-1 i-1 , the ordinate y of trajectory point i i , the ordinate y of trajectory point i-1 i-1 , the vector value v between trajectory point i-1 and trajectory point i i-1 , the vector value v between trajectory point i and trajectory point i+1 i And the interaction relationship between variables, forming the following functional relationship:

[0145]

[0146] pass Indicates the distance between the current trajectory point i and the previous trajectory point i-1 to capture the continuity of the gesture motion trajectory. The vector value v between the trajectory point i-1 and the trajectory point i in the formula is i-1 =(x i -x i-1 ,y i -y i-1 ), the vector value vi between trajectory point i and trajectory point i+1 = (x i+1 -xi,y i+1 -y i ), used to represent the vector between adjacent trajectory points, The inner product of the vectors is used to calculate the cosine of the angle between adjacent trajectory vectors to measure the angle change, that is, the angle change between the current trajectory point i and the two previous and next trajectory points i-1 and i+1, where v i-1 ·v i Calculate the direction of the vector. The change in the direction of the vector can represent different gesture actions. If the direction of the two vectors changes greatly, it means that the gesture has changed a lot in space, and the state value will increase accordingly, ‖v i-1 ‖·‖v i ‖The size of the calculated vector represents the distance or movement between the trajectory points. If the distance between the two points is small, it means that the gesture movement amplitude is small, and the state value increases accordingly, which means that the gesture has changed direction within a short distance, and it is more likely to be a state transition point. α is used to adjust the degree of influence of the angle between vectors on the state value. Its adjustment can change the sensitivity of the state value to trajectory changes. For example, the switching gesture of the bent palm should be more sensitive to angle changes, while the switching gesture of the straight palm is the opposite. In this way, the state transition boundary points of different switching gestures can be quickly and accurately marked, providing a basis for subsequent analysis, thereby effectively improving the efficiency of gesture recognition. At the same time, the weight value in the formula can be adjusted according to the actual situation and applied to different gesture recognitions, thereby improving the flexibility and applicability of the algorithm.

[0147] Preferably, step S3 comprises the following steps:

[0148] Step S31: marking the eyelid center point of the eye state data to generate eye change trajectory data;

[0149] Step S32: evaluating the blink interval based on the eye change trajectory data to generate blink frequency data;

[0150] Step S33: marking the eyeball mass center of the eye state data to generate mass center motion trajectory data;

[0151] Step S34: supplementing the center of mass motion trajectory data with occlusion trajectory points according to the blink frequency data to generate center of mass motion correction data;

[0152] Step S35: performing spatiotemporal correlation trajectory fusion according to the eye change trajectory data and the center of mass motion correction data to generate fused trajectory data;

[0153] Step S36: performing a fixation time analysis based on the fused trajectory data to generate focus time distribution data;

[0154] Step S37: Predict intention based on the focus duration distribution data to generate eye intention data.

[0155] The present invention can provide eye movement trajectory information by marking the center point of the eye and tracking its movement over time. It is helpful to understand the movement pattern of the user's eyes when reading, such as scanning, skipping or concentrated reading. By evaluating the time interval between blinks, the frequency of the user's blinking can be understood. It is helpful for judging the user's concentration, fatigue level and visual fatigue state. Marking the eye center of mass and tracking its movement provides the movement path of the eye on the visual plane. It is helpful to understand the positioning, movement and jumping pattern of the user's gaze during reading. By supplementing the occlusion trajectory point, the center of mass movement trajectory data is corrected. The continuity of the eye movement trajectory can be improved and more accurate eye movement path information can be provided. Combined with the eye change trajectory data and the center of mass movement correction data, spatiotemporal correlation trajectory fusion is performed. It can integrate multiple eye movement information, provide more comprehensive and accurate eye movement trajectory data, and help understand the user's visual attention and eye movement pattern in reading. Based on the fused trajectory data, the fixation area duration analysis is performed to generate focus duration distribution data. It can reveal the length of time the user stays in different areas, which helps to understand the user's focus of attention and the degree of attention to specific content. Intention prediction is performed based on the focus duration distribution data to generate eye intention data. This can be used to infer the user's intention, such as whether the user is focusing on a certain text or picture, thereby providing more accurate predictions for automatic book switching.

[0156] As an example of the present invention, refer to Figure 3 As shown, in this example, step S3 includes:

[0157] Step S31: marking the eyelid center point of the eye state data to generate eye change trajectory data;

[0158] In the embodiment of the present invention, the eyelid center point is marked by using image processing or computer vision methods, for example, using an algorithm based on feature point detection (such as a facial recognition algorithm based on feature points). The detection can be performed through steps such as eye edge detection and contour analysis. Based on the marking of the eyelid center point, the change in eyelid position or movement is calculated. The trajectory of the eyelid center point is tracked between consecutive frames to form eye change trajectory data.

[0159] Step S32: evaluating the blink interval based on the eye change trajectory data to generate blink frequency data;

[0160] In the embodiment of the present invention, the characteristic pattern of eyelid movement is identified through the eye change trajectory data to determine the blinking action. A blink detection algorithm or pattern matching method based on time series can be used. After the blink detection, the time interval between the blink actions is calculated according to the time difference between adjacent blinks or the frequency of the blink action to obtain the blink frequency.

[0161] Step S33: marking the eyeball mass center of the eye state data to generate mass center motion trajectory data;

[0162] In the embodiment of the present invention, the center of mass position of the eyeball is identified by using the eye state data and image processing or computer vision technology. The detection can be performed by using the pupil position or the eyeball contour. According to the detection result of the center of mass of the eyeball, the change of the center of mass position over time is tracked to form the center of mass motion trajectory data.

[0163] Step S34: supplementing the center of mass motion trajectory data with occlusion trajectory points according to the blink frequency data to generate center of mass motion correction data;

[0164] In the embodiment of the present invention, the occlusion or interference part caused by the blinking action in the eye state data is detected by using the blinking frequency data. For the occlusion caused by the blinking action, the affected part in the centroid motion trajectory data is corrected or interpolated to fill the occlusion part, so as to obtain more complete and continuous centroid motion correction data.

[0165] Step S35: performing spatiotemporal correlation trajectory fusion according to the eye change trajectory data and the center of mass motion correction data to generate fused trajectory data;

[0166] In an embodiment of the present invention, the timestamps or time periods of the eye change trajectory data and the center of mass motion correction data are determined for matching and fusion. The eye change trajectory data and the center of mass motion correction data are temporally and spatially associated. The data can be fused by synchronizing the timestamps of the two data or using an interpolation method. The associated eye change trajectory data and the center of mass motion correction data are fused, and the fused trajectory data may be obtained by data superposition, weighted averaging or other suitable methods.

[0167] Step S36: performing a fixation time analysis based on the fused trajectory data to generate focus time distribution data;

[0168] In the embodiment of the present invention, different areas of user gaze are identified or divided by using the fused trajectory data. The gaze area can be identified by spatial cluster analysis. The duration of stay in different gaze areas is counted and analyzed, and the observation duration or frequency of different areas is recorded.

[0169] Step S37: Predict intention based on the focus duration distribution data to generate eye intention data.

[0170] In an embodiment of the present invention, features such as observation duration (gaze duration) and frequency (number of occurrences or duration of each gaze area) are extracted by collecting eye gaze data and extracting markers therefrom. Information such as the spatial distribution of different gaze areas and the transfer pattern between gaze points may also be included. The labeled data is used as a training data set, which contains eye feature data and corresponding intention labels. The extracted features are processed, such as standardization, normalization or other preprocessing operations, to facilitate model training and prediction. An appropriate machine learning model is selected for training, such as using support vector machine (SVM), neural network, decision tree and other methods. The model is trained using the training set, and the model performance is evaluated and hyperparameters are adjusted using the validation set to generate a pre-trained eye intention prediction model. The prediction results of the model can be multi-category classification (multiple eye intentions) or binary classification (whether a specific intention exists). Features (such as observation duration, frequency, etc.) in the focus duration distribution data are extracted as input features. The extracted features are applied using the pre-trained eye intention prediction model to predict eye intention and generate eye intention data.

[0171] Preferably, step S4 comprises the following steps:

[0172] Step S41: performing trajectory similarity evaluation on the fused trajectory data based on the gesture motion trajectory data to generate trajectory similarity value data;

[0173] Step S42: performing feature correlation analysis based on the gesture motion feature data and the focus duration distribution data to generate feature correlation data;

[0174] Step S43: stacking the predicted intention according to the gesture intention data and the eye intention data to generate comprehensive intention prediction data;

[0175] Step S44: weighting the trajectory similarity value data and the feature correlation data, and correcting the prediction result of the comprehensive intention prediction data to generate switching intention analysis data;

[0176] Step S45: Perform automatic book switching processing according to the switching intention analysis data to generate switching book data.

[0177] The present invention evaluates the fusion trajectory data through gesture motion trajectory data to generate trajectory similarity value data. It is helpful to measure the similarity between gesture and eye movement, and help identify whether the gesture movement is consistent with the eye movement. If the trajectory movement is consistent, the probability that the user expects the book to switch is greater. The feature correlation data is generated by combining the gesture motion feature data and the focus duration distribution data. It is helpful to determine the correlation between gesture and attention distribution, and can reveal the connection between gesture features and the user's focus area. By integrating gesture intention data and eye intention data, comprehensive intention prediction data is generated. A more comprehensive prediction can be provided, combining gesture and eye movement information, which helps to understand the user's intention and motivation more accurately. : By weighting the trajectory similarity value data and feature correlation data, combined with the comprehensive intention prediction data, switching intention analysis data is generated. It can comprehensively consider the trajectory similarity and feature correlation when judging the switching intention, and correct the prediction results by assigning different weights to improve the precision and accuracy of automatic switching. Automatic switching processing is performed according to the switching intention analysis data to generate switching book data. It is the core of the whole process. It uses the switching intention analysis data obtained from the previous comprehensive analysis to make specific automatic switching decisions, realize automatic switching of books, and improve user experience and reading fluency.

[0178] As an example of the present invention, refer to Figure 4 As shown, in this example, step S4 includes:

[0179] Step S41: performing trajectory similarity evaluation on the fused trajectory data based on the gesture motion trajectory data to generate trajectory similarity value data;

[0180] In the embodiment of the present invention, key features such as motion direction, trajectory length, etc. are extracted from the gesture motion trajectory data. A suitable similarity measurement method (such as Euclidean distance, cosine similarity, dynamic time warping (DTW), etc.) is used to calculate the similarity between different trajectories. The gesture motion trajectory data is compared with the fused trajectory data, and the similarity score between them is calculated to generate trajectory similarity value data.

[0181] Step S42: performing feature correlation analysis based on the gesture motion feature data and the focus duration distribution data to generate feature correlation data;

[0182] In the embodiment of the present invention, the direction, speed, acceleration and other features are selected from the gesture motion feature data and the focus duration distribution data for analysis. Statistical methods (such as Pearson correlation coefficient, Spearman correlation coefficient) or machine learning methods (such as feature importance evaluation, decision tree, etc.) are used to evaluate the correlation between different features. The correlation score between the features is calculated, which indicates the degree of correlation between them, and the feature correlation data is generated.

[0183] Step S43: stacking the predicted intention according to the gesture intention data and the eye intention data to generate comprehensive intention prediction data;

[0184] In an embodiment of the present invention, the gesture intention data and the eye intention data are integrated into a comprehensive data set. A suitable stacking model (such as a stacking model in an integrated learning method, or a deep learning model) is used to train the selected stacking model using the gesture intention prediction data and the eye intention prediction data obtained by the training model. The trained model is used to predict new data to obtain comprehensive intention prediction data.

[0185] Step S44: weighting the trajectory similarity value data and the feature correlation data, and correcting the prediction result of the comprehensive intention prediction data to generate switching intention analysis data;

[0186] In an embodiment of the present invention, the trajectory similarity value data and the feature correlation data are integrated and appropriate weights are assigned to them. Weight assignment can be performed by using a weight assignment algorithm or model such as weighted average, logistic regression, etc. The weighted data is used to correct the comprehensive intention prediction data. For example, by correcting the confidence of the predicted intention or adjusting the prediction result, a more accurate intention prediction can be obtained.

[0187] Step S45: Perform automatic book switching processing according to the switching intention analysis data to generate switching book data.

[0188] In the embodiment of the present invention, the intention of the book to be switched is analyzed based on the corrected prediction data. The automatic book switching logic is designed and implemented. This may include defining switching rules, specifying switching priorities, triggering conditions for switching operations, etc. Based on the intention analysis results and the automatic switching logic, the operation of automatically switching books is implemented.

[0189] Preferably, step S5 comprises the following steps:

[0190] Step S51: extracting the switching time of the switching book data to generate the switching time data;

[0191] Step S52: Perform switching interval analysis based on the switching time data to generate switching frequency data; perform smoothing on the switching frequency data to generate switching habit data;

[0192] Step S53: using a habit stability evaluation formula and based on the switching frequency data, performing a habit stability evaluation on the switching habit data to generate habit stability data;

[0193] Step S54: switching the intention recognition method according to the habit stability data to generate simplified recognition strategy data;

[0194] Step S55: Soliciting user opinions based on the simplified recognition strategy data to generate recognition strategy correction data;

[0195] Step S56: Recognize the intention of switching books according to the identification strategy correction data, and automatically switch books to generate re-switching book data.

[0196] The present invention can help analyze the time required for users to switch between reading different contents by obtaining the time data of book switching. By analyzing the switching time data, the switching frequency data is obtained. Smoothing helps to eliminate possible noise or sudden changes, making the switching frequency more readable and predictable. Using the habit stability evaluation formula, the user's switching habits are evaluated based on the switching frequency data. It helps to understand whether the user's reading behavior is stable and whether there is a certain regular switching pattern. Through the habit stability data, a simplified recognition strategy is determined, which helps to simplify the decision-making process of automatic switching and improve the switching accuracy. Consulting users can understand the recognition method expected by users, so that the switching is more in line with the actual needs of users, thereby simplifying the process of book switching recognition and reducing computing power. Based on the revised recognition strategy, the book switching intention is identified and automatic switching is performed, and finally the re-switching book data is generated. It helps to improve the system's ability to accurately capture user intentions and the efficiency of executing switching.

[0197] In an embodiment of the present invention, by switching book data, the time point of each book switching is marked to obtain switching time data. The interval between adjacent switching time points is calculated according to the switching time data. The switching interval data is used to calculate the frequency of book switching, that is, the number of switches occurring per unit time. The switching frequency data is smoothed, and a sliding average or other smoothing algorithm can be used to reduce the noise in the data. Based on the smoothed switching frequency data, the habit data of book switching is obtained. This may include the mean and variance of the switching frequency. A given habit stability evaluation formula is used, which fully considers the weight parameters of the switching frequency, the total number of switching times, the switching frequency and other parameters to efficiently and accurately calculate the habit stability score, which is used to determine whether the user's switching habits tend to be stable. The evaluation formula is applied to calculate the stability score of the switching habit and obtain the habit stability data. Define a specific switching standard, for example, based on a preset score threshold, when the habit stability score is greater than the preset score threshold, it is determined that the book switching intention can be predicted only by gesture or eye dynamic data, and simplified recognition strategy data is generated, thereby reducing computing power and improving recognition efficiency. Feedback the simplified recognition strategy data to the user end and consult the user's opinions. For example, some users may prefer to use hand movements for recognition, while some users may prefer to use eye movements for recognition, etc., to collect the user's inclination and opinions on the simplified recognition strategy data. According to the user's opinions, make corresponding adjustments or corrections to the recognition strategy. This can be to adjust the algorithm parameters, simplify the switching criteria, etc., to generate the recognition strategy correction data. Use the recognition strategy-corrected data to identify the book switching intention. According to the identified switching intention, trigger the operation of automatically switching books.

[0198] Preferably, the habit stability evaluation formula in step S53 is as follows:

[0199]

[0200] Where W is the habit stability score, e is the base of the natural logarithm, γ is the weight parameter of the switching frequency, n is the total number of switching times, and f i is the frequency of the ith switching, t 0 is the start time of the evaluation, t 1 is the end time of the evaluation, f(t) is the change value of the switching frequency, δ is the weight parameter of the classification frequency, m is the total number of readings, M j is the average switching frequency of the jth reading, V j is the switching frequency variance value of the jth reading, and ω is the deviation correction value.

[0201] The present invention constructs a habit stability evaluation formula for evaluating the habit stability of switching habit data based on switching frequency data to generate habit stability data. The formula fully considers the base of the natural logarithm e, the weight parameter γ of the switching frequency, the total number of switching times n, the frequency f of the i-th switching i , the start time of the evaluation is t 0 , the end time of the evaluation is t 1 , switching frequency change value f(t), classification frequency weight parameter δ, total number of readings m, average switching frequency M of the jth reading j , the switching frequency variance value V of the jth reading j , the deviation correction value ω and the interaction relationship between variables constitute the following functional relationship:

[0202]

[0203] pass The sum of the inverse squares of the switching frequencies is calculated. This sum can become large if the switching frequencies are very high or some of the frequencies are very low. A large sum can reduce the overall score, suggesting an unstable frequency distribution. The role of the exponential function is to convert the inverse square value into a weighted impact on the total score. γ controls the weight of the switching frequency on the overall score. For example, when γ is small, the total score will not be affected much even if the frequency is high, and the switching frequency will not have a significant impact on the score. When γ is large, the switching frequency will have a greater impact on the total score, which means that a higher switching frequency will lead to a lower total score. It is an integral about time, which takes into account the inverse of the derivative of the absolute value of the frequency change over the time period. A smaller derivative change rate will lead to a larger integral result, which means that the frequency change is relatively small, corresponding to a higher overall score, indicating a stable frequency change. It is a weighted sum of the deviations between the switching frequency of each reading and its average switching frequency. The sum reflects the degree to which the overall frequency deviates from the average of each reading. If the overall frequency is closer to the average frequency of each reading, the sum will be smaller and the score will be relatively higher. The deviation sum is corrected by referring to the exponential function. The weight parameter δ of the classification frequency adjusts the influence of the exponential term on the overall score. A smaller δ will amplify the influence of the deviation sum. This part of the expression evaluates the overall frequency stability. A larger exponential term indicates that the overall stability is lower. In this way, we can gain in-depth insights into user behavior habits. To understand whether users tend to maintain a relatively stable switching frequency or whether there is a trend of frequent switching in a short period of time. Provide a basis for subsequent personalized services. For example, if the switching habit stability is high, you can choose to identify the book switching intention only through gestures or eye movements to improve the efficiency of recognition. The functional relationship is corrected by using the deviation correction value ω. ω can be used as a nonlinear factor to consider more parameters, or formulated based on the historical stability evaluation results to reduce the error impact caused by abnormal data and error data, thereby more accurately generating the habit stability score W, and improving the accuracy and reliability of habit stability evaluation of switching habit data. At the same time, the weights and deviation correction values ​​in the formula can be adjusted according to actual conditions and applied to the habit stability assessment of different users, thereby improving the flexibility and applicability of the algorithm.

[0204] A device for realizing automatic switching of books during point reading, comprising:

[0205] The switching gesture recognition module is used to use the camera device to capture and recognize hand movements and generate recognition success signal data; based on the recognition success signal data, hand movement tracking is performed, and eye state capture is performed to generate dynamic gesture data and eye state data.

[0206] The gesture intention analysis module is used to analyze the motion characteristics of dynamic gesture data to generate gesture motion characteristic data; perform action intention prediction based on the gesture motion characteristic data to generate gesture intention data;

[0207] The eye intention analysis module is used to analyze the eye movement trajectory of the eye state data to generate fused trajectory data; analyze the fixation area duration based on the fused trajectory data to generate focus duration distribution data; and evaluate the switching intention based on the focus duration distribution data to generate eye intention data;

[0208] The comprehensive intention evaluation module is used to use the gesture intention data and the eye intention data to predict the intention stacking and generate the switching intention analysis data; perform automatic book switching processing according to the switching intention analysis data and generate the switching book data;

[0209] The recognition strategy simplification module is used to analyze the switching habits based on the switching book data and generate the switching habit data; to simplify the recognition method based on the switching habit data and generate the simplified recognition strategy data; to perform book switching recognition based on the simplified recognition strategy data and generate the book re-switching data.

[0210] The beneficial effect of the present application is that the present invention timely captures the user's hand movements and eye states through a camera device to provide real-time data. Identify hand movements and generate recognition success signal data to provide trigger points and markers for subsequent processing. Generate dynamic gesture data and eye state data to provide a basis for subsequent analysis and understanding of user intentions. By analyzing dynamic gesture data, motion features such as trajectory, speed, acceleration, etc. are extracted to gain an in-depth understanding of the motion pattern and characteristics of gestures. Generating gesture motion feature data is an abstraction and generalization of gesture motion patterns, providing a basis for pattern recognition and intent analysis. Intention prediction is performed based on motion feature data to infer the user's gesture intention, thereby better understanding the user's behavior and intention. By analyzing the eye movement trajectory, the eye movement pattern and behavior data are obtained, and this information is integrated into fusion trajectory data to more comprehensively understand the user's observation behavior. Analyze the user's gaze duration in different areas, generate focus duration distribution data, and help identify the user's attention to a specific area. Evaluate the switching intention based on the focus duration distribution data, and generate eye intention data to understand whether the user has the intention to switch behavior. By stacking gesture intention data and eye intention data, more comprehensive and accurate switching intention analysis data is obtained, thereby better capturing the user's switching intention. Automatic switching of books is achieved based on the switching intention analysis data, and actual switching book data is generated, providing a more coherent experience between the user and the content. The user's switching frequency, habit patterns, etc. are analyzed to form switching habit data, which helps to understand the user's behavior pattern. Book switching is identified by simplifying the recognition strategy data, making the system more intelligent and adaptable, which helps to improve the user experience. Therefore, the method and device for realizing automatic switching of books in point reading of the present invention simultaneously analyzes the user's dynamic gesture data and eye state data to predict the automatic switching intention, thereby improving the accuracy of automatic switching recognition, and simplifies the recognition strategy by analyzing the user's switching habits, thereby improving the speed of automatic switching recognition.

[0211] Therefore, the embodiments should be regarded as illustrative and non-restrictive from all points, and the scope of the present invention is limited by the appended claims rather than the above description, and it is therefore intended that all changes falling within the meaning and range of equivalent elements of the application documents are included in the present invention.

[0212] The above description is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but should conform to the widest scope consistent with the principles and novel features invented herein.

Claims

1. A method for realizing automatic switching of books in point reading, characterized in that: The following steps are involved: Step S1: using a camera device to perform hand motion capture and recognition, and generate recognition success signal data; based on the recognition success signal data, performing hand motion tracking and eye state capture, and generating dynamic gesture data and eye state data; Step S2: Analyze the motion characteristics of the dynamic gesture data to generate gesture motion characteristic data; predict the action intention based on the gesture motion characteristic data to generate gesture intention data. Step S2 is specifically as follows: Step S21: extracting feature points from the dynamic gesture data to generate feature point motion data; Step S22: performing feature point fitting processing according to the feature point motion data to generate gesture motion trajectory data; Step S23: Perform motion feature analysis based on the gesture motion trajectory data to generate gesture motion feature data. Step S23 is specifically as follows: Step S231: using the boundary point calculation formula to mark the state transition boundary of the gesture motion trajectory data to generate gesture change point data; Step S232: discretizing the feature point motion data using the gesture change point data to generate trajectory segment data; Step S233: performing curvature calculation on the trajectory segment data to generate trajectory segment curvature data; Step S234: performing differential direction evaluation on the trajectory segment data to generate angle change rate data; Performing curvature correction on the trajectory segment curvature data using the angle change rate data to generate curvature correction data; Step S235: using the curvature correction data to perform trajectory change correction on the motion trajectory data to generate trajectory correction data; Step S236: performing speed change evaluation on the trajectory correction data to generate speed change data; Perform energy change analysis based on speed change data to generate energy change data; Step S237: marking the trajectory correction data with energy points based on the energy change data to generate gesture motion feature data; Step S24: performing feature encoding processing on the gesture motion feature data to generate gesture feature vector data; Step S25: using a pre-trained gesture classifier to predict the gesture feature vector data for action intention, and generating gesture intention data; Step S3: performing eye movement trajectory analysis on the eye state data to generate fused trajectory data; performing gaze area duration analysis based on the fused trajectory data to generate focus duration distribution data; Switching intention is evaluated based on the focus duration distribution data to generate eye intention data; Step S4: using the gesture intention data and the eye intention data to stack the predicted intentions and generate switching intention analysis data; Automatically switch books based on the switching intention analysis data to generate switching book data; Step S5: Perform switching habit analysis based on the switching book data to generate switching habit data; Simplify the recognition method based on the switching habit data and generate simplified recognition strategy data; Book switching identification is performed based on simplified identification strategy data to generate book switching data.

2. The method for realizing automatic switching of books in point reading according to claim 1, characterized in that: Step S1 includes the following steps: Step S11: Acquire side palm image data; perform feature point trajectory marking on the side palm image data to generate image data to be trained; Step S12: training a preset neural network model based on the image data to be trained to generate a side palm recognition model; Step S13: using a camera device to capture the hand and generate hand image data; Step S14: screening the side palm contour of the hand image data to generate side palm contour data; Step S15: using the side palm recognition model to perform side palm recognition screening on the side palm contour data to generate recognition success signal data; Step S16: Tracking hand movements and capturing eye states based on the recognition success signal data to generate dynamic gesture data and eye state data.

3. The method for realizing automatic switching of books in point reading according to claim 1, characterized in that: The boundary point calculation formula in step S231 is as follows: ； In the formula, For trajectory points The status value of For trajectory points The horizontal axis of For trajectory points The horizontal axis of For trajectory points The vertical coordinate of For trajectory points The vertical coordinate of is the weight value of the angle transformation, For trajectory points With trajectory points The vector value of For trajectory points With trajectory points The vector value of .

4. The method for realizing automatic switching of books in point reading according to claim 1, characterized in that: Step S3 includes the following steps: Step S31: marking the eyelid center point of the eye state data to generate eye change trajectory data; Step S32: evaluating the blink interval based on the eye change trajectory data to generate blink frequency data; Step S33: marking the eyeball mass center of the eye state data to generate mass center motion trajectory data; Step S34: supplementing the center of mass motion trajectory data with occlusion trajectory points according to the blink frequency data to generate center of mass motion correction data; Step S35: performing spatiotemporal correlation trajectory fusion according to the eye change trajectory data and the center of mass motion correction data to generate fused trajectory data; Step S36: performing a fixation time analysis based on the fused trajectory data to generate focus time distribution data; Step S37: Predict intention based on the focus duration distribution data to generate eye intention data.

5. The method for realizing automatic switching of books in point reading according to claim 1, characterized in that: Step S4 includes the following steps: Step S41: performing trajectory similarity evaluation on the fused trajectory data based on the gesture motion trajectory data to generate trajectory similarity value data; Step S42: performing feature correlation analysis based on the gesture motion feature data and the focus duration distribution data to generate feature correlation data; Step S43: stacking the predicted intention according to the gesture intention data and the eye intention data to generate comprehensive intention prediction data; Step S44: weighting the trajectory similarity value data and the feature correlation data, and correcting the prediction result of the comprehensive intention prediction data to generate switching intention analysis data; Step S45: Perform automatic book switching processing according to the switching intention analysis data to generate switching book data.

6. The method for realizing automatic switching of books in point reading according to claim 1, characterized in that: Step S5 includes the following steps: Step S51: extracting the switching time of the switching book data to generate the switching time data; Step S52: Perform switching interval analysis based on the switching time data to generate switching frequency data; perform smoothing on the switching frequency data to generate switching habit data; Step S53: using a habit stability evaluation formula and based on the switching frequency data, performing a habit stability evaluation on the switching habit data to generate habit stability data; Step S54: switching the intention recognition method according to the habit stability data to generate simplified recognition strategy data; Step S55: Soliciting user opinions based on the simplified recognition strategy data to generate recognition strategy correction data; Step S56: Recognize the intention of switching books according to the identification strategy correction data, and automatically switch books to generate re-switching book data.

7. A device for realizing automatic switching of books during point reading, characterized in that: The method for realizing automatic switching of books in point reading as claimed in claim 1, wherein the device for realizing automatic switching of books in point reading comprises: Switching the gesture recognition module is used to use the camera device to capture and recognize hand movements and generate recognition success signal data; based on the recognition success signal data, hand movement tracking is performed, and eye state capture is performed to generate dynamic gesture data and eye state data; The gesture intention analysis module is used to analyze the motion characteristics of dynamic gesture data to generate gesture motion characteristic data; perform action intention prediction based on the gesture motion characteristic data to generate gesture intention data; The eye intention analysis module is used to analyze the eye movement trajectory of the eye state data to generate fused trajectory data; analyze the fixation area duration based on the fused trajectory data to generate focus duration distribution data; and evaluate the switching intention based on the focus duration distribution data to generate eye intention data; The comprehensive intention evaluation module is used to use the gesture intention data and the eye intention data to predict the intention stacking and generate the switching intention analysis data; perform automatic book switching processing according to the switching intention analysis data and generate the switching book data; The recognition strategy simplification module is used to analyze the switching habits based on the switching book data and generate the switching habit data; to simplify the recognition method based on the switching habit data and generate the simplified recognition strategy data; to perform book switching recognition based on the simplified recognition strategy data and generate the book re-switching data.

Images