Dot Matrix Pen Stroke Loss Reason Detection Method and System Based on Lightweight Convolutional Network

Abstract

This application relates to the technical field of detecting the reasons for missing strokes, and particularly to a method and system for detecting the reasons for missing strokes of a dot matrix pen based on a lightweight convolutional network. The method includes obtaining real-time writing data and importing the real-time writing data into a trained lightweight convolutional network model for feature recognition to obtain real-time writing features, where the real-time writing features include a movement trajectory, hand pressure, and a timestamp; performing visualization processing on the real-time writing features to obtain a visualized movement trajectory map corresponding to the real-time writing features, and the visualized movement trajectory map includes the movement trajectory of the dot matrix pen during writing and the hand pressure at each writing moment; determining the target reason for missing strokes based on the visualized movement trajectory map and feeding back the target reason for missing strokes. This application facilitates improving the timeliness and accuracy in determining the reasons for missing strokes.

Description

Technical Field

[0001] The present application relates to the technical field of detecting the reasons for missing strokes of a dot matrix pen, and in particular to a method and system for detecting the reasons for missing strokes of a dot matrix pen based on a lightweight convolutional network. Background Art

[0002] With the continuous development of technology, digital transformation has become an irresistible trend in all industries. As a type of intelligent learning tool, the dot matrix pen meets the requirements of digital transformation and can help users better adapt to the digital working environment and learning environment. In addition, compared with traditional pen-and-paper writing, the dot matrix pen can capture handwriting more accurately, making the writing smoother and more natural. Therefore, in recent years, the dot matrix pen has been favored by more and more people in work and study.

[0003] However, despite the many advantages of the dot matrix pen, there is still a certain proportion of missing stroke problems in its use. If the reasons for missing strokes cannot be quickly and accurately located, it will not only interfere with the normal use of the dot matrix pen and lead to incomplete data records, but may also reduce the user's satisfaction due to the decrease in writing efficiency. Summary of the Invention

[0004] In order to improve the timeliness and accuracy in determining the reasons for missing strokes, the present application provides a method and system for detecting the reasons for missing strokes of a dot matrix pen based on a lightweight convolutional network.

[0005] In the first aspect, the present application provides a method for detecting the reasons for missing strokes of a dot matrix pen based on a lightweight convolutional network, adopting the following technical solution:

[0006] A method for detecting the reasons for missing strokes of a dot matrix pen based on a lightweight convolutional network includes:

[0007] Obtain real-time writing data, and import the real-time writing data into a trained lightweight convolutional network model for feature recognition to obtain real-time writing features, where the real-time writing features include a motion trajectory, hand grip pressure, and a timestamp;

[0008] Perform visualization processing on the real-time writing features to obtain a visualized motion trajectory map corresponding to the real-time writing features, where the visualized motion trajectory map includes the motion trajectory of the dot matrix pen during writing and the hand grip pressure at each writing moment;

[0009] Determine the target reason for missing strokes based on the visualized motion trajectory map, and feedback the target reason for missing strokes.

[0010] By adopting the above technical solutions, the trained lightweight convolutional network model is used to identify the real-time writing features contained in the real-time writing data, which is convenient for improving the accuracy when determining the real-time writing features. In addition, since the visual motion trajectory diagram can intuitively display the dynamic changes during the writing process, therefore, by visualizing the real-time writing features, it is convenient to more accurately judge the reasons for missing strokes. Through the automated data processing and writing feature recognition process, the possibility of human intervention is reduced, thus facilitating the reduction of misjudgment and delay caused by human factors. Moreover, through the instant feedback mechanism, it is convenient for the user to understand the reasons for missing strokes in the first time and take corresponding corrective measures, thereby facilitating the improvement of the timeliness and accuracy of the entire detection process.

[0011] In a possible implementation manner, determining the target reason for missing strokes based on the visual motion trajectory diagram includes:

[0012] Identifying abnormal writing features and the corresponding real-time abnormal feature values for each abnormal writing feature from the visual motion trajectory diagram;

[0013] Based on all abnormal writing features and the corresponding real-time abnormal feature values for each abnormal writing feature, determining the proportion of real-time abnormal features;

[0014] Based on the proportion of real-time abnormal features and the preset mapping relationship for missing strokes, determining the target reason for missing strokes corresponding to the proportion of real-time abnormal features, where the preset mapping relationship for missing strokes is the corresponding relationship between the proportion of real-time abnormal features and the target reason for missing strokes.

[0015] By adopting the above technical solutions, by identifying each abnormal writing feature and the corresponding abnormal feature value for each abnormal writing feature from the visual motion trajectory diagram, that is, by quantifying the abnormal writing features, it is convenient for relevant personnel to intuitively view the existing abnormal writing situations and abnormal degrees during the writing process. In addition, by analyzing the proportion between multiple abnormal writing features and then determining the corresponding target reason for missing strokes based on the proportion of abnormal features, that is, by considering abnormal writing features from multiple dimensions to determine the target reason for missing strokes, it is convenient to improve the accuracy and reliability when determining the target reason for missing strokes.

[0016] In a possible implementation manner, when the number of abnormalities of abnormal writing features in the visual motion trajectory diagram is higher than the preset abnormality threshold, the method further includes:

[0017] Based on each abnormal writing feature and the corresponding real-time abnormal feature value for each abnormal writing feature, determining a real-time change display diagram of abnormal features, where the real-time change display diagram of abnormal features includes change bars of abnormal feature values corresponding to each abnormal writing feature;

[0018] Determine multiple feature combinations based on each abnormal writing feature in the real-time change display graph, and determine the predicted reasons for missing strokes according to the real-time abnormal feature values corresponding to each abnormal writing feature in each feature combination;

[0019] Record the determination time of each predicted reason for missing strokes, and sort all the predicted reasons for missing strokes based on each determination time to form a list of reasons for missing strokes.

[0020] By adopting the above technical solution, by integrating each abnormal writing feature and its corresponding real-time abnormal feature value into the real-time change display graph of abnormal features, it is convenient to monitor the abnormal conditions in the writing process in a timely manner. In addition, combining each abnormal writing feature included in the real-time change display graph is convenient for considering the interaction between different abnormal writing features, thereby facilitating the improvement of the comprehensiveness and in-depthness of analyzing the reasons for missing strokes, and also facilitating the avoidance of one-sidedness in analysis caused by a single abnormal writing feature.

[0021] In a possible implementation manner, the method further includes:

[0022] Identify the display limit corresponding to the real-time change display graph of abnormal features and the display length of each feature value change bar, and determine the abnormal display change bar as the change bar with a display length greater than the display limit;

[0023] When the real-time change display graph of abnormal features contains an abnormal display change bar, determine the scaling ratio based on the real-time abnormal feature values corresponding to each abnormal writing feature in the real-time change display graph of abnormal features;

[0024] Scale the abnormal feature value bars corresponding to each abnormal writing feature based on the scaling ratio to obtain an updated real-time change display graph of abnormal features;

[0025] Determine the feature parameter combination corresponding to each predicted reason for missing strokes from the updated real-time change display graph of abnormal features, where the feature parameter combination includes the abnormal writing feature corresponding to the predicted reason for missing strokes and the abnormal feature value corresponding to each abnormal writing feature;

[0026] Mark the change bar area corresponding to each feature parameter combination from the updated real-time change display graph of abnormal features, and superimpose the corresponding predicted reasons for missing strokes.

[0027] By adopting the above technical solution, the display length of each eigenvalue change bar is supervised in real time through the display limit, which is convenient for timely detecting abnormal display change bars. And when an abnormal display change bar is detected, each abnormal eigenvalue bar is scaled in time, which is convenient for avoiding the situation where abnormal eigenvalue bars cannot be displayed. In addition, by marking the change bar area corresponding to each feature parameter combination in the updated abnormal feature real-time change display graph and superimposing the corresponding predicted reasons for missing strokes, it is convenient for relevant staff to intuitively view the reasons for missing strokes of users during the writing process.

[0028] In a possible implementation manner, when the target reason for missing strokes is hand occlusion, the method further includes:

[0029] Obtain historical writing data without missing strokes, and determine the habitual hand-holding posture from the historical writing data without missing strokes;

[0030] Determine the hand-holding posture and hand-holding pressure value corresponding to the pen-losing stage from the real-time writing data;

[0031] Match the habitual hand-holding posture with the hand-holding posture at the time of pen loss to determine hand-holding posture adjustment prompt information;

[0032] When the hand-holding pressure value at the time of pen loss appears again in the real-time writing data, feedback the hand-holding posture adjustment prompt information.

[0033] By adopting the above technical solution, personalized hand-holding prompt information is generated through different hand-holding postures and different hand-holding pressure values, so as to provide targeted guidance for users. This personalized feedback not only helps users correct their pen-holding postures in time, but also improves the writing experience and accuracy of users.

[0034] In a possible implementation manner, the method further includes:

[0035] When it is detected that the visualized motion trajectory graph contains abnormal writing features, record the abnormal frequency of the abnormal writing features within a preset time period;

[0036] Determine the writing scenario level based on the real-time writing data;

[0037] Determine the pen-loss detection time period based on the abnormal frequency and the writing scenario level.

[0038] By adopting the above technical solution, after abnormal writing features appear, the abnormal frequency of the abnormal writing features and the corresponding writing scenario level are analyzed to determine the need to trace the cause of missing strokes at the current moment. Finally, the corresponding detection period is determined based on the need, rather than immediately tracing the cause of missing strokes after identifying abnormal writing features, which facilitates a more reasonable arrangement of the time and resources for tracing processing and avoids unnecessary tension and resource waste.

[0039] In a second aspect, the present application provides a detection system, adopting the following technical solution:

[0040] A detection system, the detection system includes:

[0041] At least one processor;

[0042] A memory;

[0043] At least one application program, wherein the at least one application program is stored in the memory and is configured to be executed by the at least one processor, and the at least one application program is configured to: execute the above method for detecting the cause of missing strokes of a dot matrix pen based on a lightweight convolutional network.

[0044] In a third aspect, the present application provides a computer-readable storage medium, adopting the following technical solution:

[0045] A computer-readable storage medium, including: a computer program stored that can be loaded and executed by a processor to execute the above method for detecting the cause of missing strokes of a dot matrix pen based on a lightweight convolutional network.

[0046] In a fourth aspect, the present application provides a computer program product, adopting the following technical solution:

[0047] A computer program product, including a computer program, where the computer program, when executed by a processor, implements the above method for detecting the cause of missing strokes of a dot matrix pen based on a lightweight convolutional network.

[0048] In summary, the present application includes at least one of the following beneficial technical effects:

[0049] By using a trained lightweight convolutional network model to identify the real-time writing features contained in the real-time writing data, it is convenient to improve the accuracy when determining the real-time writing features. In addition, since the visualized motion trajectory diagram can intuitively display the dynamic changes during the writing process, therefore, by visualizing the real-time writing features, it is convenient to more accurately judge the reasons for missing strokes. Through the automated data processing and writing feature recognition process, the possibility of human intervention is reduced, thereby facilitating the reduction of misjudgments and delays caused by human factors. And, through an immediate feedback mechanism, it is convenient for the user to understand the reasons for missing strokes in the first time and take corresponding corrective measures, thereby facilitating the improvement of the timeliness and accuracy of the entire detection process.

[0050] Generate personalized hand-holding prompt information through different hand-holding postures and different hand-holding pressure values, so as to facilitate targeted guidance for users. This personalized feedback not only helps users correct their pen-holding postures in a timely manner, but also facilitates the improvement of users' writing experience and accuracy. Brief Description of the Drawings

[0051] Figure 1 is a schematic flowchart of a method for detecting the reasons for missing strokes of a dot matrix pen based on a lightweight convolutional network in an embodiment of the present application;

[0052] Figure 2 is a schematic diagram of selecting a variable bar area box in an embodiment of the present application;

[0053] Figure 3 is a schematic flowchart of determining hand-holding posture adjustment prompt information in an embodiment of the present application;

[0054] Figure 4 is a schematic structural diagram of a detection system in an embodiment of the present application. Detailed Description of the Embodiment

[0055] The following will Figures 1 to 4 make a further detailed description of the present application.

[0056] Those skilled in the art can make modifications to this embodiment without creative contributions according to needs after reading this specification, but as long as they are within the scope of the claims of the present application, they are protected by the Patent Law.

[0057] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are some, but not all, of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without making creative efforts fall within the scope of protection of the present application.

[0058] It should be noted that in the alternative embodiments of the present application, for relevant data such as object information, when the embodiments in the present application are applied to specific products or technologies, object permission or consent is required, and the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions. That is to say, if the embodiments in the present application involve data related to an object, it needs to be obtained under the authorization and consent of the object, the authorization and consent of relevant departments, and compliance with the relevant laws, regulations, and standards of the country and region. In the embodiments, if personal information is involved, the acquisition of all personal information requires the consent of the individual. If sensitive information is involved, the separate consent of the information subject is required. The embodiments also need to be implemented under the authorization and consent of the object.

[0059] Specifically, the embodiment of the present application provides a method for detecting the reason for stroke loss of a dot matrix pen based on a lightweight convolutional network, which is executed by a detection system. The detection system can be a server or a terminal device. Among them, the server can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The terminal device can be a smart phone, a tablet computer, a notebook computer, a desktop computer, etc., but is not limited thereto. The terminal device and the server can be directly or indirectly connected through wired or wireless communication methods, and the embodiments of the present application do not limit this.

[0060] Reference Figure 1 , Figure 1 is a schematic flowchart of a method for detecting the reason for stroke loss of a dot matrix pen based on a lightweight convolutional network in the embodiment of the present application. The method includes steps S110 - S130, where:

[0061] Step S110: Obtain real-time writing data, and import the real-time writing data into a trained lightweight convolutional network model for feature recognition to obtain real-time writing features. The real-time writing features include a movement trajectory, a hand grip pressure, and a timestamp.

[0062] Specifically, real-time writing data is the data generated by the user during the writing process using the dot matrix pen. The real-time writing data can be collected by the image acquisition device and pressure sensing built into the dot matrix pen. In addition to image recognition, the dot matrix pen can also use the touch recording technology to detect or identify the contact points between the tip of the dot matrix pen and the writing paper, so as to record the stroke trajectory generated during the writing process. The specific method of obtaining the real-time writing data is not specifically limited in the embodiments of the present application. The obtained real-time writing data is preprocessed to remove noise and outliers in the real-time writing data. In addition, the real-time writing data can also be standardized to make it more in line with the input requirements of the lightweight convolutional network. The preprocessed real-time writing data is imported into the trained lightweight convolutional network model to obtain the real-time writing features corresponding to the real-time writing data. The real-time writing features include but are not limited to the movement trajectory, hand grip pressure, and timestamp. Among them, the movement trajectory is the contact mark between the tip of the dot matrix pen and the paper during the user's use of the dot matrix pen, which can reflect information such as the writing fluency, stroke order, and writing style of the writer; the hand grip pressure is the pressure exerted by the user's hand on the pen or the pen on the paper surface during the writing process using the dot matrix pen, which is convenient for reflecting the force and writing strength of the user; the timestamp is the time information generated by each data point during the writing process, which is convenient for reflecting the writing speed and rhythm of the writer.

[0063] The training process of the lightweight convolutional network model includes: first, select a to-be-trained lightweight convolutional network model suitable for processing image and sequence data, such as MobileNet, ShuffleNet, etc. The specific to-be-trained lightweight network model is not specifically limited in the embodiments of the present application. Then, use the preprocessed historical writing data to train the to-be-trained lightweight convolutional network model. At the same time, appropriate loss functions and optimizers need to be set to minimize the prediction error and improve the model performance. The performance of the model can also be evaluated by methods such as cross-validation, and the hyperparameters can be adjusted to optimize the training results. The specific method of model training is not specifically limited in the embodiments of the present application, as long as it can identify the real-time writing features contained in the real-time writing data.

[0064] Step S120: Perform visualization processing on the real-time writing features to obtain a visualized movement trajectory map corresponding to the real-time writing features. The visualized movement trajectory map includes the movement trajectory of the dot matrix pen during the writing process and the hand grip pressure at each writing moment.

[0065] Specifically, by converting the real-time writing features into a visualized running trajectory graph, it is convenient for relevant staff to intuitively view the dynamic changes of the user during the writing process. First, the running trajectory included in the real-time writing features can be represented by lines or curves. The thickness or color of the lines or curves can be determined based on the hand pressure corresponding to each writing moment of the user. That is, the thickness of the line can be used to characterize the magnitude of the hand pressure. When drawing the visualized motion trajectory graph, the lines or curves can be adjusted by tracking the movement of the pen tip and recording its direction at each moment to ensure that the natural extension and inflection points of the lines or curves in the visualized motion trajectory graph can accurately reflect the direction changes in the actual writing process.

[0066] Step S130: Determine the target stroke loss reason based on the visualized motion trajectory graph and feedback the target stroke loss reason.

[0067] Specifically, based on the visualized motion trajectory graph, the reasons that may cause stroke loss can be analyzed and determined. For example, if there is a sudden interruption or a large fluctuation in force in the visualized motion trajectory graph, it may indicate that the stroke loss is caused by poor contact between the pen tip and the paper or unstable writing force of the user. Different stroke loss reasons correspond to different abnormal writing features. To improve the accuracy of determining the target stroke loss reason, determining the target stroke loss reason based on the visualized motion trajectory graph can specifically include:

[0068] Identify the abnormal writing features and the corresponding real-time abnormal feature values for each abnormal writing feature from the visualized motion trajectory graph; determine the proportion of real-time abnormal features based on all abnormal writing features and the corresponding real-time abnormal feature values for each abnormal writing feature; determine the target stroke loss reason corresponding to the proportion of real-time abnormal features based on the proportion of real-time abnormal features and the preset stroke loss mapping relationship, where the preset stroke loss mapping relationship is the corresponding relationship between the proportion of real-time abnormal features and the target stroke loss reason.

[0069] Specifically, the abnormal writing feature can be a motion trajectory with discontinuity, breakpoints or sudden changes in direction, a hand grip pressure that exceeds a preset stable pressure range, or a handwriting with a clarity lower than a preset standard clarity, etc., wherein the preset stable pressure range and the preset standard clarity are not specifically limited in the embodiments of the present application. The real-time abnormal feature value can reflect the abnormal degree of the corresponding real-time abnormal writing feature. When determining the real-time abnormal feature value corresponding to the real-time abnormal writing feature, the visual motion trajectory diagram can be imported into the preset feature value recognition model. The preset feature value recognition model can recognize the real-time abnormal feature values corresponding to all real-time abnormal writing features. For example, it is known that the natural extension and inflection point of the lines or curves in the visual motion trajectory diagram can accurately reflect the direction change in the actual writing process. Based on the preset feature value recognition model, the bending amplitude, duration and bending direction of the lines or curves can be identified from the visual motion trajectory diagram, and the corresponding real-time abnormal feature value can be output based on this. The preset feature value recognition model can be trained based on a large number of sample data sets containing abnormal writing features and annotated abnormal writing feature values. The specific training process is not limited in the embodiments of the present application. Since the writing process of the user may be continuous, the required writing data is collected and analyzed in real time.

[0070] Based on the preset feature value recognition model, the real-time abnormal feature values corresponding to all abnormal writing features can be determined, and all abnormal writing features and their corresponding real-time abnormal feature values are integrated to determine the real-time abnormal feature ratio. For example, for the existing abnormal writing features A, B and C, the real-time abnormal feature ratio is A:B:C=1:2:2. At this time, the corresponding target lost stroke cause can be determined based on the preset lost stroke mapping relationship, wherein the preset lost stroke mapping relationship is the correspondence between the real-time abnormal feature ratio and the target lost stroke cause. The specific content of the mapping relationship is not specifically limited in the embodiments of the present application, and can be determined by relevant staff based on historical experimental data and uploaded to the detection system. The target lost stroke cause is determined after considering the abnormal writing features in multiple dimensions, so as to improve the accuracy and reliability of determining the target lost stroke cause.

[0071] After determining the target stroke loss reason, the target stroke loss reason is fed back. Since the writing data is updated as the user writes, the target stroke loss reason may also be added and changed.

[0072] For the embodiments of the present application, by using the trained lightweight convolutional network model to identify the real-time writing features contained in the real-time writing data, it is convenient to improve the accuracy when determining the real-time writing features. In addition, since the visualized motion trajectory graph can intuitively display the dynamic changes during the writing process, therefore, by visualizing the real-time writing features, it is convenient to more accurately judge the reasons for missing strokes. Through the automated data processing and writing feature recognition process, the possibility of human intervention is reduced, thereby facilitating the reduction of misjudgment and delay caused by human factors. And, through the instant feedback mechanism, it is convenient for the user to understand the reasons for missing strokes in the first time and take corresponding corrective measures, thereby facilitating the improvement of the timeliness and accuracy of the entire detection process.

[0073] Furthermore, in order to facilitate the improvement of the comprehensiveness when analyzing the reasons for missing strokes, when the number of anomalies of the abnormal writing features in the visualized motion trajectory graph is higher than the preset anomaly threshold, the method provided by the embodiments of the present application further includes:

[0074] Based on each abnormal writing feature and the real-time abnormal feature value corresponding to each abnormal writing feature, determine the real-time change display graph of abnormal features. The real-time change display graph of abnormal features contains the abnormal feature value change bars corresponding to each abnormal writing feature; determine multiple feature combinations based on each abnormal writing feature in the real-time change display graph, and determine the predicted reasons for missing strokes according to the real-time abnormal feature values corresponding to each abnormal writing feature in each feature combination; record the determination moment of each predicted reason for missing strokes, and sort all the predicted reasons for missing strokes based on each determination moment to form a list of reasons for missing strokes.

[0075] Specifically, the preset anomaly threshold can be 4 or 5. The specific value is not specifically limited in the embodiments of the present application. When the number of anomalies of the abnormal writing features is higher than the preset anomaly threshold, the real-time change display graph of abnormal features can be drawn to more intuitively display the changes and developments of multiple abnormal writing features during the continuous writing process. The real-time change display graph of abnormal features contains a display coordinate system and the abnormal feature value change bars corresponding to each abnormal writing feature. The display coordinate system contains a time axis and an abnormal feature value change axis. Each abnormal feature value change bar is used to represent the changes that occur to the corresponding abnormal writing feature during the continuous writing process. As the real-time abnormal feature value increases, the corresponding abnormal feature value change bar will gradually extend. By integrating each abnormal writing feature and its corresponding real-time abnormal feature value into the real-time change display graph of abnormal features, it is convenient to monitor the abnormal situations during the writing process in a timely manner.

[0076] Multiple feature combinations can be formed among different abnormal writing features. For different feature combinations and the corresponding proportion of combined features, the corresponding predicted reasons for missing strokes may be different. Among them, the proportion of combined features is the proportion of features among various abnormal writing features in the feature combination. For example, when the proportion of combined features is A:B:C = 1:2:2, the corresponding predicted reason for missing strokes may be reason 1; when the proportion of combined features is A:B:D = 1:2:2, the corresponding predicted reason for missing strokes may be reason 2; when the proportion of combined features is A:C:D = 1:3:2, the corresponding predicted reason for missing strokes may be reason 3.

[0077] Since the abnormal writing features corresponding to different predicted reasons for missing strokes are different, and the change rates of the real-time abnormal feature values of different abnormal writing features are different, therefore, the determination times of different predicted reasons for missing strokes are also different. The corresponding predicted reasons for missing strokes can be sorted based on each determination time, and all the predicted reasons for missing strokes in the generated list of reasons for missing strokes are sorted in chronological order. Combining the various abnormal writing features included in the real-time change display diagram is convenient for considering the interaction between different abnormal writing features, thereby facilitating the improvement of the comprehensiveness and in-depthness of the analysis of the reasons for missing strokes, and also facilitating the avoidance of one-sidedness in the analysis caused by a single abnormal writing feature.

[0078] To avoid the situation where the abnormal feature value bars cannot be displayed, the method provided in the embodiments of the present application further includes:

[0079] Identifying the display limit corresponding to the real-time change display diagram of abnormal features and the display length of each feature value change bar, and determining the abnormal display change bar as the abnormal feature value change bar with a display length greater than the display limit; when the real-time change display diagram of abnormal features contains an abnormal display change bar, determining the scaling ratio based on the real-time abnormal feature values corresponding to the various abnormal writing features in the real-time change display diagram of abnormal features; scaling the abnormal feature value bars corresponding to the various abnormal writing features based on the scaling ratio to obtain an updated real-time change display diagram of abnormal features.

[0080] Specifically, the display limit corresponding to the real-time change display diagram of abnormal features can be set according to one's own needs. For example, when the display limit is 10, when the display length of the feature value change bar grows to the display limit, the feature value change bar can be shrunk. At this time, the abnormal feature value change bar with a display length greater than the display limit can be determined as the abnormal display change bar. When shrinking the abnormal display change bar, it is necessary to shrink all the abnormal feature value change bars in the real-time change display diagram of abnormal features at the same time. When the real-time change display diagram of abnormal features contains abnormal display change bars, the display lengths corresponding to all the abnormal feature value change bars can be identified, and the scaling ratio can be determined based on all the display lengths. Specifically, the average length of the display lengths of all the abnormal feature value change bars can be calculated, and then the corresponding scaling ratio can be determined based on the average length. For example, if the average length is 5, the corresponding scaling ratio is 1 / 5, that is, each abnormal feature value change bar in the real-time change display diagram is shrunk by 5 times. The specific method for determining the scaling ratio is not specifically limited in the embodiments of the present application.

[0081] Determine the feature parameter combination corresponding to each predicted stroke loss reason from the updated real-time change display diagram of abnormal features. The feature parameter combination includes the abnormal writing features corresponding to the predicted stroke loss reason and the abnormal feature values corresponding to each abnormal writing feature; mark the change bar area corresponding to each feature parameter combination from the updated real-time change display diagram of abnormal features, and superimpose the corresponding predicted stroke loss reasons.

[0082] Specifically, the feature parameter combinations corresponding to different predicted stroke loss reasons are different. Since the predicted stroke loss reasons have been determined at this time, the abnormal feature values included in the feature parameter combination corresponding to the predicted stroke loss reason are fixed values and can be selected by frame from the updated real-time change display diagram of abnormal features. As Figure 2 shown, Figure 2 in the figure, there is an overlapping area in the abnormal feature value change bars selected for the predicted stroke loss reason 1 and the predicted stroke loss reason 2. The frame selection border can be a solid line or a dotted line. The specific form of the frame selection border is not specifically limited in the embodiments of the present application. As long as the stroke loss reasons of the user during writing can be intuitively viewed after frame selection and superposition of the predicted stroke loss reasons.

[0083] When the target stroke loss reason is hand occlusion, the method provided by the embodiments of the present application further includes steps S210 - S240, as Figure 3 shown, where:

[0084] Step S210: Obtain historical writing data without stroke loss, and determine the habitual hand-holding posture from the historical writing data without stroke loss.

[0085] Specifically, the historical writing data without stroke loss refers to the writing data during a historical time period when using a dot matrix pen without any stroke loss. Hand-holding features are extracted from the historical writing data without stroke loss, such as the degree of finger bending, the angle between the palm and the paper surface, the relative position between fingers, the holding position of the pen shaft, etc. Then, the habitual hand-holding features are determined by observing the distribution of the clustering centers or eigenvalues. Finally, based on the habitual hand-holding features, the hand-holding posture of the user during the historical writing process is simulated.

[0086] Step S220: Determine the pen-dropping hand-holding posture and the pen-dropping hand-holding pressure value corresponding to the pen-dropping stage from the real-time writing data.

[0087] Specifically, the method for determining the pen-dropping hand-holding posture can refer to the method for determining the habitual hand-holding posture from the historical writing data without stroke loss, which will not be elaborated here. The pen-dropping hand-holding posture is also a simulated hand-holding posture. The pen-dropping hand-holding pressure value is the pressure exerted on the dot matrix pen in the pen-dropping hand-holding posture, which is jointly composed of the contact pressures of multiple contact points.

[0088] Step S230: Match the habitual hand-holding posture with the pen-dropping hand-holding posture to determine the hand-holding posture adjustment prompt information.

[0089] Step S240: When the pen-dropping hand-holding pressure value appears again in the real-time writing data, feedback the hand-holding posture adjustment prompt information.

[0090] Specifically, machine learning algorithms are used to match the features of the habitual hand-holding posture and the pen-dropping hand-holding posture to determine the key differences between the habitual hand-holding posture and the pen-dropping hand-holding posture. These key differences may include insufficient finger bending, too large or too small an angle between the palm and the paper surface, incorrect relative position between fingers, etc. Based on the key differences between the hand-holding postures, hand-holding posture adjustment prompt information is generated. The hand-holding posture adjustment prompt information can be "Please increase the finger bending degree to hold the pen shaft more tightly with your fingers", "Please reduce the angle between the palm and the paper surface to keep the wrist in a natural state", or "Please adjust the relative position between your fingers to ensure that the thumb, index finger, and middle finger form a stable triangular support", etc. The specific hand-holding posture adjustment prompt information is not limited in the embodiments of this application.

[0091] By using the different hand-holding postures and different hand-holding pressure values, personalized hand-holding prompt information is generated to facilitate targeted guidance for the user. This personalized feedback not only helps the user correct the pen-holding posture in a timely manner but also improves the user's writing experience and accuracy.

[0092] To facilitate a more reasonable arrangement of the time and resources for traceability processing, the method provided in the embodiments of this application further includes:

[0093] When it is detected that the visualized motion trajectory map contains abnormal writing features, record the abnormal frequency of the abnormal writing features within a preset time period; determine the writing scenario level based on the real-time writing data; and determine the stroke loss detection period based on the abnormal frequency and the writing scenario level.

[0094] Specifically, the preset time period is a period of time after the moment when it is detected that the visualized motion trajectory map contains abnormal writing features. The duration corresponding to the preset time period can be 20 seconds or 30 seconds. The specific duration is not specifically limited in the embodiments of the present application and can be set by relevant technical personnel according to actual needs. In the embodiments of the present application, the reason for the abnormality is not traced immediately after the abnormal writing features are detected, that is, the reason for the stroke loss is not traced immediately after the abnormal writing features are detected. Instead, the abnormal level needs to be determined by first analyzing the abnormal frequency of the abnormal writing features and the writing scenario level within the preset time period, and then the corresponding stroke loss detection period is determined based on the abnormal level. The higher the abnormal level, the closer the corresponding stroke loss detection period is to the preset time period.

[0095] When determining the abnormal level based on the abnormal frequency and the writing scenario level, the first abnormal score corresponding to the abnormal frequency can be determined according to the preset first mapping relationship first, and then the second abnormal score corresponding to the writing scenario level can be determined based on the writing scenario level and the preset second mapping relationship. Finally, the abnormal level is jointly determined based on the first abnormal score and the second abnormal score. Among them, the preset first mapping relationship is the corresponding relationship between the abnormal frequency and the first abnormal score, and the preset second mapping relationship is the corresponding relationship between the writing scenario level and the second abnormal score. The specific content is not specifically limited in the embodiments of the present application and can be determined by relevant staff according to historical experimental data and then uploaded to the detection system. When determining the writing scenario level based on the real-time writing data, the writing type can be determined based on the real-time writing data. The writing type includes document signing, examination, draft, etc. Different writing types correspond to different writing scenario levels, and there is a corresponding relationship between the writing type and the writing scenario level. If the writing scenario level of the dot matrix pen is relatively high, that is, it indicates that the usage scenario of the dot matrix pen has extremely high requirements for writing accuracy, such as legal documents, contract signing, or important notes, etc. At this time, each stroke loss may affect the integrity and accuracy of the written content. In this case, it is very necessary and timely to trace the reason for the stroke loss. Therefore, the higher the writing scenario level, the higher the corresponding second abnormal score.

[0096] By analyzing the abnormal frequency of the abnormal writing features and the corresponding writing scenario level after the abnormal writing features appear, the requirement for tracing the reason for the stroke loss at the current moment is determined. Finally, the corresponding detection period is determined based on the requirement, rather than immediately performing the stroke loss reason tracing process after identifying the abnormal writing features, which is convenient for more reasonably arranging the time and resources for the tracing process and avoiding unnecessary tension and resource waste.

[0097] In an embodiment of the present application, a detection system is provided, as Figure 4 shown, Figure 4 The detection system 400 shown includes: a processor 401 and a memory 403. Among them, the processor 401 and the memory 403 are connected, such as connected through a bus 402. Optionally, the detection system 400 may further include a transceiver 404. It should be noted that in practical applications, the transceiver 404 is not limited to one, and the structure of the detection system 400 does not constitute a limitation to the embodiment of the present application.

[0098] The processor 401 may be a CPU (Central Processing Unit, central processing unit), a general-purpose processor, a DSP (Digital Signal Processor, data signal processor), an ASIC (Application Specific Integrated Circuit, application-specific integrated circuit), an FPGA (Field Programmable Gate Array, field programmable gate array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It can implement or execute various exemplary logic blocks, modules and circuits described in combination with the disclosure of the present application. The processor 401 may also be a combination that realizes computing functions, such as a combination including one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0099] The bus 402 may include a path for transmitting information between the above components. The bus 402 may be a PCI (Peripheral Component Interconnect, peripheral component interconnect standard) bus or an EISA (Extended Industry Standard Architecture, extended industry standard architecture) bus, etc. The bus 402 may be divided into an address bus, a data bus, a control bus, etc. For the sake of representation, Figure 4 only one line is shown in the figure, but it does not mean that there is only one bus or one type of bus.

[0100] The memory 403 can be a ROM (Read Only Memory), or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory), or other types of dynamic storage devices that can store information and instructions. It can also be an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory), or other optical disc storage, optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media, or other magnetic storage devices, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0101] The memory 403 is used to store the application program code for executing the solution of this application, and is controlled by the processor 401 for execution. The processor 401 is used to execute the application program code stored in the memory 403 to implement the content shown in the foregoing method embodiments.

[0102] Among them, the detection system includes but is not limited to: mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Tablet Computers), PMPs (Portable Multimedia Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. It can also be a server, etc. Figure 4 The shown detection system is only an example and should not impose any limitations on the functions and usage scope of the embodiments of this application.

[0103] The embodiments of this application provide a computer-readable storage medium, on which a computer program is stored. When it runs on a computer, it enables the computer to execute the corresponding content in the foregoing method embodiments.

[0104] An embodiment of the present application provides a computer program product, which includes a computer program that, when executed by a processor, implements the method in any of the above embodiments. Compared with the related art, in the embodiment of the present application, by using a trained lightweight convolutional network model to identify the real-time writing features contained in the real-time writing data, it is convenient to improve the accuracy when determining the real-time writing features. In addition, since the visualized motion trajectory diagram can intuitively display the dynamic changes during the writing process, therefore, by visualizing the real-time writing features, it is convenient to more accurately judge the reason for missing strokes. Through the automated data processing and writing feature recognition process, the possibility of human intervention is reduced, thereby facilitating the reduction of misjudgment and delay caused by human factors. And, through the instant feedback mechanism, it is convenient for the user to understand the reason for missing strokes in the first time and take corresponding corrective measures, thereby facilitating the improvement of the timeliness and accuracy of the entire detection process.

[0105] It should be understood that although the steps in the flowchart of the accompanying drawings are shown in sequence according to the indication of the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear indication in this article, the execution of these steps has no strict order limit, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed alternately or alternately with at least a part of other steps or sub-steps or stages of other steps.

[0106] The above are only some embodiments of the present application. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present application, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present application.

Claims

1. A method for detecting the reason of stroke loss of a dot matrix pen based on a lightweight convolutional network, characterized in that, Including: Obtain real-time writing data, and import the real-time writing data into a trained lightweight convolutional network model for feature recognition to obtain real-time writing features, where the real-time writing features include a movement trajectory, a hand grip pressure, and a timestamp; Perform visualization processing on the real-time writing features to obtain a visualized movement trajectory map corresponding to the real-time writing features, where the visualized movement trajectory map includes the movement trajectory of the dot matrix pen during writing and the hand grip pressure at each writing moment; Determine the target reason for missing strokes based on the visualized movement trajectory map, and feedback the target reason for missing strokes; Among them, determining the target reason for missing strokes based on the visualized movement trajectory map includes: Identify abnormal writing features and real-time abnormal feature values corresponding to each abnormal writing feature from the visualized movement trajectory map; Determine the proportion of real-time abnormal features based on all abnormal writing features and the real-time abnormal feature values corresponding to each abnormal writing feature; Determine the target reason for missing strokes corresponding to the proportion of real-time abnormal features based on the proportion of real-time abnormal features and a preset mapping relationship for missing strokes, where the preset mapping relationship for missing strokes is the corresponding relationship between the proportion of real-time abnormal features and the target reason for missing strokes; Among them, when the number of abnormalities of the abnormal writing features in the visualized movement trajectory map is higher than a preset abnormality threshold, it further includes: Determine a real-time change display map of abnormal features based on each abnormal writing feature and the real-time abnormal feature values corresponding to each abnormal writing feature, where the real-time change display map of abnormal features includes change bars of abnormal feature values corresponding to each abnormal writing feature; Determine multiple feature combinations based on the abnormal writing features in the real-time change display map, and determine the predicted reason for missing strokes according to the real-time abnormal feature values corresponding to each abnormal writing feature in each feature combination; Record the determination time of each predicted reason for missing strokes, and sort all the predicted reasons for missing strokes based on each determination time to form a list of reasons for missing strokes; Among them, it further includes: Identify the display limit corresponding to the real-time change display map of abnormal features and the display length of each change bar of feature values, and determine the abnormally displayed change bar as the change bar of abnormal feature values with a display length greater than the display limit; When the real-time change display map of abnormal features includes an abnormally displayed change bar, determine a scaling ratio based on the real-time abnormal feature values corresponding to the abnormal writing features in the real-time change display map of abnormal features; Scale the change bars of abnormal feature values corresponding to each abnormal writing feature based on the scaling ratio to obtain an updated real-time change display map of abnormal features; Determine a feature parameter combination corresponding to each predicted reason for missing strokes from the updated real-time change display map of abnormal features, where the feature parameter combination includes the abnormal writing features corresponding to the predicted reason for missing strokes and the abnormal feature values corresponding to each abnormal writing feature; Mark the change bar area corresponding to each feature parameter combination from the updated real-time change display map of abnormal features, and superimpose the corresponding predicted reasons for missing strokes.

2. The method for detecting the reason of stroke loss of a dot matrix pen based on a lightweight convolutional network according to claim 1, characterized in that When the target reason for missing strokes is hand occlusion, it further includes: Obtain historical stroke - free writing data, and determine the habitual hand - holding posture from the historical stroke - free writing data; Determine the pen - dropping hand - holding posture and the pen - dropping hand - holding pressure value corresponding to the pen - dropping stage from the real - time writing data; Match the habitual hand - holding posture with the pen - dropping hand - holding posture to determine the hand - holding posture adjustment prompt information; When the pen - dropping hand - holding pressure value appears again in the real - time writing data, feedback the hand - holding posture adjustment prompt information.

3. A method for detecting the reason of stroke loss of a dot matrix pen based on a lightweight convolutional network according to claim 1, characterized in that, Further include: When it is detected that the visualized motion trajectory map contains abnormal writing features, record the abnormal frequency of the abnormal writing features within a preset time period; Determine the writing scenario level based on the real - time writing data; Determine the pen - stroke - dropping detection period based on the abnormal frequency and the writing scenario level.

4. A detection system, characterized in that, The detection system includes: At least one processor; A memory; At least one application program, wherein the at least one application program is stored in the memory and is configured to be executed by the at least one processor. The at least one application program is configured to: execute a method for detecting the reason of pen - stroke - dropping of a dot matrix pen based on a lightweight convolutional network according to any one of claims 1 - 3.

5. A computer-readable storage medium, characterized in that, Include: A computer program stored with a method for detecting the reason of pen - stroke - dropping of a dot matrix pen based on a lightweight convolutional network according to any one of claims 1 - 3 that can be loaded and executed by a processor.

6. A computer program product, characterized in that, Include a computer program, and when the computer program is executed by a processor, it implements the steps of a method for detecting the reason of pen - stroke - dropping of a dot matrix pen based on a lightweight convolutional network according to any one of claims 1 - 3.

Images