Lattice pen lost stroke reason detection method and system based on lightweight convolutional network

Abstract

The invention relates to the technical field of stroke loss reason detection, in particular to a dot matrix pen stroke loss reason detection method and system based on a lightweight convolutional network, and the method comprises the steps: obtaining real-time writing data, importing the real-time writing data into a trained lightweight convolutional network model, carrying out the feature recognition, obtaining real-time writing features, and carrying out the recognition of the real-time writing features. The real-time writing characteristics comprise a movement track, handheld pressure and a timestamp; the real-time writing features are subjected to visualization processing, a visualization motion trail diagram corresponding to the real-time writing features is obtained, and the visualization motion trail diagram comprises the motion trail of the dot matrix pen in the writing process and the holding pressure at each writing moment; and determining a target stroke loss reason based on the visual motion trail diagram, and feeding back the target stroke loss reason. The method is convenient for improving the timeliness and accuracy of determining the stroke loss reason.

Description

Technical Field

[0001] The present application relates to the technical field of stroke loss cause detection, and in particular to a method and system for detecting stroke loss causes of a dot matrix pen based on a lightweight convolutional network. Background Art

[0002] With the continuous development of science and technology, digital transformation has become the general trend of all walks of life. As a kind of intelligent learning tool, the dot pen meets the needs of digital transformation and can help users better adapt to the digital working and learning environment. In addition, compared with traditional paper and pen writing, the dot pen can capture handwriting more accurately, making writing more smooth and natural. Therefore, in recent years, the dot pen has been favored by more and more people in work and study.

[0003] However, despite the many advantages of the dot matrix pen, there is still a certain proportion of stroke loss problems during use. If the cause of the lost strokes cannot be quickly and accurately located, it will not only interfere with the normal use of the dot matrix pen and cause incomplete data recording, but may also reduce writing efficiency and lead to a decrease in user satisfaction. Summary of the invention

[0004] In order to improve the timeliness and accuracy of determining the cause of lost strokes, the present application provides a method and system for detecting the cause of lost strokes of a dot matrix pen based on a lightweight convolutional network.

[0005] In a first aspect, the present application provides a method for detecting the cause of lost strokes of a dot matrix pen based on a lightweight convolutional network, which adopts the following technical solution: A method for detecting the cause of lost strokes of a dot-matrix pen based on a lightweight convolutional network, comprising: Acquire real-time writing data, and import the real-time writing data into a trained lightweight convolutional network model for feature recognition to obtain real-time writing features, wherein the real-time writing features include motion trajectory, hand grip pressure, and timestamp; Performing visualization processing on the real-time writing feature to obtain a visualized motion trajectory diagram corresponding to the real-time writing feature, wherein the visualized motion trajectory diagram includes the motion trajectory of the dot matrix pen during the writing process and the hand holding pressure at each writing moment; The reason for the target missing strokes is determined based on the visualized motion trajectory diagram, and the reason for the target missing strokes is fed back.

[0006] By adopting the above technical solution, by using the trained lightweight convolutional network model to identify the real-time writing features contained in the real-time writing data, it is convenient to improve the accuracy of determining the real-time writing features. In addition, since the visualized motion trajectory diagram can intuitively display the dynamic changes in the writing process, by visualizing the real-time writing features, it is convenient to more accurately determine the cause of lost strokes. Through automated data processing and writing feature recognition processes, the possibility of human intervention is reduced, thereby facilitating the reduction of misjudgments and delays caused by human factors. Moreover, through the instant feedback mechanism, it is convenient for users to understand the reasons for lost strokes at the first time and take corresponding corrective measures, thereby facilitating the improvement of the timeliness and accuracy of the entire detection process.

[0007] In a possible implementation, determining the cause of target stroke loss based on the visualized motion trajectory graph includes: Identify abnormal writing features and real-time abnormal feature values ​​corresponding to each abnormal writing feature from the visualized motion trajectory diagram; Determine the real-time abnormal feature proportion based on all abnormal writing features and the real-time abnormal feature value corresponding to each abnormal writing feature; Based on the real-time abnormal feature proportion and the preset lost stroke mapping relationship, the target lost stroke cause corresponding to the real-time abnormal feature proportion is determined, and the preset lost stroke mapping relationship is the corresponding relationship between the real-time abnormal feature proportion and the target lost stroke cause.

[0008] By adopting the above technical solution, by identifying each abnormal writing feature and the abnormal feature value corresponding to each abnormal writing feature from the visualized motion trajectory diagram, that is, by quantifying the abnormal writing features, it is convenient for relevant personnel to intuitively view the abnormal writing conditions and the degree of abnormality in the writing process. In addition, by analyzing the ratio between multiple abnormal writing features, the corresponding target stroke loss cause is determined based on the abnormal feature ratio, that is, the target stroke loss cause is determined after considering the abnormal writing features in multiple dimensions, which facilitates improving the accuracy and reliability of determining the target stroke loss cause.

[0009] In a possible implementation, when the number of abnormal writing features in the visualized motion trajectory graph is higher than a preset abnormal threshold, the method further includes: Based on each abnormal writing feature and the real-time abnormal feature value corresponding to each abnormal writing feature, determine a real-time abnormal feature change display diagram, wherein the abnormal feature change display diagram includes an abnormal feature value change bar corresponding to each abnormal writing feature; Determine multiple feature combinations based on each abnormal writing feature in the real-time change display diagram, and determine the predicted cause of lost strokes according to the real-time abnormal feature value corresponding to each abnormal writing feature in each feature combination; The determination time of each predicted stroke loss reason is recorded, and all predicted stroke loss reasons are sorted based on each determination time to form a stroke loss reason list.

[0010] By adopting the above technical solution, by integrating each abnormal writing feature and its corresponding real-time abnormal feature value into the real-time change display diagram of the abnormal feature, it is convenient to timely monitor the abnormal situation in the writing process. In addition, the various abnormal writing features contained in the real-time change display diagram are combined to consider the interaction between different abnormal writing features, thereby facilitating the improvement of the comprehensiveness and depth of the analysis of the causes of lost strokes, and also facilitating the avoidance of one-sided analysis caused by a single abnormal writing feature.

[0011] In one possible implementation, the method further includes: Identify the display limit corresponding to the abnormal feature real-time change display graph and the display length of each feature value change bar, and determine the abnormal feature value change bar whose display length is greater than the display limit as an abnormal display change bar; When the abnormal feature real-time change display graph includes an abnormal display change bar, a zoom ratio is determined based on the real-time abnormal feature values ​​corresponding to each abnormal writing feature in the abnormal feature real-time change display graph; Scaling the abnormal feature value bars corresponding to each abnormal writing feature based on the scaling ratio to obtain an updated abnormal feature real-time change display diagram; Determine a feature parameter combination corresponding to each predicted stroke loss reason from the updated abnormal feature real-time change display diagram, the feature parameter combination including an abnormal writing feature corresponding to the predicted stroke loss reason and an abnormal feature value corresponding to each abnormal writing feature; The change bar area corresponding to each feature parameter combination is marked in the updated abnormal feature real-time change display diagram, and the corresponding predicted stroke loss reason is superimposed.

[0012] By adopting the above technical scheme, the display length of each characteristic value change bar is monitored in real time through the display limit, so as to timely discover abnormal display change bars, and by timely scaling each abnormal characteristic value bar when an abnormal display change bar is detected, it is convenient to avoid the situation where the abnormal characteristic value bar cannot be displayed. In addition, by marking the change bar area corresponding to each characteristic parameter combination in the updated abnormal characteristic real-time change display diagram, and superimposing the corresponding predicted reasons for lost strokes, it is convenient for relevant staff to intuitively view the reasons for lost strokes during the writing process.

[0013] In a possible implementation, when the reason for the target stroke loss is hand-holding occlusion, the method further includes: Acquire historical writing data without missing strokes, and determine the habitual hand holding posture from the historical writing data without missing strokes; Determine the pen-dropping hand holding posture and the pen-dropping hand holding pressure value corresponding to the pen-dropping stage from the real-time writing data; Matching the habitual hand holding posture with the pen-losing hand holding posture to determine hand holding posture adjustment prompt information; When the pen-losing hand-gripping pressure value appears again in the real-time writing data, the hand-gripping posture adjustment prompt information is fed back.

[0014] By adopting the above technical solution, personalized hand-gripping prompt information is generated through different hand-gripping postures and different hand-gripping pressure values, so as to provide targeted guidance to users. This personalized feedback not only helps users to correct their pen-gripping postures in time, but also facilitates improving users' writing experience and accuracy.

[0015] In one possible implementation, the method further includes: When it is detected that the visualized motion trajectory graph contains abnormal writing features, the abnormal frequency of the abnormal writing features within a preset time period is recorded; determining a writing scene level based on the real-time writing data; A missing stroke detection period is determined based on the abnormal frequency and the writing scene level.

[0016] By adopting the above technical solution, after the abnormal writing features appear, the abnormal frequency of the abnormal writing features and the corresponding writing scene level are analyzed to determine the need for tracing the cause of lost strokes at the current moment, and finally determine the corresponding detection time period based on the need, instead of immediately tracing the cause of lost strokes after the abnormal writing features are identified. This makes it easier to arrange the time and resources for tracing the cause more reasonably and avoid unnecessary tension and waste of resources.

[0017] In a second aspect, the present application provides a detection system, which adopts the following technical solution: A detection system, the detection system comprising: at least one processor; Memory; At least one application, wherein the at least one application is stored in a memory and configured to be executed by at least one processor, and the at least one application is configured to: execute the above-mentioned method for detecting causes of lost strokes of a dot matrix pen based on a lightweight convolutional network.

[0018] In a third aspect, the present application provides a computer-readable storage medium, which adopts the following technical solution: A computer-readable storage medium includes: a computer program that can be loaded by a processor and execute the above-mentioned method for detecting the cause of lost strokes of a dot matrix pen based on a lightweight convolutional network.

[0019] In a fourth aspect, the present application provides a computer program product, which adopts the following technical solution: A computer program product includes a computer program, which, when executed by a processor, implements the above-mentioned method for detecting the cause of lost strokes of a dot matrix pen based on a lightweight convolutional network.

[0020] In summary, the present application includes at least one of the following beneficial technical effects: By using a trained lightweight convolutional network model to identify the real-time writing features contained in the real-time writing data, it is convenient to improve the accuracy of determining the real-time writing features. In addition, since the visualized motion trajectory diagram can intuitively display the dynamic changes in the writing process, by visualizing the real-time writing features, it is convenient to more accurately determine the cause of lost strokes. Through automated data processing and writing feature recognition processes, the possibility of human intervention is reduced, thereby reducing misjudgments and delays caused by human factors. Moreover, through the instant feedback mechanism, users can understand the reasons for lost strokes at the first time and take corresponding corrective measures, thereby improving the timeliness and accuracy of the entire detection process.

[0021] Through different hand-holding postures and different hand-holding pressure values, personalized hand-holding prompt information is generated to provide targeted guidance to users. This personalized feedback not only helps users correct their pen-holding posture in time, but also helps improve their writing experience and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a flow chart of a method for detecting the cause of lost strokes of a dot matrix pen based on a lightweight convolutional network in an embodiment of the present application; Figure 2 This is a schematic diagram of a change bar area selection in an embodiment of the present application; Figure 3 This is a schematic diagram of a process for determining hand-holding posture adjustment prompt information in an embodiment of the present application; Figure 4 It is a structural schematic diagram of a detection system in an embodiment of the present application. DETAILED DESCRIPTION

[0023] The following is combined with Figures 1 to 4 This application is described in further detail.

[0024] After reading this specification, those skilled in the art may make non-creative modifications to this embodiment as needed, but such modifications are protected by patent law as long as they are within the scope of the claims of this application.

[0025] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0026] It should be noted that in the optional embodiments of the present application, the object information and other related data involved, when the embodiments in the present application are applied to specific products or technologies, need to obtain the permission or consent of the object, and the collection, use and processing of the relevant data need to comply with the relevant laws, regulations and standards of the relevant countries and regions. In other words, if the embodiments of the present application involve data related to the object, it needs to be obtained with the authorization and consent of the object, the authorization and consent of the relevant departments, and in compliance with the relevant laws, regulations and standards of the country and region. If personal information is involved in the embodiments, the acquisition of all personal information needs to obtain the consent of the individual. If sensitive information is involved, the separate consent of the information subject needs to be obtained. The embodiments also need to be implemented with the authorization and consent of the object.

[0027] Specifically, the embodiment of the present application provides a method for detecting the cause of lost strokes of a dot matrix pen based on a lightweight convolutional network, which is executed by a detection system, and the detection system can be a server or a terminal device, wherein the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The terminal device can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc., but is not limited thereto. The terminal device and the server can be directly or indirectly connected via wired or wireless communication, and the embodiment of the present application does not limit this.

[0028] refer to Figure 1 , Figure 1 1 is a flow chart of a method for detecting the cause of lost strokes of a dot matrix pen based on a lightweight convolutional network in an embodiment of the present application. The method comprises steps S110 to S130, wherein: Step S110: Acquire real-time writing data, and import the real-time writing data into a trained lightweight convolutional network model for feature recognition to obtain real-time writing features, which include motion trajectory, hand grip pressure, and timestamp.

[0029] Specifically, real-time writing data is data generated by the user when using a dot matrix pen to write. The real-time writing data can be collected by the built-in image acquisition device and pressure sensor of the dot matrix pen. In addition to image recognition, the dot matrix pen can also use touch recording technology to detect or identify the contact point between the pen tip of the dot matrix pen and the writing paper, thereby recording the stroke trajectory generated during the writing process. The specific method of obtaining real-time writing data is not specifically limited in the embodiments of this application. The acquired real-time writing data is preprocessed to remove noise and outliers in the real-time writing data. In addition, the real-time writing data can also be standardized to make it more in line with the input requirements of the lightweight convolutional network. The standardized real-time writing data is imported into the trained lightweight convolutional network model to obtain the real-time writing features corresponding to the real-time writing data. The real-time writing features include but are not limited to motion trajectory, hand holding pressure and timestamp, wherein the motion trajectory is the contact trace between the pen tip and the paper when the user uses the dot matrix pen, which can reflect the writer's writing fluency, stroke sequence and writing style; the hand holding pressure is the pressure of the user's hand on the pen or the pen on the paper when the user uses the dot matrix pen to write, which is convenient for reflecting the user's force and writing strength; the timestamp is the time information generated by each data point in the writing process, which is convenient for reflecting the writer's writing speed and rhythm.

[0030] The training process of the lightweight convolutional network model includes: first selecting a lightweight convolutional network model to be trained that is suitable for processing image and sequence data, such as MobileNet, ShuffleNet, etc. The specific lightweight network model to be trained is not specifically limited in the embodiment of this application, and then using the pre-processed historical writing data to train the lightweight convolutional network model to be trained. At the same time, it is necessary to set a suitable loss function and optimizer to minimize the prediction error and improve the model performance. The performance of the model can also be evaluated by cross-validation and other methods, and the hyperparameters can be adjusted to optimize the training results. The specific method of model training is not specifically limited in the embodiment of this application, as long as the real-time writing features contained in the real-time writing data can be identified.

[0031] Step S120: Visualize the real-time writing feature to obtain a visualized motion trajectory diagram corresponding to the real-time writing feature. The visualized motion trajectory diagram includes the motion trajectory of the dot matrix pen during the writing process and the hand holding pressure at each writing moment.

[0032] Specifically, by converting the real-time writing features into a visual running trajectory diagram, it is convenient for relevant staff to intuitively view the dynamic changes of the user during the writing process. The running trajectory contained in the real-time writing features can be represented by lines or curves. The thickness or color of the lines or curves can be determined based on the user's corresponding hand holding pressure at each writing moment, that is, the thickness of the line can be used to represent the size of the hand holding pressure. When drawing a visual motion trajectory diagram, the lines or curves can be adjusted by tracking the movement of the pen tip and recording its direction at each moment to ensure that the natural extension and inflection points of the lines or curves in the visual motion trajectory diagram can accurately reflect the direction changes in the actual writing process.

[0033] Step S130: determining the target stroke loss reason based on the visualized motion trajectory diagram, and providing feedback on the target stroke loss reason.

[0034] Specifically, based on the visualized motion trajectory graph, it is possible to analyze and determine the possible causes of lost strokes. For example, if there is a sudden interruption or a large fluctuation in force in the visualized motion trajectory graph, it may indicate that the lost strokes are caused by poor contact between the pen tip and the paper or unstable writing force of the user. Different causes of lost strokes correspond to different abnormal writing features. In order to improve the accuracy of determining the cause of target lost strokes, determining the cause of target lost strokes based on the visualized motion trajectory graph may specifically include: Identify abnormal writing features and the real-time abnormal feature value corresponding to each abnormal writing feature from the visualized motion trajectory diagram; determine the real-time abnormal feature proportion based on all abnormal writing features and the real-time abnormal feature value corresponding to each abnormal writing feature; determine the target lost stroke cause corresponding to the real-time abnormal feature proportion based on the real-time abnormal feature proportion and the preset lost stroke mapping relationship, and the preset lost stroke mapping relationship is the correspondence between the real-time abnormal feature proportion and the target lost stroke cause.

[0035] Specifically, the abnormal writing feature can be a motion trajectory with discontinuity, breakpoints or sudden changes in direction, a hand grip pressure that exceeds a preset stable pressure range, or a handwriting with a clarity lower than a preset standard clarity, etc., wherein the preset stable pressure range and the preset standard clarity are not specifically limited in the embodiments of the present application. The real-time abnormal feature value can reflect the abnormal degree of the corresponding real-time abnormal writing feature. When determining the real-time abnormal feature value corresponding to the real-time abnormal writing feature, the visual motion trajectory diagram can be imported into the preset feature value recognition model. The preset feature value recognition model can recognize the real-time abnormal feature values ​​corresponding to all real-time abnormal writing features. For example, it is known that the natural extension and inflection point of the lines or curves in the visual motion trajectory diagram can accurately reflect the direction change in the actual writing process. Based on the preset feature value recognition model, the bending amplitude, duration and bending direction of the lines or curves can be identified from the visual motion trajectory diagram, and the corresponding real-time abnormal feature value can be output based on this. The preset feature value recognition model can be trained based on a large number of sample data sets containing abnormal writing features and annotated abnormal writing feature values. The specific training process is not limited in the embodiments of the present application. Since the writing process of the user may be continuous, the required writing data is collected and analyzed in real time.

[0036] Based on the preset feature value recognition model, the real-time abnormal feature values ​​corresponding to all abnormal writing features can be determined, and all abnormal writing features and their corresponding real-time abnormal feature values ​​are integrated to determine the real-time abnormal feature ratio. For example, for the existing abnormal writing features A, B and C, the real-time abnormal feature ratio is A:B:C=1:2:2. At this time, the corresponding target lost stroke cause can be determined based on the preset lost stroke mapping relationship, wherein the preset lost stroke mapping relationship is the correspondence between the real-time abnormal feature ratio and the target lost stroke cause. The specific content of the mapping relationship is not specifically limited in the embodiments of the present application, and can be determined by relevant staff based on historical experimental data and uploaded to the detection system. The target lost stroke cause is determined after considering the abnormal writing features in multiple dimensions, so as to improve the accuracy and reliability of determining the target lost stroke cause.

[0037] After determining the target stroke loss reason, the target stroke loss reason is fed back. Since the writing data is updated as the user writes, the target stroke loss reason may also be added and changed.

[0038] For the embodiment of the present application, by using a trained lightweight convolutional network model to identify the real-time writing features contained in the real-time writing data, it is convenient to improve the accuracy of determining the real-time writing features. In addition, since the visualized motion trajectory diagram can intuitively display the dynamic changes in the writing process, by visualizing the real-time writing features, it is convenient to more accurately determine the cause of lost strokes. Through the automated data processing and writing feature recognition process, the possibility of human intervention is reduced, thereby facilitating the reduction of misjudgments and delays caused by human factors. Moreover, through the instant feedback mechanism, it is convenient for users to understand the reasons for lost strokes at the first time and take corresponding corrective measures, thereby facilitating the improvement of the timeliness and accuracy of the entire detection process.

[0039] Furthermore, in order to improve the comprehensiveness of the analysis of the reasons for lost strokes, when the number of abnormal writing features in the visualized motion trajectory graph is higher than a preset abnormal threshold, the method provided in the embodiment of the present application further includes: Based on each abnormal writing feature and the real-time abnormal feature value corresponding to each abnormal writing feature, a real-time change display graph of the abnormal feature is determined, and the abnormal feature change display graph includes an abnormal feature value change bar corresponding to each abnormal writing feature; based on each abnormal writing feature in the real-time change display graph, multiple feature combinations are determined, and according to the real-time abnormal feature value corresponding to each abnormal writing feature in each feature combination, the predicted cause of lost strokes is determined; the determination moment of each predicted cause of lost strokes is recorded, and all predicted causes of lost strokes are sorted based on each determination moment to form a list of causes of lost strokes.

[0040] Specifically, the preset abnormal threshold value can be 4 or 5. The specific value is not specifically limited in the embodiment of the present application. When the number of abnormalities of the abnormal writing feature is higher than the preset abnormal threshold value, the changes and developments of multiple abnormal writing features in the continuous writing process can be more intuitively displayed by drawing a real-time change display diagram of the abnormal features. The abnormal feature change display diagram includes a display coordinate system and abnormal feature value change bars corresponding to each abnormal writing feature. The display coordinate system includes a time axis and an abnormal feature value change axis. Each abnormal feature value change bar is used to characterize the changes in the corresponding abnormal writing feature during the continuous writing process. As the real-time abnormal feature value increases, the corresponding abnormal feature value change bar will gradually lengthen. By integrating each abnormal writing feature and its corresponding real-time abnormal feature value into the abnormal feature real-time change display diagram, it is convenient to timely monitor abnormal conditions in the writing process.

[0041] Different abnormal writing features can form multiple feature combinations. Different feature combinations and the proportion of combined features corresponding to the feature combinations may have different corresponding predicted reasons for lost strokes. The proportion of combined features is the proportion of features between each abnormal writing feature in the feature combination. For example, when the proportion of combined features is A:B:C=1:2:2, the corresponding predicted reason for lost strokes may be reason 1; when the proportion of combined features is A:B:D=1:2:2, the corresponding predicted reason for lost strokes may be reason 2; when the proportion of combined features is A:C:D=1:3:2, the corresponding predicted reason for lost strokes may be reason 3.

[0042] Since different predicted reasons for lost strokes correspond to different abnormal writing features, and the change rates of real-time abnormal feature values ​​of different abnormal writing features are different, the determination moments of different predicted reasons for lost strokes are also different. The corresponding predicted reasons for lost strokes can be sorted based on each determination moment, and all predicted reasons for lost strokes in the generated list of lost strokes are sorted in chronological order. Combining the various abnormal writing features contained in the real-time change display diagram facilitates consideration of the interaction between different abnormal writing features, thereby facilitating the comprehensiveness and depth of the analysis of the reasons for lost strokes, and also facilitating the avoidance of one-sided analysis due to a single abnormal writing feature.

[0043] In order to avoid the situation where the abnormal characteristic value bar cannot be displayed, the method provided in the embodiment of the present application further includes: Identify the display limit corresponding to the real-time change display graph of abnormal features and the display length of each feature value change bar, and determine the abnormal feature value change bar whose display length is greater than the display limit as an abnormal display change bar; when the real-time change display graph of abnormal features contains abnormal display change bars, determine the scaling ratio based on the real-time abnormal feature values ​​corresponding to each abnormal writing feature in the real-time change display graph of abnormal features; scale the abnormal feature value bars corresponding to each abnormal writing feature based on the scaling ratio to obtain an updated real-time change display graph of abnormal features.

[0044] Specifically, the display limit corresponding to the display diagram of the real-time change of abnormal features can be set according to its own needs. For example, when the display limit is 10, the characteristic value change bar can be shrunk when the display length of the characteristic value change bar grows to the display limit. At this time, the abnormal characteristic value change bar whose display length is greater than the display limit can be determined as an abnormal display change bar. When shrinking the abnormal display change bar, it is necessary to shrink all the abnormal characteristic value change bars in the real-time change display diagram of the abnormal features at the same time. When the abnormal display change bar is included in the real-time change display diagram of the abnormal features, the display length corresponding to all the abnormal characteristic value change bars can be identified, and the scaling ratio can be determined based on all the display lengths. Specifically, the display length of all the abnormal characteristic value change bars can be calculated by the average value length, and then the corresponding scaling ratio can be determined based on the average value length. For example, the average value length is 5, and the corresponding scaling ratio is 1 / 5, that is, each abnormal characteristic value change bar in the real-time change display diagram is reduced by 5 times. The specific method of determining the scaling ratio is not specifically limited in the embodiment of the present application.

[0045] Determine the feature parameter combination corresponding to each predicted cause of lost strokes from the updated real-time change display diagram of abnormal features, the feature parameter combination including the abnormal writing feature corresponding to the predicted cause of lost strokes and the abnormal feature value corresponding to each abnormal writing feature; mark the change bar area corresponding to each feature parameter combination from the updated real-time change display diagram of abnormal features, and superimpose the corresponding predicted cause of lost strokes.

[0046] Specifically, different predicted stroke loss reasons correspond to different feature parameter combinations. Since the predicted stroke loss reason has been determined at this time, the abnormal feature value contained in the feature parameter combination corresponding to the predicted stroke loss reason is a fixed value, which can be selected from the updated abnormal feature real-time change display diagram, such as Figure 2 As shown, Figure 2 There is an overlapping area in the abnormal characteristic value change bars selected by the predicted lost stroke reason 1 and the predicted lost stroke reason 2. The selection border can be a solid line or a dotted line. The specific form of the selection border is not specifically limited in the embodiment of the present application. As long as the selection and the predicted lost stroke reason are superimposed, the reason for the user's lost strokes during the writing process can be intuitively viewed.

[0047] When the target stroke loss is caused by hand occlusion, the method provided in the embodiment of the present application further includes steps S210 to S240. Figure 3 As shown, where: Step S210: Acquire historical writing data without missing strokes, and determine the habitual hand-holding posture from the historical writing data without missing strokes.

[0048] Specifically, the historical writing data without lost strokes refers to the writing data without lost strokes when writing with a dot pen within a historical time period. Hand holding features are extracted from the historical writing data without lost strokes, such as the degree of finger bending, the angle between the palm and the paper, the relative position between the fingers, the holding position of the pen, etc., and then the habitual hand holding features are determined by observing the distribution of cluster centers or eigenvalues. Finally, the hand holding posture of the user in the historical writing process is simulated based on the habitual hand holding features.

[0049] Step S220: determining the pen-dropping hand holding posture and the pen-dropping hand holding pressure value corresponding to the pen-dropping stage from the real-time writing data.

[0050] Specifically, the method of determining the pen-dropping hand holding posture can refer to the method of determining the habitual hand holding posture from the historical writing data without lost strokes, which will not be elaborated here. The pen-dropping hand holding posture is also a simulated hand holding posture. The pen-dropping hand holding pressure value is the pressure applied by the hand to the dot matrix pen in the pen-dropping hand holding posture, which is composed of the contact pressures of multiple contact points.

[0051] Step S230: Match the habitual hand-holding posture with the pen-dropping hand-holding posture to determine hand-holding posture adjustment prompt information.

[0052] Step S240: When the pen-losing hand grip pressure value appears again in the real-time writing data, the hand grip posture adjustment prompt information is fed back.

[0053] Specifically, a machine learning algorithm is used to match the features of the habitual hand holding posture and the pen dropping hand holding posture to determine the key differences between the habitual hand holding posture and the pen dropping hand holding posture. These key differences may include insufficient bending of the fingers, too large or too small angle between the palm and the paper, incorrect relative position between the fingers, etc., and hand holding posture adjustment prompt information is generated based on the key differences between the hand holding postures. The hand holding posture adjustment prompt information may be "Please increase the bending of the fingers so that the fingers can hold the pen more tightly", "Please reduce the angle between the palm and the paper to keep the wrist in a natural state", or "Please adjust the relative position between the fingers to ensure that the thumb, index finger and middle finger form a stable triangular support", etc. The specific hand holding posture adjustment prompt information is not limited in the embodiments of the present application.

[0054] Through different hand-holding postures and different hand-holding pressure values, personalized hand-holding prompt information is generated to provide targeted guidance to users. This personalized feedback not only helps users correct their pen-holding posture in time, but also helps improve their writing experience and accuracy.

[0055] In order to more reasonably arrange the time and resources for tracing processing, the method provided in the embodiment of the present application also includes: When abnormal writing features are detected in the visualized motion trajectory diagram, the abnormal frequency of the abnormal writing features within a preset time period is recorded; the writing scene level is determined based on the real-time writing data; and the lost stroke detection period is determined based on the abnormal frequency and the writing scene level.

[0056] Specifically, the preset time period is a period of time after the moment when the visual motion trajectory diagram contains abnormal writing features. The duration corresponding to the preset time period can be 20 seconds or 30 seconds. The specific duration is not specifically limited in the embodiment of the present application and can be set by relevant technical personnel according to actual needs. In the embodiment of the present application, the cause of the abnormality is not immediately traced after the abnormal writing feature is detected, that is, the cause of the lost strokes is not immediately traced after the abnormal writing feature is detected. Instead, it is necessary to first analyze the abnormal frequency and writing scene level of the abnormal writing feature within the preset time period and then determine the abnormal level, and then determine the corresponding lost stroke detection period based on the abnormal level. The higher the abnormal level, the closer the corresponding lost stroke detection period is to the preset time period.

[0057] When determining the abnormality level based on the abnormal frequency and the writing scene level, the first abnormality score corresponding to the abnormal frequency can be determined based on the preset first mapping relationship, and then the second abnormality score corresponding to the writing scene level can be determined based on the writing scene level and the preset second mapping relationship. Finally, the abnormality level is determined based on the first abnormality score and the second abnormality score, wherein the preset first mapping relationship is the correspondence between the abnormal frequency and the first abnormality score, and the preset second mapping relationship is the correspondence between the writing scene level and the second abnormality score. The specific content is not specifically limited in the embodiments of the present application, and can be determined by relevant staff based on historical experimental data and uploaded to the detection system. When determining the writing scene level based on real-time writing data, the writing type can be determined based on the real-time writing data. The writing types include document signing, examinations, drafts, etc. Different writing types correspond to different writing scene levels, and there is a corresponding relationship between the writing type and the writing scene level. If the writing scene level of the dot matrix pen is high, that is, the usage scenario of the dot matrix pen has extremely high requirements for writing accuracy, such as legal documents, contract signing or important notes, then every missed stroke may affect the integrity and accuracy of the written content. In this case, it is very necessary and timely to trace the cause of the missed strokes. Therefore, the higher the writing scene level, the higher the corresponding second anomaly score.

[0058] By analyzing the abnormal frequency of abnormal writing features and the corresponding writing scene level after the abnormal writing features appear, the need for tracing the cause of lost strokes at the current moment is determined, and finally the corresponding detection period is determined based on the need, instead of immediately tracing the cause of lost strokes after identifying the abnormal writing features. This facilitates a more reasonable arrangement of time and resources for tracing processing and avoids unnecessary tension and waste of resources.

[0059] The present application provides a detection system, such as Figure 4 As shown, Figure 4 The detection system 400 shown includes: a processor 401 and a memory 403. The processor 401 and the memory 403 are connected, such as through a bus 402. Optionally, the detection system 400 may also include a transceiver 404. It should be noted that in actual applications, the transceiver 404 is not limited to one, and the structure of the detection system 400 does not constitute a limitation on the embodiments of the present application.

[0060] Processor 401 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. Processor 401 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0061] The bus 402 may include a path to transmit information between the above components. The bus 402 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 402 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 The fact that only one line is used in the diagram does not mean that there is only one bus or only one type of bus.

[0062] The memory 403 may be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0063] The memory 403 is used to store the application code for executing the solution of the present application, and the execution is controlled by the processor 401. The processor 401 is used to execute the application code stored in the memory 403 to implement the contents shown in the above method embodiment.

[0064] The detection system includes but is not limited to: mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle terminals (such as vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. It can also be a server, etc. Figure 4 The detection system shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0065] An embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer-readable storage medium is run on a computer, the computer can execute the corresponding content in the aforementioned method embodiment.

[0066] The embodiment of the present application provides a computer program product, which includes a computer program, which implements the method in any of the above embodiments when executed by a processor. Compared with the related art, the embodiment of the present application uses a trained lightweight convolutional network model to identify the real-time writing features contained in the real-time writing data, so as to improve the accuracy of determining the real-time writing features. In addition, since the visual motion trajectory diagram can intuitively display the dynamic changes in the writing process, the real-time writing features can be visualized to more accurately determine the cause of the lost strokes. Through the automated data processing and writing feature recognition process, the possibility of human intervention is reduced, thereby facilitating the reduction of misjudgments and delays caused by human factors. Moreover, through the instant feedback mechanism, it is convenient for users to understand the cause of the lost strokes at the first time and take corresponding corrective measures, thereby facilitating the improvement of the timeliness and accuracy of the entire detection process.

[0067] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0068] The above description is only a partial implementation method of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for detecting the cause of missing strokes of a dot-matrix pen based on a lightweight convolutional network, characterized in that: include: Acquire real-time writing data, and import the real-time writing data into a trained lightweight convolutional network model for feature recognition to obtain real-time writing features, wherein the real-time writing features include motion trajectory, hand grip pressure, and timestamp; Performing visualization processing on the real-time writing feature to obtain a visualized motion trajectory diagram corresponding to the real-time writing feature, wherein the visualized motion trajectory diagram includes the motion trajectory of the dot matrix pen during the writing process and the hand holding pressure at each writing moment; The reason for the target missing strokes is determined based on the visualized motion trajectory diagram, and the reason for the target missing strokes is fed back.

2. The method for detecting the cause of missing strokes of a dot matrix pen based on a lightweight convolutional network according to claim 1, characterized in that: The determining the cause of target stroke loss based on the visualized motion trajectory diagram includes: Identify abnormal writing features and real-time abnormal feature values ​​corresponding to each abnormal writing feature from the visualized motion trajectory diagram; Determine the real-time abnormal feature proportion based on all abnormal writing features and the real-time abnormal feature value corresponding to each abnormal writing feature; Based on the real-time abnormal feature proportion and the preset lost stroke mapping relationship, the target lost stroke cause corresponding to the real-time abnormal feature proportion is determined, and the preset lost stroke mapping relationship is the corresponding relationship between the real-time abnormal feature proportion and the target lost stroke cause.

3. The method for detecting the cause of missing strokes of a dot matrix pen based on a lightweight convolutional network according to claim 2, characterized in that: When the number of abnormal writing features in the visual motion trajectory graph is higher than a preset abnormal threshold, the method further includes: Based on each abnormal writing feature and the real-time abnormal feature value corresponding to each abnormal writing feature, determine a real-time abnormal feature change display diagram, wherein the abnormal feature change display diagram includes an abnormal feature value change bar corresponding to each abnormal writing feature; Determine multiple feature combinations based on each abnormal writing feature in the real-time change display diagram, and determine the predicted cause of lost strokes according to the real-time abnormal feature value corresponding to each abnormal writing feature in each feature combination; The determination time of each predicted stroke loss reason is recorded, and all predicted stroke loss reasons are sorted based on each determination time to form a stroke loss reason list.

4. The method for detecting the cause of missing strokes of a dot matrix pen based on a lightweight convolutional network according to claim 3, characterized in that: Also includes: Identify the display limit corresponding to the abnormal feature real-time change display graph and the display length of each feature value change bar, and determine the abnormal feature value change bar whose display length is greater than the display limit as an abnormal display change bar; When the abnormal feature real-time change display graph includes an abnormal display change bar, a zoom ratio is determined based on the real-time abnormal feature values ​​corresponding to each abnormal writing feature in the abnormal feature real-time change display graph; Scaling the abnormal feature value bars corresponding to each abnormal writing feature based on the scaling ratio to obtain an updated abnormal feature real-time change display diagram; Determine a feature parameter combination corresponding to each predicted stroke loss reason from the updated abnormal feature real-time change display diagram, the feature parameter combination including an abnormal writing feature corresponding to the predicted stroke loss reason and an abnormal feature value corresponding to each abnormal writing feature; The change bar area corresponding to each feature parameter combination is marked in the updated abnormal feature real-time change display diagram, and the corresponding predicted stroke loss reason is superimposed.

5. The method for detecting the cause of missing strokes of a dot matrix pen based on a lightweight convolutional network according to claim 1, characterized in that: When the target stroke loss reason is hand-holding occlusion, the method further includes: Acquire historical writing data without missing strokes, and determine the habitual hand holding posture from the historical writing data without missing strokes; Determine the pen-dropping hand holding posture and the pen-dropping hand holding pressure value corresponding to the pen-dropping stage from the real-time writing data; Matching the habitual hand holding posture with the pen-losing hand holding posture to determine hand holding posture adjustment prompt information; When the pen-losing hand-gripping pressure value appears again in the real-time writing data, the hand-gripping posture adjustment prompt information is fed back.

6. The method for detecting the cause of missing strokes of a dot matrix pen based on a lightweight convolutional network according to claim 2, characterized in that: Also includes: When it is detected that the visualized motion trajectory graph contains abnormal writing features, the abnormal frequency of the abnormal writing features within a preset time period is recorded; determining a writing scene level based on the real-time writing data; A missing stroke detection period is determined based on the abnormal frequency and the writing scene level.

7. A detection system, characterized in that: The detection system includes: at least one processor; Memory; At least one application, wherein the at least one application is stored in a memory and configured to be executed by at least one processor, and the at least one application is configured to: execute a method for detecting causes of lost strokes of a dot matrix pen based on a lightweight convolutional network as described in any one of claims 1-6.

8. A computer-readable storage medium, characterized in that: include: A computer program is stored which can be loaded by a processor and executes a method for detecting causes of lost strokes of a dot matrix pen based on a lightweight convolutional network as described in any one of claims 1-6.

9. A computer program product, characterized in that It includes a computer program, which, when executed by a processor, implements the steps of a method for detecting causes of lost strokes of a dot matrix pen based on a lightweight convolutional network as described in any one of claims 1 to 6.

Images