Point report prediction method and device and electronic equipment

Abstract

The invention provides a report point prediction method, a report point prediction device and electronic equipment. The method comprises the following steps: processing collected report point information of a plurality of historical report points to obtain feature information; the feature information is processed by adopting the posture prediction model, the prediction result related to the scene in which the touch pen executes the preset writing operation (pen lifting operation or turning operation) is output, then the number of report points is determined according to the prediction result, when the number of report points is smaller than the first preset number or equal to 0, the predicted report points originally needing to be predicted are reduced or canceled, and the prediction efficiency is improved. For the scene that the touch pen executes the preset writing operation (pen raising operation or turning operation) in a large probability, the phenomenon of line flying or stroke retraction caused by prediction of report points can be reduced, and the writing experience of a user is improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence (AI) technology, and more specifically, to a prediction method, a prediction device and an electronic device for reporting points. Background Art

[0002] With the development of touch technology, more and more electronic devices (such as tablet computers, laptops, mobile phones, etc.) use touch for human-computer interaction. In addition to finger touch operation, the display screen of the terminal device can also be operated by touch with a stylus pen, which is used for taking notes, drawing, etc. in the office.

[0003] When the stylus is writing on an electronic device such as a tablet computer, since the touch response, point calculation and drawing of the reported points require the device to process and calculate, the above process must have a certain delay. If the delay is long, the position of the drawing point on the display screen (commonly known as the water outlet position of the display screen) and the actual contact point position of the stylus on the display screen (that is, the position of the stylus tip) have a certain distance and delay, which makes the writing process have a hand-following delay, which seriously affects the user experience. Based on this, the relevant technology proposes various prediction algorithms, through which multiple reporting points can be predicted in advance, and the predicted reporting points can be drawn, which can reduce the hand-following delay in the writing process. However, if the predicted reporting points are drawn in scenarios such as lifting the pen or turning, flying lines (the water outlet position exceeds the pen tip position) or stroke retraction will occur, affecting the user's writing experience.

[0004] Based on this, it is necessary to provide a technology that can reduce the phenomenon of flying lines or stroke retraction during the writing process to improve the user experience. Summary of the invention

[0005] The embodiment of the present application provides a prediction method, prediction device and electronic device for reporting points, which processes the reporting point information of multiple historical reporting points formed when the stylus is writing to obtain feature information including at least one writing feature, uses a posture prediction model to process the feature information, and outputs a prediction result related to the scene in which the stylus performs a preset writing operation (pen lifting operation or turning operation), and then determines the number of reporting points according to the prediction result. When the determined number of reporting points is less than the first preset number or equal to 0, the reporting points that originally need to be predicted (i.e., predicted reporting points) can be reduced or cancelled. In this way, for the scene in which the stylus is likely to perform the preset writing operation (pen lifting operation or turning operation), the phenomenon of flying lines or stroke retraction caused by the prediction of reporting points can be reduced, thereby improving the user's writing experience. In addition, since the number of reporting points is associated with the preset writing operation, the number of reporting points related to the preset writing operation can be obtained, which can improve the flexibility of reporting point prediction.

[0006] In a first aspect, a method for predicting a reporting point is provided, which is applied to an electronic device, and the method comprises: obtaining reporting point information of N historical reporting points formed by a stylus pen writing on the electronic device, where N is an integer greater than 1; determining feature information based on the reporting point information, and the feature information comprises at least one of the following writing features: trajectory curvature, pressure change rate, speed change rate, and trajectory change rate, wherein the trajectory curvature is used to indicate the degree of curvature of a trajectory formed based on the N historical reporting points, the pressure change rate is used to indicate the degree of pressure change in the process of the stylus pen writing on the electronic device to form N historical reporting points, the speed change rate is used to indicate the degree of speed change in the process of the stylus pen writing on the electronic device to form N historical reporting points, and the trajectory change rate The rate is used to indicate the degree of change of the trajectory formed based on N historical reporting points compared to the trajectory formed by multiple historical reporting points obtained in the last prediction process; the posture prediction model is used to process the feature information to obtain a prediction result, and the prediction result is used to indicate the first probability of the stylus performing a preset writing operation or to indicate whether the stylus performs a preset writing operation, wherein the preset writing operation is a pen lifting operation or a turning operation; according to the prediction result, the number of reporting points is determined, and the number of reporting points is greater than or equal to 0 and less than or equal to a first preset number; when the number of reporting points is greater than 0 and less than or equal to the first preset number, the predicted reporting points of the current frame are predicted according to the number of reporting points; or, when the number of reporting points is equal to 0, the real reporting points of the current frame are drawn and displayed.

[0007] Among them, the N historical reporting points are the real historical reporting points that have been generated before the current frame is drawn (or rendered). The current frame is a frame that the electronic device is about to draw based on the writing content of the stylus. For example, the electronic device has just finished drawing the mth frame, and the stylus continues to contact the display screen. In the m+1th frame, the electronic device will continue to draw the reporting points based on the reporting points generated by the contact points. Here, the current frame is the m+1th frame.

[0008] Exemplarily, the N historical reporting points are the most recent N historical reporting points that have been generated before the current frame (eg, the m+1th frame) is drawn (or rendered).

[0009] When the number of reported points is greater than 0 and less than or equal to the first preset number, the predicted reported points of the current frame are predicted according to the number of reported points, and the real reported points and predicted reported points of the current frame are drawn and displayed. Wherein, when the number of reported points is greater than 0 and less than the first preset number, the predicted reported points of the current frame are predicted according to the number of reported points less than the first preset number; when the number of reported points is equal to the first preset number, the predicted reported points of the current frame are predicted according to the number of reported points of the first preset number.

[0010] The first preset number is a preset fixed value for predicting the reporting points. In the prior art, the electronic device always predicts the reporting points according to the first preset number, so the phenomenon of flying lines or stroke retraction may occur in the scene of turning or lifting the pen.

[0011] Among the four writing features in the above example, the curvature of the track is used to indicate the curvature of the track formed based on N historical reporting points. In implementation, if the stylus performs a turning operation (i.e., writing a curve), the curvature of the track will be larger; conversely, if the stylus does not perform a turning operation, for example, the stylus is writing a straight line, the curvature of the track will be smaller. Ideally, the curvature of the track is almost small. Therefore, the curvature of the track can, to a certain extent, indicate the probability (or trend) of the stylus performing a turning operation. In theory, if the stylus performs a turning operation, the number of predicted reporting points can be reduced or the prediction of the predicted reporting points can be canceled to reduce the phenomenon of flying lines or stroke retractions caused by prediction errors.

[0012] The pressure change rate is used to indicate the degree of pressure change in the process of the stylus pen writing on the electronic device to form N historical reporting points. In implementation, if the stylus pen performs a pen-lifting operation, the pressure will change significantly, specifically, the pressure becomes smaller. Therefore, the pressure change rate can, to a certain extent, indicate the probability (or trend) of the stylus pen performing a pen-lifting operation. Theoretically, if the stylus pen performs a pen-lifting operation, the number of predicted reporting points can be reduced or the prediction of predicted reporting points can be canceled to reduce the phenomenon of flying lines or stroke retraction caused by prediction errors.

[0013] The speed change rate is used to indicate the degree of speed change of the stylus in the process of writing on an electronic device to form N historical reporting points. In implementation, if the stylus performs a turning operation or a pen-lifting operation, the writing speed will change significantly. Specifically, the speed change rate will change significantly if the writing speed becomes faster or slower. Therefore, the speed change rate can, to a certain extent, indicate the probability (or trend) of the stylus performing a turning operation or a pen-lifting operation. In theory, a significant change in the speed change rate means a high probability that the writing speed has changed significantly, which can reduce the number of predicted reporting points or cancel the prediction of predicted reporting points, thereby reducing the phenomenon of flying lines or stroke retractions due to prediction errors.

[0014] The trajectory change rate is used to indicate the degree of change of the trajectory formed based on N historical reporting points compared to the trajectory formed by multiple historical reporting points obtained in the last prediction process. In implementation, if the stylus performs a turning operation or a pen-lifting operation, the writing speed will change significantly. Specifically, the writing speed becomes faster or slower, and the trajectory change rate will change significantly. Therefore, the trajectory change rate can, to a certain extent, represent the probability (or trend) of the stylus performing a turning operation or a pen-lifting operation. Therefore, in theory, a significant change in the trajectory change rate means a significant change in the writing speed, which can reduce the number of predicted reporting points or cancel the prediction of predicted reporting points, thereby reducing the phenomenon of flying lines or stroke retractions caused by prediction errors.

[0015] It should be understood that although both the velocity change rate and the trajectory change rate can reflect the writing speed, they have different focuses. The velocity change rate reflects the change of writing speed from a microscopic perspective, while the trajectory change rate reflects the change of writing speed from a macroscopic perspective. In some scenarios (such as scenarios where the writing speed changes frequently), the trajectory change rate can better reflect the change of writing speed from a macroscopic perspective.

[0016] The method for predicting reporting points provided in the embodiment of the present application processes the reporting point information of multiple (for example, N) historical reporting points formed when the stylus is writing to obtain feature information including at least one writing feature. These writing features can effectively characterize some state changes of the stylus performing a preset writing operation (pen lifting operation or turning operation). Therefore, the feature information is used as the input of a posture prediction model, and the posture prediction model is used to process the feature information to output a prediction result. The output prediction result can be related to the scenario in which the stylus performs the preset writing operation (pen lifting operation or turning operation), that is, the prediction result is used to indicate the first probability of the stylus performing the preset writing operation or to indicate whether the stylus performs the preset writing operation. Based on this, the number of reporting points determined according to the prediction result is related to the actual writing scenario. In this way, on the one hand, when the number of reported points is less than the first preset number or equal to 0, it means that the predicted reported points that were originally required to be predicted are reduced or cancelled. In this case, the stylus is likely to perform the preset writing operation. Therefore, reducing or canceling the predicted reported points that were originally required to be predicted can effectively reduce the phenomenon of flying lines or stroke retraction caused by the prediction of reported points, thereby improving the user's writing experience. On the other hand, when the number of reported points is equal to the first preset number, this means that the stylus is likely not to perform the preset writing operation, and the possibility of flying lines or stroke retraction itself is very small. Therefore, using the first preset number to predict the reported points can effectively reduce the hand-following delay in the writing process and improve the hand-following experience. In addition, since the number of reported points is associated with the preset writing operation, the number of reported points related to the preset writing operation can be obtained, which can improve the flexibility of the reported point prediction.

[0017] Before obtaining the point reporting information of N historical points formed by the stylus pen writing on the electronic device, that is, before executing the point reporting prediction method, the neural network model can be trained to obtain the posture prediction model. The training process is as follows: in some embodiments, a plurality of sample point reporting data are obtained, and the sample point reporting data are generated based on a plurality of historical point reporting data generated when the stylus pen writes on the electronic device. The plurality of sample point reporting data include a first type of sample point reporting data and a second type of sample point reporting data. The first type of sample point reporting data is data collected when the stylus pen performs a normal writing operation, and the second type of sample point reporting data is data collected when the stylus pen performs a preset writing operation; each sample point reporting data is processed to obtain feature information of each sample point reporting data; the feature information of each sample point reporting data is used to train the neural network model to obtain the posture prediction model.

[0018] The point reporting prediction method provided in the embodiment of the present application trains a neural network model through multiple sample point reporting data. The posture prediction model obtained after training is equivalent to establishing a relationship between the writing operation and the prediction result. Therefore, when using the posture prediction model for prediction, the feature information of the stylus when writing is used is used as input. After processing the feature information through the posture prediction model, the prediction result related to the preset writing operation can be output.

[0019] When the prediction results are different, the process of determining the number of reported points according to the prediction results is also different.

[0020] In an embodiment where the prediction result is used to indicate a first probability, in the above step of determining the number of reported points based on the prediction result, in some embodiments, it is determined whether the stylus performs a preset writing operation based on the first probability; when it is determined that the stylus performs a preset writing operation, the number of reported points is determined to be a first number or 0; or, when it is determined that the stylus does not perform a preset writing operation, the number of reported points is determined to be a first preset number.

[0021] It can be understood that the method for predicting reported points provided in the embodiment of the present application is that, under normal circumstances, it is possible to determine that the stylus pen performs a preset writing operation only when the value of the first probability is large. Therefore, it is first determined whether the stylus pen performs the preset writing operation based on the first probability. When it is determined that the stylus pen performs the preset writing operation, the number of reported points is reduced or canceled (that is, the number of reported points is the first number or 0), which is equivalent to filtering the first probability with a smaller value in advance. Therefore, the result of determining whether the stylus pen performs the preset writing operation is more consistent with the actual scene, and the judgment result is more reliable and accurate. Therefore, the number of reported points determined based on this situation is also more consistent with the actual scene. On the one hand, it can better reduce the phenomenon of flying lines or stroke retraction caused by the prediction of reported points, thereby improving the user's writing experience; on the other hand, it can also better reduce the hand-following delay in the writing process and improve the hand-following experience.

[0022] In the above step of determining that the number of reported points is a first number, in some embodiments, the first number is a second preset number.

[0023] The method for predicting reporting points provided in the embodiment of the present application directly determines the preset second preset number as the first number as the number of reporting points. It has simple logic, is easy to implement in code, is more practical, and saves processing time to a certain extent.

[0024] In the above step of determining that the number of reported points is the first number, in other embodiments, when it is determined that the stylus pen performs a preset writing operation, the number of reported points is determined to be the first number according to the first preset number and the first probability.

[0025] The method for predicting reporting points provided in the embodiment of the present application is based on a first probability indicated by a prediction result. When it is determined that the stylus performs a preset writing operation, a first number as the number of reporting points is determined according to the first probability and a first preset number. Since the number of reporting points is determined based on the probability of the predicted actual scenario, the obtained number of reporting points is more consistent with the actual scenario, that is, the obtained number of reporting points is more accurate, and the flexibility of the reporting point prediction is improved.

[0026] In some embodiments, the prediction result is also used to indicate a second probability that the stylus does not perform a preset writing operation; and, in the above step of determining whether the stylus performs a preset writing operation based on the first probability, when the first probability is greater than the second probability, it is determined that the stylus performs the preset writing operation; or, when the first probability is less than the second probability, it is determined that the stylus does not perform the preset writing operation.

[0027] The method for predicting reported points provided in the embodiment of the present application is based on the prediction result output by the posture prediction model, which is not only used to indicate the first probability, but also to indicate the second probability. The multi-dimensional output can improve the accuracy of machine learning to a certain extent, and further determines whether the stylus performs a preset writing operation based on the comparison between the first probability and the second probability. It can more accurately predict whether the actual writing operation is the preset writing operation, so that when determining that the stylus performs the preset writing operation, the number of reported points that can be determined to be less than the first preset number can better avoid the phenomenon of flying lines or stroke retraction.

[0028] In an embodiment where the prediction result is used to indicate a first probability, in the above step of determining the number of reported points based on the prediction result, in some embodiments, the number of reported points is determined based on a first preset number and a first probability.

[0029] The method for predicting reporting points provided in the embodiment of the present application does not need to determine in advance whether the stylus performs a preset writing operation, but can directly determine the number of reporting points based on the first probability and the preset first preset number. The logic is simple, the code is easy to implement, and the processing time is saved to a certain extent. In addition, since the number of reporting points is determined based on the probability of the predicted actual scene (first probability), the obtained number of reporting points is more consistent with the actual scene, that is, the accuracy of the obtained number of reporting points is higher, and the flexibility of reporting point prediction is improved.

[0030] In an embodiment where the prediction result is used to indicate whether the stylus performs a preset writing operation, in the above step of determining the number of reported points based on the prediction result, in some embodiments, when the prediction result indicates that the stylus performs a preset writing operation, the number of reported points is determined to be a second preset number or 0, and the second preset number is less than the first preset number; or, when the prediction result indicates that the stylus does not perform the preset writing operation, the number of reported points is determined to be the first preset number.

[0031] In the method for predicting reporting points provided in the embodiment of the present application, the electronic device can directly determine whether the stylus performs a preset writing operation based on the prediction result output by the posture prediction model. Since there is no need to further determine whether the stylus performs the preset writing operation based on the prediction result, the logic is simple and the code is easy to implement. Moreover, when the stylus performs the preset writing operation, the preset second preset number or 0 is determined as the number of reporting points, and no other additional formulas are required to convert the number of reporting points. The process is also easy to implement and saves processing time to a certain extent.

[0032] In an embodiment of the present application, the reporting point information of the above-mentioned N historical reporting points may include at least one of the time information of each historical reporting point, the coordinate information of each historical reporting point, and the pressure information of each historical reporting point, which is related to the determined writing features.

[0033] In some embodiments, the reporting point information includes coordinate information of each historical reporting point, and the characteristic information includes the curvature of the trajectory; and, in the above step of determining the characteristic information based on the reporting point information, exemplarily, the moving distance and the moving trajectory length are determined based on the coordinate information of each historical reporting point, the moving distance is used to represent the straight-line distance between the 1st historical reporting point and the Nth historical reporting point among N historical reporting points, the time of the 1st historical reporting point is earlier than the time of the Nth historical reporting point, and the moving trajectory length is used to represent the length of the trajectory formed based on the N historical reporting points; the trajectory curvature is determined based on the moving distance and the moving trajectory length.

[0034] The method for predicting reported points provided in the embodiment of the present application uses the trajectory curvature determined by the above-mentioned moving distance and moving trajectory length to better characterize the degree of curvature of the trajectory formed based on N historical reported points, which is beneficial to the accuracy of model training and the accuracy of the prediction results output during use.

[0035] In the above embodiment where the characteristic information includes the trajectory curvature and the trajectory curvature is obtained based on the moving distance and / or the moving trajectory length, in some embodiments, the characteristic information also includes the moving distance and / or the moving trajectory length.

[0036] In the prediction method for reporting points provided in the embodiment of the present application, in addition to considering the main features, some secondary features need to be considered in the neural network algorithm, so as to consider the factors affecting the results from multiple dimensions, and thus obtain relatively accurate data. It can be understood that the curvature of the trajectory is calculated by the moving distance and the moving trajectory length, and is input into the posture prediction model as an absolute dominant feature. However, considering that the moving distance and the moving trajectory length may be affected by other factors (for example, adding, subtracting, multiplying and dividing other values ​​to obtain the moving distance or the moving trajectory length), the curvature of the trajectory obtained based on the moving distance and the moving trajectory length may also be affected by other factors. Therefore, the relatively primitive parameters such as the moving distance and / or the moving trajectory length are also input into the posture prediction model as writing features. Since more writing features are considered, the accuracy of the prediction results can be improved.

[0037] In some embodiments, the reporting point information includes pressure information of each historical reporting point, and the characteristic information includes the pressure change rate; and, in the above-mentioned step of determining the characteristic information based on the reporting point information, exemplarily, the pressure change rate is determined based on the pressure information of at least part of the historical reporting points.

[0038] In the above-mentioned step of determining the pressure change value based on the pressure information of at least part of the historical reporting points, in some embodiments, the ratio between the maximum pressure among the pressures of N historical reporting points and the pressure of the Nth historical reporting point is determined as the pressure change rate, and the Nth historical reporting point is the latest reporting point among the N historical reporting points.

[0039] The method for predicting reporting points provided in the embodiment of the present application uses the ratio between the maximum pressure of N historical reporting points and the pressure of the Nth historical reporting point to determine the pressure change rate, that is, the pressure of the latest reporting point is compared with the maximum pressure of the N reporting points to obtain the pressure change rate, which can better characterize the degree of pressure change in the process of the stylus pen writing on the electronic device to form N historical reporting points, which is beneficial to the accuracy of the model training and the accuracy of the prediction results output during use.

[0040] In some embodiments, the reporting point information includes coordinate information and time information of each historical reporting point, and the characteristic information includes the speed change rate; and, in the above-mentioned step of determining the characteristic information based on the reporting point information, exemplarily, the speed change rate is determined based on the coordinate information and time information of at least part of the historical reporting points.

[0041] In the above-mentioned step of determining the speed change rate based on the coordinate information and time information of at least part of the historical reporting points, in some embodiments, N-1 speeds are determined based on the coordinate information and time information of two adjacent historical reporting points among the N historical reporting points; the ratio between the maximum speed among the N-1 speeds and the N-1th speed is determined as the speed change rate, and the N-1th speed is the latest speed among the N-1 speeds.

[0042] The method for predicting reporting points provided in the embodiment of the present application uses the ratio between the maximum speed among N-1 speeds and the N-1th speed to determine the speed change rate, that is, the speed change rate is obtained by comparing the latest speed with the maximum speed. This can better characterize the degree of speed change in the process of the stylus pen writing on the electronic device to form N historical reporting points, which is beneficial to the accuracy of the model training and the accuracy of the prediction results output during use.

[0043] In some embodiments, the reporting point information includes coordinate information of each historical reporting point, and the characteristic information includes a trajectory change rate; and in the above step of determining the characteristic information based on the reporting point information, exemplarily, a first moving trajectory length is determined based on the coordinate information of each historical reporting point, and the first moving trajectory length is used to represent the length of a trajectory formed based on N historical reporting points; the trajectory change rate is determined based on the first moving trajectory length and the second moving trajectory length, and the second moving trajectory length is used to represent the length of a trajectory formed based on multiple historical reporting points obtained in the last prediction process.

[0044] In the method for predicting the reporting points provided in the embodiment of the present application, the trajectory change rate is used to indicate the degree of change of the trajectory formed based on N historical reporting points compared to the trajectory formed by multiple historical reporting points obtained in the last prediction process, which actually reflects the change in the writing speed corresponding to the two previous and subsequent trajectories. The trajectory change rate determined by the above-mentioned first moving trajectory length and second moving trajectory length can better characterize the change in the writing speed corresponding to the two previous and subsequent trajectories, which is beneficial to the accuracy of the model training and the accuracy of the prediction results output during use.

[0045] In a second aspect, an electronic device is provided, and the electronic device is used to execute the method provided in the first aspect. Specifically, the electronic device may include a module for executing any possible implementation manner of the first aspect.

[0046] In a third aspect, an electronic device is provided, comprising a processor. The processor is coupled to a memory and can be used to execute instructions in the memory to implement the method in any possible implementation of the first aspect. Optionally, the electronic device also includes a memory. Optionally, the device also includes a communication interface, and the processor is coupled to the communication interface.

[0047] According to a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a device, the device implements the method in any possible implementation manner of the first aspect.

[0048] According to a fifth aspect, a computer program product comprising instructions is provided, wherein when the instructions are executed by a computer, the device implements the method in any possible implementation manner of the first aspect.

[0049] In a sixth aspect, a chip is provided, comprising: an input interface, an output interface, a processor and a memory, wherein the input interface, the output interface, the processor and the memory are connected via an internal connection path, and the processor is used to execute the code in the memory. When the code is executed, the processor is used to execute the method in any possible implementation of the first aspect above. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 The embodiments of the present application provide a communication scenario between a stylus pen and an electronic device.

[0051] Figure 2 This is a scene diagram of a user using a stylus to write on a display screen of an electronic device, provided in an embodiment of the present application.

[0052] Figure 3 It is a schematic diagram of the processing process of the electronic device when the stylus provided in the embodiment of the present application is writing.

[0053] Figure 4 It is a schematic diagram of the relationship between the stylus pen and the reporting point when the stylus pen provided in an embodiment of the present application is writing.

[0054] Figure 5 It is a schematic diagram of the flying line phenomenon when the stylus provided in an embodiment of the present application is writing in the scenario of lifting the pen or turning the pen.

[0055] Figure 6 It is a schematic diagram of the structure of the electronic device 100 provided in an embodiment of the present application.

[0056] Figure 7 It is a software structure block diagram of the electronic device 100 provided in an embodiment of the present application.

[0057] Figure 8It is a schematic flow chart of the prediction method 300 of the reporting point provided in the embodiment of the present application.

[0058] Fig. 9 It is a schematic diagram of the writing content when the stylus provided in an embodiment of the present application performs a turning operation.

[0059] Fig.10 Schematic diagram of the pressure variation curve of the stylus during writing provided by an embodiment of the present application.

[0060] Fig.11 It is a schematic diagram of the content written by the stylus provided in the embodiment of the present application at different writing speeds.

[0061] Fig.12 It is a schematic flow chart of the prediction method 400 of the reporting point provided in the embodiment of the present application.

[0062] Fig.13 It is a schematic flow chart of the prediction method 500 of the reporting point provided in the embodiment of the present application.

[0063] Fig.14 It is a schematic flow chart of the prediction method 600 of the reporting point provided in the embodiment of the present application.

[0064] Fig.15 It is a schematic flowchart of the training process 700 of the posture prediction model provided in an embodiment of the present application.

[0065] Fig.16 It is a network structure diagram of the posture prediction model provided in the embodiment of the present application.

[0066] Fig.17 This is another network structure diagram of the posture prediction model provided in an embodiment of the present application.

[0067] Fig.18 It is an exemplary block diagram of the prediction device 800 for reporting time provided in an embodiment of the present application.

[0068] Fig.19 It is a schematic structural diagram of an electronic device 900 provided in an embodiment of the present application. DETAILED DESCRIPTION

[0069] The technical solution in this application will be described below in conjunction with the accompanying drawings.

[0070] The embodiments of the present application relate to a scenario in which a user writes on an electronic device using a stylus pen. Figure 1 The embodiments of the present application provide a communication scenario between a stylus pen and an electronic device. Figure 2 This is a scene diagram of a user using a stylus to write on a display screen of an electronic device, provided in an embodiment of the present application.

[0071] refer to Figure 1 The stylus pen 200 and the electronic device 100 can be connected wirelessly or by wire. The wireless connection can be established by, for example, Bluetooth or wireless local area networks (WLAN) (such as wireless fidelity (WIFI)). After the two are connected, refer to Figure 2 The user uses the stylus pen 200 to write on the display screen 100a of the electronic device. The point where the stylus pen 200 contacts the display screen 100a is the touch point. After the electronic device 100 performs calculations based on the touch points of the stylus pen 200 on the display screen 100a, it can draw the path that the stylus pen 200 passes on the display screen 100a to display the corresponding line, which is the content drawn by the stylus pen 200.

[0072] Exemplarily, the electronic device 100 may be any device with a touch screen, such as a mobile phone, a tablet computer, a laptop computer, a wearable device, etc. The embodiment of the present application does not impose any limitation on the form of the electronic device.

[0073] The stylus 200 includes a passive stylus and an active stylus, and the present application mainly relates to an active stylus. In the following, unless otherwise specified, the stylus of the embodiments of the present application are all active stylus. During the implementation process, when the stylus 200 contacts the display screen 100a of the electronic device 100, a driving signal will be transmitted to the electronic device 100, and the electronic device 100 can determine the position of the contact point according to the change of the parameter value generated by the driving signal, thereby drawing the content written by the user on the display screen 100a according to the contact point. In some embodiments, the stylus 200 is an active capacitive stylus, then the driving signal sent by the stylus 200 is a voltage signal, and the larger the voltage signal received by the electronic device 100, the larger the capacitance value change, and the position of the contact point can be determined based on the increased capacitance value. In other embodiments, the stylus 200 is an active electromagnetic stylus, then the driving signal sent by the stylus 200 is an electromagnetic signal, the electromagnetic signal received by the electronic device 100 becomes larger, and the magnetic flux of the contact point also becomes larger, and the position of the contact point can be determined based on the increased magnetic flux.

[0074] When the stylus pen 200 writes on the display screen 100a, the writing is different from that of a normal pen on paper, which can be displayed immediately. The electronic device needs to calculate the reported points based on the contact points of the stylus pen 200 on the display screen 100a, and then draw (or render) the reported points before drawing the corresponding lines.

[0075] Figure 3 is a schematic diagram of the processing process of the electronic device when the stylus pen provided in the embodiment of the present application is writing. Figure 3After the stylus touches the display screen, the display screen needs a certain amount of time to respond; after the display screen responds, the physical layer (or bottom layer) of the electronic device calculates the report point based on the contact point, and the physical layer reports the report point to the upper layer (such as the application framework layer). This process also has a certain delay, that is, the report delay; the upper layer renders the report point according to the report point to obtain the reported point after drawing (recorded as the drawing point), and this process also requires a certain amount of processing time to complete. Finally, the drawing point is sent to the display screen for display. It can be seen that in the above process, since the touch response, report point calculation and drawing report point need to be processed and calculated by the device, the above process must have a certain delay. It can be understood that when the line drawing is completed, the actual contact point of the stylus may have continued to move backward, resulting in a certain distance and delay between the position of the drawing point of the display screen (commonly known as the water outlet position of the display screen) and the position of the actual contact point of the stylus on the display screen (that is, the pen tip position of the stylus), which makes the writing process have a hand-following delay, which seriously affects the user experience.

[0076] Figure 4 Schematic diagram of the relationship between the stylus pen and the reporting point when the stylus pen provided in the embodiment of the present application is writing. Figure 4 , the stylus has moved to the position of the actual touch point a, and the position of the drawing point c on the display screen does not overlap with the position of the actual touch point a. There is a certain distance between the two, resulting in the actual touch point of the stylus moving backward when the drawing point is displayed, resulting in a serious non-following phenomenon.

[0077] Based on this, the relevant technology has proposed various prediction algorithms, through which multiple reporting points (referred to as predicted reporting points) can be predicted in advance, and the predicted reporting points can be drawn, which can reduce the hand-following delay in the writing process. In the implementation, when each frame is drawn, the electronic device will erase the predicted reporting points of the previous frame, and at the same time calculate and draw the real reporting points and predicted reporting points of the current frame, wherein the real reporting points are the reporting points calculated based on the contact points, which are equivalent to the points where the stylus actually contacts the display screen. In this way, the display problem caused by the deviation between the position of the predicted reporting points of the previous frame and the real reporting points of the next frame can be avoided. Drawing the predicted reporting points is only to solve the hand-following delay problem of the current frame. When the real reporting points are generated when the next frame is drawn, the real reporting points are used for drawing, and the predicted reporting points of the previous frame are no longer needed, so that the display screen displays the trajectory of the real reporting points, thereby avoiding the display problem caused by the deviation between the predicted reporting points of the previous frame and the real reporting points of the next frame.

[0078] Continue to refer Figure 4 For the current frame, points b1, b2 and b3 are predicted points predicted in advance, and point b1 is the last predicted point. When drawing and displaying, not only the real points calculated based on the contact points are drawn and displayed, but also the predicted points are drawn and displayed. Figure 4In the figure, the points b1, b2 and b3 are drawn and displayed. It can be seen that the distance between the position of the point b1 and the actual contact point is reduced, which can reduce the hand tracking delay during the writing process. When drawing the next frame, the electronic device will erase the predicted points b1, b2 and b3 of the previous frame, and redraw them based on the actual points of the next frame and the predicted points of the next frame.

[0079] In the above-mentioned solution of reducing the hand-following delay in the writing process by predicting the points, if the predicted points are drawn in scenarios such as lifting the pen or turning, flying lines (the water outlet position exceeds the pen tip position) or stroke retraction may occur, affecting the user's writing experience. Figure 5 Schematic diagram of the flying line phenomenon when the stylus provided in the embodiment of the present application is writing in the scenario of lifting the pen or turning the pen. Figure 5 In (a) and (b), the stylus performs a pen lift or turn operation, but the current frame has already drawn and displayed the predicted points (the trajectory of the dotted line is the trajectory formed by the predicted points). It can be seen that the position of the drawing point on the display screen (the water outlet position) exceeds the actual contact point position (the pen tip position), resulting in a flying line phenomenon. In addition, since the next frame drawing will erase the predicted points of the previous frame, if the previous frame has a flying line phenomenon, then the next frame will also have a short stroke retraction (actually the process of erasing the predicted points of the previous frame). The above-mentioned flying line or stroke retraction phenomenon greatly affects the user's writing experience.

[0080] Based on this, in order to solve the above problem, the embodiment of the present application provides a prediction method for reporting points, which processes the reporting point information of multiple historical reporting points formed when the stylus is writing, and obtains feature information including at least one writing feature. These writing features can effectively characterize some state changes of the stylus performing a preset writing operation (pen lifting operation or turning operation), and uses a posture prediction model to process the feature information, and outputs a prediction result related to the scene in which the stylus performs the preset writing operation. For example, the prediction result is used to indicate the first probability of the stylus performing the preset writing operation or to indicate whether the stylus performs the preset writing operation. Based on this, the number of reporting points is determined according to the prediction result. When the determined number of reporting points is less than the first preset number or equal to 0, the reporting points that originally need to be predicted (i.e., predicted reporting points) can be reduced or cancelled. In this way, for the scene in which the stylus is likely to perform the preset writing operation (pen lifting operation or turning operation), the phenomenon of flying lines or stroke retraction caused by the prediction of reporting points can be reduced, thereby improving the user's writing experience. In addition, since the number of reporting points is associated with the preset writing operation, the number of reporting points related to the preset writing operation can be obtained, which can improve the flexibility of reporting point prediction.

[0081] Before describing the embodiments of the present application, a summary of the relevant terms involved in the entire text is given.

[0082] Contact point: The point where the stylus actually touches the display.

[0083] Report point: a point used for drawing and displayed on the display screen, including real report point and predicted report point.

[0084] Real reporting point: The reporting point corresponding to each contact point obtained based on the contact point calculation.

[0085] Predicted point reporting: used to predict the trajectory of the stylus after the actual point reporting. The predicted point reporting can be calculated from the actual point reporting. In the current frame of drawing, there is no corresponding touch point for the predicted point reporting.

[0086] Historical reporting points: Real historical reporting points that have been generated before the current frame is drawn can be called historical reporting points. Unless otherwise specified, the historical reporting points in the embodiments of the present application are all real reporting points.

[0087] The following, combined Figures 6 to 17 , the technical method of the embodiments of the present application is described in detail.

[0088] Figure 6 1 is a schematic diagram of the structure of the electronic device 100 provided in an embodiment of the present application. Exemplarily, the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc. It can be understood that when the electronic device 100 is a mobile phone, the electronic device 100 includes a mobile communication module 150 and a SIM card interface 195. In addition, the display screen 194 here may correspond to the above Figure 1 display screen 100a.

[0089] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0090] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0091] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0092] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0093] In some embodiments, the processor 110 may determine the position of the touch point based on a parameter value (eg, a voltage signal or an electromagnetic signal) generated by a driving signal sent by the stylus, and then draw the content written by the user on the display screen 194 based on the touch point.

[0094] In some embodiments, the processor 110 may process the information of multiple historical reporting points to obtain at least one writing feature, and use a posture prediction model to process the at least one writing feature to obtain a prediction result, the prediction result is related to the scenario in which the stylus performs a preset writing operation (pen lifting operation or turning operation), and then determine the number of reporting points based on the prediction result, so as to reduce or cancel the reporting points that originally need to be predicted (i.e., predicted reporting points) when the stylus is likely to perform the preset writing operation. In this way, the phenomenon of flying lines or stroke retraction caused by the prediction of reporting points can be reduced, thereby improving the user's writing experience. In addition, when it is determined that the number of reporting points is greater than 0 (i.e., the reporting points need to be predicted), the historical reporting points are used to determine the predicted reporting points, for example, the historical reporting points are processed based on the reporting point prediction model to obtain predicted reporting points, and the predicted reporting points are displayed on the display screen 194.

[0095] The wireless communication function of the electronic device 100 can be realized by antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor, etc. Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals.

[0096] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0097] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0098] In the embodiment of the present application, the wireless communication module 160 can receive the driving signal sent by the stylus, and the processor 110 can determine the position of the touch point according to the parameter value generated by the driving signal, and then draw the content written by the user on the display screen 194 according to the touch point. In some embodiments, the antenna 1 of the electronic device 100 is coupled with the mobile communication module 150, and the antenna 2 is coupled with the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0099] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0100] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0101] The display screen 194 also includes a display driver integrated circuit (DDIC). The DDIC is one of the main control components of the display panel. Its main function is to send functional signals and data to the display panel in the form of electrical signals. By controlling the brightness and color of the screen, image information such as letters and pictures can be displayed on the screen. As a key component connecting the processor and the display screen, the DDIC plays a very important role in image display.

[0102] In the embodiment where the driving signal sent by the stylus is a voltage signal, the voltage signal can change the electric field at the contact point, thereby changing the electrode capacitance at the contact point. For example, the DDIC of the display screen 194 can determine the position of the contact point by detecting the change in the electrode capacitance. In the embodiment where the driving signal sent by the stylus is an electromagnetic signal, for example, the electromagnetic signal interacts with the electromagnetic induction plate behind the display screen 194. When the stylus is close to the display screen 194, the induction line under the electromagnetic induction plate behind the display screen 194 changes. The DDIC of the display screen 194 receives the signal based on the antenna array in the horizontal and vertical directions, and calculates the coordinate position of the stylus through the change in magnetic flux.

[0103] The electronic device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0104] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0105] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0106] In an embodiment of the present application, specifically, the code for executing the posture prediction model is stored in the internal memory 121, and the code for processing the historical point reporting information to obtain at least one writing feature and predicting the point reporting based on the prediction results of the posture prediction model are also stored in the internal memory 121.

[0107] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0108] Among the various types of sensors of the sensor module 180, the embodiments of the present application mainly relate to the touch sensor 180K, which is also called a "touch panel". The touch sensor 180K can be arranged on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also called a "touch screen". The touch sensor 180K is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor 180K can also be arranged on the surface of the electronic device 100, which is different from the position of the display screen 194.

[0109] The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present application takes the Android system of the layered architecture as an example to exemplify the software structure of the electronic device 100.

[0110] Figure 7 1 is a software structure diagram of the electronic device 100 provided in the embodiment of the present application. The layered architecture divides the software into several layers, each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system includes a native framework layer (native), an application framework frame (framework) and an application (APP).

[0111] The local framework layer mainly includes some local services and some link libraries, etc. This layer can be implemented in C and C++ languages ​​to interact with the underlying hardware. Exemplarily, the local framework layer includes a point acquisition module, which can respond to the touch operation of the stylus pen contacting the display screen and obtain the point information of the real point collected by the display screen.

[0112] In one example, the real reported point obtained by the reported point acquisition module may include the coordinate information and time information of the reported point. The time indicated by the time information may be an absolute time, which refers to the internationally used universal time, or Greenwich Mean Time. After the reported point acquisition module obtains the reported point, it may also perform denoising on the obtained reported point and send the denoised reported point to the application framework layer.

[0113] The application framework layer includes a system server, which is used to provide various services that may be used by the application framework layer, such as touch events. The touch events are used to provide service support for events that occur on the display screen when the stylus moves on the display screen.

[0114] The application framework layer also includes multiple application programming interfaces (APIs). The application layer can obtain relevant data from the bottom layer (such as the local framework layer) by calling the API. For example, the application framework layer includes a reporting point API. The writing application obtains the real reporting point information from the reporting point acquisition module of the local framework layer by calling the reporting point API.

[0115] The application framework layer may also include some algorithms to complete any possible functions and upload data to other applications in the application layer to complete a specific task.

[0116] The application layer may include a series of application packages. The application packages may include applications such as writing applications, WLAN, Bluetooth, etc. as shown in the figure, and may also include applications such as camera, gallery, calendar, call, map, navigation, music, video, short message, etc. which are not shown in the figure.

[0117] Exemplarily, the writing application may be an application such as a memo, a drawing, or the like.

[0118] Among them, the writing application can render the reporting points (including real reporting points and predicted reporting points) and display them on the display screen.

[0119] In an embodiment of the present application, the electronic device can determine the number of reporting points based on the posture prediction model. When the number of reporting points is greater than 0, the predicted reporting points of the current frame are predicted based on the predicted reporting point model according to the number of reporting points. Here, the posture prediction model and / or the reporting point prediction model can be configured in the writing application or in the application framework layer, and the embodiment of the present application does not make any limitation. In addition, at the software level, the reporting point prediction method of the embodiment of the present application can be implemented in one or more layers of the electronic device, and the software level of the specific implementation of the reporting point prediction method of the present application is not specifically limited by the embodiment of the present application.

[0120] In some embodiments, Figure 7 As shown, the posture prediction model and the reporting point prediction model are configured in the writing application.

[0121] In one example, the writing application can cache the point reporting information of the most recent multiple (e.g., N) historical points from the acquired real point reporting information, and the writing application obtains at least one writing feature based on the point reporting information of the multiple historical points, processes the at least one writing feature using a gesture prediction model, and outputs a prediction result to determine the number of predicted points that need to be predicted. The writing application predicts the predicted points based on the cached point reporting information of the multiple historical points using a point reporting prediction model, and outputs a predicted number of points.

[0122] In another example, the reporting point acquisition module of the local framework layer can obtain the reporting point information of the most recent multiple (for example, N) historical reporting points, and the writing application can obtain the reporting point information of multiple historical reporting points by calling the API, or the application framework layer can obtain the reporting point information of the most recent multiple (for example, N) historical reporting points from the reporting point information of the real reporting points reported by the local framework layer, and the writing application obtains the reporting point information of the most recent multiple historical reporting points from the application framework layer, and then the writing application obtains at least one writing feature based on the reporting point information of the multiple historical reporting points, uses the posture prediction model to process the at least one writing feature, and outputs the prediction result to determine the number of predicted reporting points that need to be predicted. The writing application uses the reporting point prediction model to predict the reporting points based on the reporting point information of the cached multiple historical reporting points, and outputs the predicted reporting point of the number of reporting points.

[0123] In other embodiments, the posture prediction model and the predicted reporting model can also be configured in the application framework layer (not shown in the figure). Exemplarily, the application framework layer can obtain the reporting information of multiple (for example, N) recent historical reporting points from the reporting information of the real reporting points reported by the local framework layer, and the application framework layer obtains at least one writing feature based on the reporting information of the multiple historical reporting points, and uses the posture prediction model to process the at least one writing feature, and outputs a prediction result to determine the number of predicted reporting points that need to be predicted. The application framework layer can use the reporting prediction model to predict the reporting points based on the reporting information of the multiple historical reporting points obtained, and output the predicted reporting points of the number of reporting points. Furthermore, the writing application can obtain the predicted reporting points from the application framework layer for rendering.

[0124] In other embodiments, the posture prediction model and the point prediction model may also be located in different layers, and the embodiments of the present application do not impose any limitation thereto.

[0125] It should be understood that the embodiments of the present application are only illustrated using the Android system as an example. In other operating systems (such as Windows system, IOS system, etc.), as long as the functions implemented by each functional module are similar to those in the embodiments of the present application, the solutions of the present application can also be implemented.

[0126] Figure 8 300 is a schematic flow chart of the prediction method 300 of the time report provided in the embodiment of the present application. It should be understood that the execution subject of the execution method 300 can be an electronic device, or a processor or chip in the electronic device. For the convenience of description, the electronic device is used as the execution subject to illustrate the embodiment of the present application.

[0127] In step S310, the electronic device obtains information of N historical reporting points formed by writing with the stylus pen on the electronic device.

[0128] In the implementation, the writing application is turned on, and the stylus touches and slides on the display screen of the electronic device, indicating that the stylus is writing on the electronic device. Based on the writing operation of the stylus on the display screen, the electronic device can obtain the reporting point information of N historical reporting points. Exemplarily, after the stylus touches the display screen of the electronic device, the electronic device drives the display screen and obtains the touch information of the display screen, and determines the reporting point information of the reporting point through the touch information, and the electronic device obtains the reporting point information of N historical reporting points from the determined reporting points.

[0129] It should be understood that the N historical reporting points here are real historical reporting points that have been generated before the current frame is drawn (or rendered). The N historical reporting points are used to determine the number of reporting points. The current frame is a frame that the electronic device is about to draw based on the writing content of the stylus. For example, the electronic device has just finished drawing the mth frame, and the stylus continues to contact the display screen. In the m+1th frame, the electronic device will continue to draw the reporting points based on the reporting points generated by the contact points. The current frame here is the m+1th frame.

[0130] In some embodiments, N may be pre-set. For example, N may be any number such as 6, 8, 10, 15, 18, 20, 30, etc., and this application does not impose any limitation thereto.

[0131] In some embodiments, the N historical reporting points are the most recent N historical reporting points generated before the current frame (e.g., the m+1th frame) is drawn (or rendered). The N historical reporting points may include real reporting points to be drawn that have been generated in the current frame (e.g., the m+1th frame), and may also include real reporting points that have been drawn in at least one frame (e.g., the mth frame) before the current frame (e.g., the m+1th frame), depending on the actual situation. In the case of a very fast writing speed, the N historical reporting points may only include at least part of the real reporting points to be drawn in the current frame (e.g., the m+1th frame). In the case of a slow writing speed, the N historical reporting points may include not only the real reporting points to be drawn in the current frame (e.g., the m+1th frame), but may also include at least part of the real reporting points that have been drawn in at least one frame (e.g., the mth frame) before the current frame. Assume that the current frame is the 5th frame, N=10, that is, the 10 historical reporting points are the most recent 10 historical reporting points generated before the 5th frame is drawn. For example, due to the fast writing speed, 15 real reporting points have been generated in the 5th frame but the 15 real reporting points have not been drawn yet. Then, the 10 historical reporting points include the 10 real reporting points that have been generated but not drawn in the 5th frame, and the time of the 10 real reporting points is later than the time of the remaining 5 real reporting points in the 5th frame; for another example, due to the slow writing speed, only 6 real reporting points are generated in the 5th frame. Then, the 10 historical reporting points include the 6 real reporting points that have been generated but not drawn in the 5th frame, and also include the 4 real reporting points that have been drawn in the 4th frame.

[0132] In one example, when the stylus starts writing, the number of historical reporting points is relatively small. When the number of historical reporting points is less than N, the stylus gesture may not be recognized first, but subsequent steps may be performed when the number of historical reporting points meets N. In other examples, when the number of historical reporting points is less than N, subsequent steps may also be performed based on the number of historical reporting points currently acquired.

[0133] In other embodiments, N may not be preset, but may be changed at any time based on actual conditions. For example, all real reporting points that have been generated but not drawn in the current frame may be used as N historical reporting points to perform subsequent steps. In this case, N is not a preset fixed value, but a variable.

[0134] In some embodiments, the reporting point information of the above-mentioned N historical reporting points may include at least one of the time information of each historical reporting point, the coordinate information of each historical reporting point, and the pressure information of each historical reporting point, which is specifically related to the writing features that need to be determined subsequently, and is described in detail below.

[0135] In step S320, the electronic device determines characteristic information according to the reporting point information.

[0136] The characteristic information is used to indicate the state change of the stylus when writing on the electronic device. The characteristic information includes at least one writing characteristic. It should be understood that the writing characteristic in the characteristic information is any characteristic (or parameter) that can characterize the state change of the stylus when writing.

[0137] In some embodiments, the characteristic information includes at least one of the following writing characteristics: trajectory curvature, pressure change rate, speed change rate, and trajectory change rate.

[0138] In other embodiments, the feature information may further include other writing features other than the above four writing features, for example, at least one of the moving distance, moving track length, the N-1th speed among the N-1 speeds, the maximum speed among the N-1 speeds, the first moving track length, the second moving track length, etc. In this way, the prediction result obtained by using more writing features is more accurate.

[0139] The following is a detailed description of the various writing features involved in the embodiments of the present application and the process of determining each writing process based on the reporting point information.

[0140] 1. Track curvature

[0141] The track curvature is used to indicate the curvature of the track formed based on N historical reporting points.

[0142] In implementation, if the stylus performs a turning operation (i.e., writing a curve), the curvature of the trajectory will be greater; conversely, if the stylus does not perform a turning operation, for example, the stylus is writing a straight line, the curvature of the trajectory will be smaller. Ideally, the curvature of the trajectory is almost small. Therefore, the curvature of the trajectory can, to a certain extent, represent the probability (or tendency) of the stylus performing a turning operation. In theory, if the stylus writes a straight line, more predicted points can be predicted to improve the writing and following performance. If the stylus writes a curve (i.e., performs a turning operation), the number of predicted points can be reduced or the prediction of predicted points can be canceled to reduce the phenomenon of flying lines or stroke retraction caused by prediction errors.

[0143] Fig. 9 is a schematic diagram of the writing content when the stylus pen according to an embodiment of the present application performs a turning operation. Fig. 9 The implementation part is based on the trajectory formed by N historical reporting points. Since the stylus performs a turning operation, the trajectory formed by the N historical reporting points is a curve, and this curve has an obvious degree of curvature.

[0144] In the case where the characteristic information includes the curvature of the track, the reporting point information includes the coordinate information of each historical reporting point. Correspondingly, the electronic device can determine the curvature of the track according to the coordinate information of each historical reporting point.

[0145] In some embodiments, the electronic device determines the moving distance and the moving trajectory length based on the coordinate information of each historical reporting point, wherein the moving distance is used to represent the straight-line distance between the 1st historical reporting point and the Nth historical reporting point among N historical reporting points, and the time of the 1st historical reporting point is earlier than the time of the Nth historical reporting point, and the moving trajectory length is used to represent the length of the trajectory formed based on the N historical reporting points; the electronic device determines the curvature of the trajectory based on the moving distance and the moving trajectory length.

[0146] It should be understood that the time of the historical reporting point mentioned in the embodiment of the present application refers to the time when the historical reporting point is generated.

[0147] Regarding the moving distance, in one example, the moving distance can be generated based on the coordinate information of the first historical reporting point and the coordinate information of the Nth historical reporting point, and the moving distance satisfies Formula 1:

[0148]

[0149] Among them, L d Indicates the moving distance, X N Indicates the coordinate of the Nth historical report point in the X direction (i.e. the horizontal coordinate), Y N Indicates the coordinate of the Nth historical report point in the Y direction (i.e., the vertical coordinate), X 1 Indicates the horizontal coordinate of the first historical report point, Y 1Indicates the vertical coordinate of the first historical report point.

[0150] Regarding the length of the moving track, in one example, the length of the moving track can be generated based on the coordinate information of each historical reporting point, and the length of the moving track satisfies Formula 2:

[0151]

[0152] Among them, L p Indicates the length of the moving trajectory, X i Indicates the horizontal coordinate of the i-th historical report point, Y i Indicates the ordinate of the i-th historical report point, i traverses from 1 to N, X i That is X 1 , X 2 , X 3 ...X N-1 , X N , Y i That is Y 1 , Y 2 , Y 3 ...Y N-1 , Y N .

[0153] Regarding the trajectory curvature, in one example, the trajectory curvature satisfies Formula 3: Among them, L d Indicates the curvature of the trajectory, L d Less than or equal to 1.

[0154] Based on the above formulas 1 to 3, it can be seen that L d The smaller it is, the greater the curvature of the trajectory is, and the greater the probability (or tendency) of the stylus pen performing a pen-lifting operation is.

[0155] In the above embodiment where the characteristic information includes the curvature of the trajectory, in some embodiments, the characteristic information also includes the moving distance and / or the moving trajectory length. In this way, the accuracy of the prediction result can be improved.

[0156] In the embodiment of the present application, the writing features in the feature information are the input of the posture prediction model. In theory, in the neural network algorithm, in addition to considering the main features, some secondary features need to be considered in order to consider the factors affecting the results from multiple dimensions, and then obtain relatively accurate data. Considering only one, it is actually most ideal to use the original data (such as the reporting information of N historical reporting points) as the input of the posture recognition model, but this will generate more noise data. Therefore, considering comprehensively, the original data (such as the reporting information of N historical reporting points) is feature extracted, and the extracted writing features are used as the input of the posture prediction model. In this way, both the noise data can be reduced and relatively accurate results can be obtained. Based on the above considerations, it can be understood that the trajectory curvature is calculated by the moving distance and the moving trajectory length, and is input into the posture prediction model as an absolute explicit feature. However, considering that the moving distance and the moving trajectory length may be affected by other factors (for example, adding, subtracting, multiplying and dividing other values ​​to obtain the moving distance or the moving trajectory length), the trajectory curvature obtained based on the moving distance and the moving trajectory length may also be affected by other factors. Therefore, the relatively primitive parameters such as the moving distance and / or the moving trajectory length are also input into the posture prediction model as writing features. Since more writing features are taken into consideration, the accuracy of the prediction results can be improved.

[0157] 2. Pressure change rate

[0158] The pressure change rate is used to indicate the degree of pressure change when the stylus pen is writing on the electronic device to form N historical reporting points.

[0159] In implementation, if the stylus performs a pen-lifting operation, the pressure will change significantly, specifically, the pressure will decrease. When the pen is lifted slowly, the pressure changes smoothly, and the pressure will slowly change from a larger value to 0. When the pen is lifted quickly, the pressure change may be cliff-like, and the pressure may change directly from the maximum value to 0. Therefore, the pressure change rate can, to a certain extent, represent the probability (or trend) of the stylus performing a pen-lifting operation. In theory, if the stylus performs a pen-lifting operation, the number of predicted points can be reduced or the prediction of predicted points can be canceled to reduce the phenomenon of flying lines or stroke retractions caused by prediction errors.

[0160] Fig.10 Schematic diagram of the pressure variation curve of the stylus during writing provided by an embodiment of the present application. Fig.10 (a) in the figure shows the pressure change curve when the pen is lifted slowly. It can be seen that due to the slow pen lift, the pressure slowly changes from the 10th reported point, and the pressure change curve is relatively smooth. The pressure slowly changes from the maximum value to 0, and the pressure becomes 0 at about the 22nd reported point. The whole change process lasts for about 12 reported points. Fig.10(b) in the figure shows the pressure change curve when the pen is lifted quickly. It can be seen that due to the rapid lifting of the pen, the pressure drops sharply and becomes 0 starting from about the 26th reporting point, and the change process only lasts for a maximum of 1 reporting point.

[0161] In the case where the characteristic information includes the pressure change rate, the reporting point information includes pressure information of each historical reporting point. Correspondingly, the electronic device can determine the pressure change rate based on at least part of the pressure information of the historical reporting points.

[0162] The pressure information of at least some of the historical reporting points represents the pressure information of some or all of the N historical reporting points. In addition, the pressure information of the historical reporting points represents the pressure applied by the stylus when the contact point corresponding to the historical reporting point contacts the display screen.

[0163] In the above step of determining the pressure change value based on the pressure information of at least part of the historical reporting points, in some embodiments, the electronic device determines the pressure change rate as the ratio between the maximum pressure of the N historical reporting points and the pressure of the Nth historical reporting point, where the Nth historical reporting point is the latest reporting point among the N historical reporting points. In this embodiment, the pressure change value is determined based on the pressure information of all the historical reporting points among the N historical reporting points.

[0164] That is, the electronic device determines the pressure with the largest value (ie, the maximum pressure) from the pressures of N historical reporting points, compares the maximum pressure with the pressure of the latest N historical reporting point among the N historical reporting points, and obtains the pressure change rate.

[0165] Among them, the pressure change rate can be obtained by formula 4: P s Indicates the pressure change rate, which is a value less than or equal to 1, P i Indicates the pressure of the i-th historical reporting point, i ranges from 1 to N, P i That is P 1 , P 2 , P 3 ...P N .

[0166] Based on the above formula 4, it can be seen that P s The larger the value, the greater the pressure change, and the greater the probability (or tendency) of the stylus pen performing a pen-lifting operation.

[0167] It should be understood that the above-mentioned method of determining the pressure change rate is only for illustrative purposes and should not be construed as limiting the embodiments of the present application.

[0168] For example, the electronic device may also compare the pressures of any two adjacent historical reporting points to obtain N-1 pressure change rates, and determine the pressure change rate with the largest value among the N-1 pressure change rates (i.e., the maximum pressure change rate) as the pressure change rate input to the posture prediction model. In this embodiment, the electronic device determines the pressure change rate based on the pressure information of all historical reporting points in the N historical reporting points.

[0169] For another example, the electronic device may also determine the ratio between the pressure at the first historical reporting point and the pressure at the Nth historical reporting point as the pressure change rate. In this embodiment, the electronic device determines the pressure change rate based on the pressure information of some historical reporting points among the N historical reporting points.

[0170] 3. Speed ​​change rate

[0171] The speed change rate is used to indicate the degree of speed change when the stylus pen writes on the electronic device to form N historical reporting points.

[0172] In implementation, if the stylus performs a turning operation or a pen-lifting operation, the writing speed will change significantly. Specifically, the faster or slower the writing speed, the faster the speed change rate will change. Therefore, the speed change rate can, to a certain extent, represent the probability (or trend) that the stylus performs a turning operation or a pen-lifting operation. It should be noted that the spacing between the reporting points is different for different writing speeds. The faster the writing speed, the larger the spacing between the reporting points, and the slower the writing speed, the smaller the spacing between the reporting points. Therefore, in theory, a significant change in the speed change rate most likely means that the writing speed has changed significantly. If the number of predicted reporting points is large, an erroneous predicted reporting point will produce obvious flying lines or stroke retraction. Therefore, the number of predicted reporting points can be reduced or the prediction of predicted reporting points can be canceled.

[0173] In the case where the characteristic information includes the speed change rate, the reporting point information includes the coordinate information and time information of each historical reporting point, and correspondingly, the electronic device can determine the speed change rate according to the coordinate information and time information of at least part of the historical reporting points. The time information of the historical reporting point is used to indicate the time when the historical reporting point was generated.

[0174] In the above-mentioned step of determining the speed change rate based on the coordinate information and time information of at least part of the historical reporting points, in some embodiments, the electronic device determines N-1 speeds based on the coordinate information and time information of two adjacent historical reporting points among the N historical reporting points; the electronic device determines the ratio between the maximum speed among the N-1 speeds and the N-1th speed as the speed change rate, and the N-1th speed is the latest speed among the N-1 speeds.

[0175] Specifically, the electronic device obtains the i-th speed according to the coordinate information and time information of the i-th historical reporting point and the i+1-th historical reporting point, i traverses from 1 to N-1, and finally obtains N-1 speed values, for example, i=1, obtains the first speed according to the coordinate information and time information of the 1st historical reporting point and the 2nd historical reporting point, i=2, obtains the second speed according to the coordinate information and time information of the 2nd historical reporting point and the 3rd historical reporting point, and so on, i=N-1, obtains the N-1-th speed according to the coordinate information and time information of the N-1-th historical reporting point and the N-th historical reporting point, thereby obtaining N-1 speeds. The electronic device determines the speed with the largest value (i.e., the maximum speed) from the N-1 speeds, and determines the ratio between the maximum speed and the N-1-th speed as the speed change rate.

[0176] Exemplarily, the i-th speed among the N-1 speeds satisfies Formula 5:

[0177]

[0178] Among them, V i represents the ith speed, V xi represents the speed of the ith velocity in the x direction, V yi represents the speed of the i-th velocity in the y direction, X i+1 Indicates the coordinates of the i+1th historical report point in the X direction, Y i+1 Indicates the coordinate of the i+1th historical report point in the Y direction, T i+1 Indicates the time when the i+1th historical report point is generated, X i Indicates the coordinate of the i-th historical report point in the X direction, Y i Indicates the coordinate of the i-th historical report point in the Y direction, T i Indicates the time when the i-th historical report point is generated, and the value of i ranges from 1 to N-1.

[0179] Regarding the speed change rate, based on the above formula 5, N-1 speeds are obtained, and the speed change rate satisfies formula 6:

[0180]

[0181] Among them, V N-1 Indicates the N-1th speed, V max Indicates the maximum speed among N-1 speeds.

[0182] It should be understood that the above-mentioned method of determining the speed change rate is only for illustrative purposes and should not be construed as limiting the embodiments of the present application.

[0183] For example, the electronic device may also compare the speed forces of any two adjacent historical reporting points to obtain N-1 speed change rates, and determine the speed change rate with the largest value among the N-1 speed change rates (i.e., the maximum speed change rate) as the speed change rate input to the posture prediction model. In this embodiment, the electronic device determines the speed change rate based on the coordinate information and time information of all historical reporting points in the N historical reporting points.

[0184] For another example, the electronic device may also determine the ratio between the speed of the first historical reporting point and the speed of the Nth historical reporting point as the speed change rate. In this embodiment, the electronic device determines the speed change rate based on the coordinate information and time information of some historical reporting points in the N historical reporting points.

[0185] In the above-mentioned embodiment where the characteristic information includes the speed change rate, in some embodiments, the characteristic information may also include the N-1th speed and / or the maximum speed among the N-1th speeds. It is understandable that the speed change rate is calculated by the maximum speed among the N-1th speed and the N-1th speed, and is input into the posture prediction model as an absolute dominant feature. However, considering that the N-1th speed and the maximum speed among the N-1th speeds may be affected by other factors, the speed change rate obtained based on the N-1th speed and the maximum speed among the N-1th speeds may also be affected by other factors. Therefore, the relatively original parameters such as the N-1th speed and the maximum speed among the N-1th speeds are also input into the posture prediction model as writing features. Since more writing features are taken into consideration, the accuracy of the prediction results can be improved.

[0186] 4. Trajectory Change Rate

[0187] The trajectory change rate is used to indicate the degree of change of the trajectory formed based on N historical reporting points compared to the trajectory formed by multiple historical reporting points obtained in the last prediction process.

[0188] Among them, the multiple historical reporting points obtained in the last prediction process are the N historical reporting points obtained before the last frame was drawn in the last prediction process and used to determine the number of predicted reporting points. Specifically, the feature information determined by the N historical reporting points obtained in the last prediction process is used to predict whether the stylus performs a pen lift operation or a turn operation, and the number of reporting points is determined based on the prediction results to determine the predicted reporting points. It should be understood that since the time interval between adjacent reporting points is the same, the duration corresponding to the N historical reporting points obtained in each prediction process is the same. For example, the time interval between adjacent reporting points is 1ms, N=10, then the duration corresponding to the 10 historical reporting points obtained in the last prediction process is 10ms, and the duration corresponding to the 10 historical reporting points obtained in the current prediction process is also 10ms.

[0189] It should be understood that for the same number (all N) of historical reporting points, if the writing speed is fast and the spacing between the reporting points is large, the length of the track formed by the N historical reporting points will be longer; if the writing speed is slow and the spacing between the reporting points is small, the length of the track formed by the N historical reporting points will be shorter. Fig.11 Schematic diagram of the content written by the stylus pen at different writing speeds provided by the embodiment of the present application. Assume that N=6, Fig.11 (a) shows the trajectory formed by the six historical reporting points when writing at a fast speed. Fig.11 (b) shows the trajectory of the six historical reporting points when writing at a slow speed. Obviously, Fig.11 The length of the trajectory corresponding to (a) is Fig.11 The length of the trajectory corresponding to (b) is longer. Fig.11 (a) is the trajectory formed by the six historical forecast points obtained in the last forecast process and Fig.11 (b) in the figure is the trajectory formed by the 6 historical reporting points obtained in the current prediction process. The length of the trajectory formed by the 6 historical reporting points obtained in the current prediction process is longer than the length of the trajectory formed by the 6 historical reporting points obtained in the previous prediction process, which means that the writing speed corresponding to the 6 historical reporting points obtained in the current prediction process is faster as a whole. Therefore, the degree of change in the trajectory formed by the N historical reporting points obtained in the two prediction processes actually reflects the change in the writing speed corresponding to the trajectory formed by the N historical reporting points obtained in the two prediction processes, that is, the change in the writing speed corresponding to the two trajectories. In implementation, if the stylus performs a turning operation or a pen-lifting operation, the writing speed will change significantly. Specifically, the faster or slower the writing speed, the change in the trajectory change rate will result in a change in the trajectory change rate. Therefore, the trajectory change rate can, to a certain extent, represent the probability (or trend) of the stylus performing a turning operation or a pen-lifting operation.

[0190] It should be pointed out that although both the velocity change rate and the trajectory change rate can reflect the writing speed, they have different focuses. The velocity change rate reflects the change of writing speed from a microscopic perspective, while the trajectory change rate reflects the change of writing speed from a macroscopic perspective. In some scenarios (such as scenarios where the writing speed changes frequently), the trajectory change rate can better reflect the change of writing speed from a macroscopic perspective. For example, the writing speeds corresponding to the 10 historical reporting points obtained in the last prediction process are constantly changing, such as fast-slow-fast-slow, and the length of the trajectory formed is 10 mm. The writing speeds corresponding to the 10 historical reporting points obtained in the current prediction process are also constantly changing, such as fast-slow-fast, and the length of the trajectory formed is 12 mm. If the rate of change is used to reflect the writing speed, the frequently changing speeds corresponding to the two predictions are not easy or cannot objectively reflect the writing speed. However, if the trajectory change rate is used to reflect the writing speed, regardless of whether the frequency of change of the writing speeds corresponding to the 10 historical reporting points obtained in the two prediction processes is frequent, the degree of change of the trajectory formed by the 10 historical reporting points obtained in the two prediction processes can determine as a whole that the writing speeds corresponding to the 10 historical reporting points obtained in the current prediction process are fast. This judgment result is very macroscopic and relatively objective.

[0191] In the case where the characteristic information includes the trajectory change rate, the reporting point information includes coordinate information of each historical reporting point, and correspondingly, the electronic device determines the trajectory change rate according to the coordinate information of each historical reporting point.

[0192] In the above-mentioned step of determining the trajectory change rate according to the coordinate information of each historical reporting point, in some embodiments, the electronic device determines a first moving trajectory length according to the coordinate information of each historical reporting point, and the first moving trajectory length is used to represent the length of the trajectory formed based on N historical reporting points; the electronic device determines the trajectory change rate according to the first moving trajectory length and the second moving trajectory length, and the second moving trajectory length is used to represent the length of the trajectory formed based on multiple historical reporting points obtained in the last prediction process.

[0193]

[0194] I will not go into details here.

[0195] In the above-mentioned embodiment where the feature information includes the trajectory change rate, in some embodiments, the feature information may also include the first moving trajectory length and / or the second moving trajectory length. It can be understood that the trajectory change rate is calculated by the first moving trajectory length and the second moving trajectory length, and is input into the posture prediction model as an absolute dominant feature. However, considering that the first moving trajectory length and the second moving trajectory length may be affected by other factors, the trajectory change rate obtained based on the first moving trajectory length and the second moving trajectory length may also be affected by other factors. Therefore, the first moving trajectory length and the second moving trajectory length, which are relatively primitive parameters, are also input into the posture prediction model as writing features. Since more writing features are taken into account, the accuracy of the prediction results can be improved.

[0196] In step S330, the electronic device processes the feature information using the posture prediction model to obtain a prediction result.

[0197] The prediction result is used to indicate a first probability that the stylus pen performs a preset writing operation or to indicate whether the stylus pen performs a preset writing operation, where the preset writing operation is a pen-lifting operation or a turning operation.

[0198] The posture prediction model can be obtained by pre-training a neural network model with feature information of each sample data in a plurality of sample point reporting data, wherein each sample data is generated based on a plurality of (for example, N) historical point reporting data generated when the stylus is writing on an electronic device, and feature information of each sample point reporting data is obtained by processing each sample point reporting data, and the feature information of each sample point reporting data is used to train the neural network model to obtain the posture prediction model.

[0199] It should be noted that the above-mentioned multiple sample point reporting data include two types of sample point reporting data. The first type of sample point reporting data includes data collected when the stylus performs normal writing operations, and the second type of sample data includes data collected when the stylus performs preset writing operations. In this way, the posture prediction model obtained after training is equivalent to establishing a relationship between the writing operation and the prediction result. Therefore, when the posture prediction model is used for prediction, the characteristic information of the stylus when writing is input, and the relevant results of the stylus performing the preset writing operation (such as the first probability of performing the preset writing operation or whether to perform the preset writing operation) can be output. It should be understood that the normal writing operation can be any writing operation other than the preset writing operation, for example, a straight line writing operation.

[0200] In addition, the neural network model used for training can be any form of neural network model, and the embodiments of the present application do not impose any limitations on this. For example, the neural network model can be a deep neural network (DNN) model.

[0201] In implementation, the feature information of each sample reporting data can be used to train the neural network model using an electronic device or other devices other than the electronic device (such as a server) to obtain a posture prediction model, and then the posture prediction model can be deployed in the electronic device. When in use, the user opens the writing application installed on the electronic device to write, and the user's writing operation or the user's operation of opening the writing application is detected. The electronic device starts a thread to load the posture prediction model. After the electronic device obtains feature information based on the reporting information of N historical reporting points, the feature information is input into the loaded posture prediction model, and the feature information is processed by the posture prediction model, and the output is used for prediction results.

[0202] In step S340, the electronic device determines the number of reporting points according to the prediction result.

[0203] The number of reported points is greater than or equal to 0 and less than or equal to a first preset number, and the first preset number is a preset fixed value for reporting point prediction. In the prior art, the electronic device always predicts the reported points according to the first preset number, that is, determines the first preset number of predicted reported points. It should be understood that the first preset number can be any value greater than 0, for example, the first preset number can be 6, 10, 12, 15, 18, 20, 30, etc.

[0204] When the number of reported points is greater than 0, the electronic device executes step S351; when the number of reported points is equal to 0, the electronic device executes step S352.

[0205] In this step, based on the different contents indicated by the prediction results, the process of the electronic device determining the number of reporting points according to the prediction results is also different. In the case where the prediction result indicates the first probability that the stylus performs a preset writing operation, the electronic device determines the number of reporting points according to the first probability. For a specific description, please refer to the relevant descriptions of methods 400 and 500 below, which will not be repeated here. In the case where the prediction result indicates whether the stylus performs a preset writing operation, the electronic device can determine the number of reporting points according to a preset rule. For a specific description, please refer to the relevant description of method 600 below, which will not be repeated here.

[0206] In step S351, when the number of reporting points is greater than 0 and less than or equal to the first preset number, the electronic device predicts the predicted reporting points of the current frame according to the number of reporting points.

[0207] It should be understood that the current frame refers to the frame currently to be drawn.

[0208] In the process of predicting the predicted reporting point of the current frame, in some embodiments, the electronic device can continue to use the reporting point information of the above N historical reporting points to predict the predicted reporting point to obtain the predicted reporting point number. Currently, in other embodiments, the electronic device can also use less than or more than N historical reporting point information to predict the reporting point, and the embodiments of the present application do not make any limitation.

[0209] After determining the predicted reporting points of the reporting point quantity, the electronic device draws and displays the real reporting points and predicted reporting points of the current frame. Here, drawing and displaying the real reporting points and predicted reporting points of the current frame means drawing the real reporting points and predicted reporting points of the current frame, and displaying the drawn real reporting points and predicted reporting points.

[0210] When the number of reported points is equal to the first preset number, it means that the probability of the stylus pen performing the preset writing operation (pen lifting operation or turning operation) is very small, then the electronic device can predict the predicted reported points of the current frame according to the first preset number, that is, determine the first preset number of predicted reported points. In this way, when the stylus pen is unlikely to perform the preset writing operation (pen lifting operation or turning operation), there will be basically no flying line or stroke retraction phenomenon, and the hand tracking delay in the writing process can be effectively reduced, improving the hand tracking experience.

[0211] When the number of reported points is greater than 0 and less than the first preset number, it means that the probability of the stylus pen performing a preset writing operation (pen lifting operation or turning operation) is relatively high, and the number of reported points can be reduced, that is, the predicted reported points of the current frame are predicted according to the number of reported points less than the first preset number. Since the number of reported points is reduced, the flying line or stroke retraction phenomenon caused by the stylus pen performing the preset writing operation (pen lifting operation or turning operation) can be effectively reduced.

[0212] In step S352, when the number of reporting points is equal to 0, the electronic device draws and displays the real reporting points of the current frame.

[0213] When the number of reported points is equal to 0, it is equivalent to canceling the prediction of predicted reported points, that is, the electronic device no longer predicts the predicted reported points, but draws and displays the real reported points of the current frame, which can effectively reduce the flying line or stroke retraction phenomenon caused by the stylus performing preset writing operations (pen lifting operations or turning operations).

[0214] As mentioned above, the process of determining the number of report points according to the prediction result of the electronic device is different depending on the content indicated by the prediction result. The following is a detailed description of the process of determining the number of report points according to the prediction result in combination with methods 400 to 600 in different cases (case 1 and case 2). Among them, case 1 represents the case where the prediction result is used to indicate the first probability of the stylus performing a preset writing operation. Methods 400 and 500 mainly describe the process of determining the number of report points in case 1. Case 2 represents the case where the prediction result is used to indicate whether the stylus performs a preset writing operation. Method 600 mainly describes the process of determining the number of report points in case 2. Similarly, an electronic device is used as the execution subject for exemplary description.

[0215] Case 1: The prediction result is used to indicate the first probability that the stylus performs a preset writing operation

[0216] Fig.12 4 is a schematic flow chart of a prediction method 400 for reporting points provided in an embodiment of the present application. In the process of determining the number of reporting points according to the prediction result in method 400, the key point is that the electronic device first determines whether the stylus performs a preset writing operation according to the first probability indicated by the prediction result, and only determines the number of reporting points when it is determined that the stylus performs the preset writing operation.

[0217] In step S410, the electronic device stylus pen writes on the electronic device to form N historical reporting point information.

[0218] In step S420, the electronic device determines characteristic information according to the reporting point information.

[0219] In step S430, the electronic device processes the feature information using the posture prediction model to obtain a prediction result, wherein the prediction result is used to indicate a first probability that the stylus pen performs a preset writing operation.

[0220] For the detailed description of step S410 to step S430 , please refer to the above description of step S310 to step S330 , which will not be repeated here.

[0221] In step S441, the electronic device determines whether the stylus pen performs a preset writing operation according to the first probability.

[0222] That is, the electronic device first determines whether the stylus pen performs a preset writing operation according to the first probability, and only determines the number of reported points when it is determined that the stylus pen performs the preset writing operation.

[0223] In some embodiments, the electronic device may compare the first probability with a preset probability, and determine that the stylus performs a preset writing operation when the first probability is greater than the preset probability, and determine that the stylus does not perform the preset writing operation when the first probability is less than the preset probability.

[0224] The preset probability may be a preset value, which is used as a reference for determining the preset writing operation. For example, the preset probability is 0.5, and when the first probability is greater than 0.5, it can be considered that the stylus performs the preset writing operation.

[0225] It should be noted that, when the first probability is equal to the preset probability, it can be classified as the result of the stylus pen performing the preset writing operation, or it can be classified as the result of the stylus pen not performing the preset writing operation, and no limitation is made here.

[0226] In other embodiments, the electronic device may compare the first probability with a second probability indicated by the prediction result, where the second probability is a variable and depends on the predicted results based on different writing situations.

[0227] Specifically, the prediction result is also used to indicate a second probability that the stylus does not perform the preset writing operation, and, in step S3412, when the first probability is greater than the second probability, the electronic device determines that the stylus performs the preset writing operation; when the first probability is less than the second probability, it determines that the stylus does not perform the preset writing operation.

[0228] That is to say, after the electronic device processes the feature information through the posture prediction model, the output prediction result is used to indicate the first probability and the second probability, and the electronic device compares the first probability and the second probability to determine whether the electronic device performs the preset writing operation. It can be understood that the prediction result is not only used to indicate the first probability, but also to indicate the second probability. The multi-dimensional output can improve the accuracy of machine learning to a certain extent, and further determine whether the stylus performs the preset writing operation based on the comparison of the first probability and the second probability, and can more accurately predict whether the actual writing operation is the preset writing operation, so that when determining that the stylus performs the preset writing operation, the number of reported points that is less than the first preset number can better avoid the phenomenon of flying lines or stroke retraction.

[0229] It should be noted that, for the case where the first probability is equal to the second probability, it can be classified as the result of the stylus performing a preset writing operation, or it can be classified as the result of the stylus not performing the preset writing operation, and no limitation is made here. After making a judgment based on step S441, if it is determined that the stylus has not performed the preset writing operation, the electronic device executes step S4423, that is, the electronic device determines that the number of reporting points is the first preset number, based on which, the electronic device executes step S452, that is, the electronic device predicts the predicted reporting points of the current frame according to the first preset number, and then displays and draws the real reporting points and predicted reporting points of the current frame.

[0230] After judging based on step S441, if it is determined that the stylus performs the preset writing operation, the electronic device performs step S4421 or step S4422, which is subject to the design of the electronic device. For the convenience of description, the way in which the electronic device performs step S4421 to determine the number of reported points is recorded as method 1, and the way in which the electronic device performs step S4422 to determine the number of reported points is recorded as method 2. Methods 1 and 2 are described in detail below.

[0231] Method 1

[0232] In step S4421, when it is determined that the stylus pen performs a preset writing operation, the electronic device determines that the number of reporting points is a first number, and the first number is less than the first preset number. Based on this, the electronic device executes step S4511, that is, the electronic device predicts the predicted reporting points of the current frame according to the first number, and then displays and draws the real reporting points and predicted reporting points of the current frame.

[0233] In the method 1, illustratively, there are two methods (denoted as method 1.1 and method 1.2) for the electronic device to determine the first number as the number of reported points, which are described in detail below.

[0234] Method 1.1

[0235] In some embodiments, the electronic device determines the second preset number as the first number, that is, the first number is the second preset number, and the second preset number is smaller than the first preset number. In other words, the first number is a preset fixed value.

[0236] It should be understood that the second preset number is any value greater than 0 and less than the first preset number. For example, the second preset number may be 6, 10, 12, 15, 18, etc.

[0237] In the implementation, a first preset number and a second preset number are set inside the electronic device. When it is determined that the stylus performs a preset writing operation, the electronic device determines the second preset number as the reporting point number and determines the predicted reporting points of the second preset number.

[0238] In the above method, the preset second preset number is directly determined as the first number as the number of reporting points. The logic is simple, the code is easy to implement, the practicability is stronger, and the processing time is saved to a certain extent.

[0239] Method 1.2

[0240] In some embodiments, the electronic device determines that the number of reported points is a first number based on a first preset number and a first probability.

[0241] In this manner, the first number is not a fixed value but a value that changes as the first probability changes.

[0242] In implementation, the electronic device can obtain the first number as the number of reported points by multiplying the first preset number and the first probability. That is, the electronic device determines the first number as the number of reported points by performing a proportional discount on the first preset number according to the first probability. For example, if the first probability is 0.8 and the first preset number is 10, then the first number is 10*0.8=8.

[0243] In the case where the product of the first preset number and the first probability is not an integer, in one example, the product of the first preset number and the first probability is rounded to obtain the first number. For example, the first probability is 0.7, the first preset number is 15, 15*0.7=10.5, and the first number is 11 by rounding. For another example, the first probability is 0.7, the first preset number is 6, 6*0.7=4.2, and the first number is 4 by rounding.

[0244] In another example, the product of the first preset number and the first probability is rounded down. For example, the first probability is 0.7, the first preset number is 15, 15*0.7=10.5, and the first number is 10 by rounding down. For another example, the first probability is 0.7, the first preset number is 6, 6*0.7=4.2, and the first number is 4 by rounding down.

[0245] In other examples, the product of the first preset number and the first probability is rounded up. For example, the first probability is 0.7, the first preset number is 15, 15*0.7=10.5, and the first number is 11 by rounding up.

[0246] In the above method, based on the first probability indicated by the prediction result, when it is determined that the stylus performs a preset writing operation, the first number as the number of reporting points is determined according to the first probability and the first preset number. Since the number of reporting points is determined based on the probability of the predicted actual scene, the obtained number of reporting points is more consistent with the actual scene, that is, the obtained number of reporting points is more accurate, and the flexibility of the reporting point prediction is improved.

[0247] Method 2

[0248] The difference between Method 2 and Method 1 is that in Method 2, when it is determined that the stylus performs a preset writing operation, in step S4422, the electronic device determines that the number of reporting points is 0, which is equivalent to canceling the prediction of the predicted reporting points, and then in step S4512 the electronic device only draws and displays the real reporting points of the current frame.

[0249] In method 400, it can be understood that, under normal circumstances, it is possible to determine that the stylus pen performs the preset writing operation only when the value of the first probability is large. Therefore, the electronic device first determines whether the stylus pen performs the preset writing operation based on the first probability. When it is determined that the stylus pen performs the preset writing operation, the number of reported points is reduced or canceled (that is, the number of reported points is the first number or 0), which is equivalent to filtering the first probability with a smaller value in advance. Therefore, the result of determining whether the stylus pen performs the preset writing operation is more consistent with the actual scene, and the judgment result is more reliable and accurate. Therefore, the number of reported points determined based on this situation is also more consistent with the actual scene. On the one hand, it can better reduce the phenomenon of flying lines or stroke retraction caused by the prediction of reported points, thereby improving the user's writing experience; on the other hand, it can also better reduce the hand-following delay in the writing process and improve the hand-following experience.

[0250] Fig.13 is a schematic flow chart of a prediction method 500 for reporting points provided in an embodiment of the present application. The main difference between method 500 and method 400 is that in method 500, the electronic device does not need to determine in advance whether the stylus performs a preset writing operation, but directly determines the number of reporting points according to the first probability indicated by the prediction result and the preset first preset number.

[0251] In step S510, the electronic device stylus writes on the electronic device and forms N historical reporting point information.

[0252] In step S520, the electronic device determines characteristic information according to the reporting point information.

[0253] In step S530, the electronic device processes the feature information using the posture prediction model to obtain a prediction result, wherein the prediction result is used to indicate a first probability that the stylus pen performs a preset writing operation.

[0254] For the detailed description of steps S510 to S530 , please refer to the above description of steps S310 to S330 , which will not be repeated here.

[0255] In step S540, the electronic device determines the number of reporting points according to the first probability and the first preset number.

[0256] There are three possible results based on the number of reported points obtained in this step: the number of reported points is equal to 0, or the number of reported points is greater than 0 and less than the first preset number, or the number of reported points is equal to the first preset number.

[0257] When the number of reported points is greater than 0 and less than or equal to the first preset number, the electronic device executes step S551; when the number of reported points is equal to 0, the electronic device executes step S552.

[0258] It can be seen that in the process of determining the number of reported points according to the first probability indicated by the prediction result, it is not necessary to determine in advance whether the stylus performs a preset writing operation in step S540, but the number of reported points is directly determined according to the first probability and the preset first preset number. In other words, the electronic device directly determines the first number as the number of reported points by performing a proportional discount on the first preset number according to the first probability.

[0259] In implementation, the electronic device can obtain the number of reported points by multiplying the first preset number and the first probability. For example, if the first probability is 0.8 and the first preset number is 10, then the number of reported points is 10*0.8=8. For another example, if the first probability is 0.2 and the first preset number is 10, then the number of reported points is 10*0.2=2.

[0260] In the case where the product of the first preset number and the first probability is not an integer, in one example, the product of the first preset number and the first probability is rounded to obtain the number of reported points. For example, the first probability is 0.7, the first preset number is 15, 15*0.7=10.5, and the number of reported points is 11 after rounding. For another example, the first probability is 0.7, the first preset number is 6, 6*0.7=4.2, and the number of reported points is 4 after rounding. For another example, the first probability is 0.04, the first preset number is 10, 10*0.04=0.4, and the number of reported points is 0 after rounding.

[0261] In another example, the product of the first preset number and the first probability is rounded down. For example, the first probability is 0.7, the first preset number is 15, 15*0.7=10.5, and the number of reported points is 10 by rounding down. For another example, the first probability is 0.1, the first preset number is 6, 6*0.1=0.6, and the number of reported points is 0 by rounding down.

[0262] It should be understood that in the above step S540, the electronic device directly determines the number of reporting points based on the first probability and the first preset number. When the number of reporting points is equal to 0, it means that the probability that the stylus performs the preset writing operation (pen lifting operation or turning operation) is the largest (such as the first probability in the above example is 0.1, the first preset number is 6, 6*0.1=0.6, and the number of reporting points is 0 by rounding down). Ideally, it can be considered that the stylus actually performs the preset writing operation (pen lifting operation or turning operation). Therefore, when the determined number of reporting points is equal to 0, it is equivalent to canceling the prediction of the predicted reporting points, that is, the electronic device no longer predicts the predicted reporting points, but draws and displays the real reporting points of the current frame, which can very effectively reduce the flying line or stroke retraction phenomenon caused by the stylus performing the preset writing operation (pen lifting operation or turning operation). When the number of reported points is greater than 0 and less than the first preset number, it means that the probability of the stylus performing the preset writing operation (pen lifting operation or turning operation) is relatively high, and the number of reported points can be appropriately reduced, that is, the predicted reported points of the current frame are predicted according to the number of reported points less than the first preset number. Since the number of reported points is reduced, the flying line or stroke retraction phenomenon caused by the stylus performing the preset writing operation (pen lifting operation or turning operation) can also be effectively reduced. When the number of reported points is equal to the first preset number, it means that the probability of the stylus performing the preset writing operation (pen lifting operation or turning operation) is very small, and the electronic device can predict the predicted reported points of the current frame according to the first preset number, that is, determine the first preset number of predicted reported points.

[0263] It should be noted that, although the method of determining the number of reported points according to the first preset number and the first probability in step S540 is similar to method 1.2 in method 400, the output results of the two are different. Since in method 1.2, it is necessary to determine in advance whether the stylus performs the preset writing operation according to the comparison of the two probabilities, the number of reported points (i.e., the first number) will be determined by the first probability and the first preset number only when it is determined that the stylus performs the preset writing operation. In this case, it is equivalent to filtering the first probability with a smaller value. Therefore, the values ​​of the first probability used to calculate the number of reported points (i.e., the first number) are all relatively large. Therefore, the possibility that the number of reported points (i.e., the first number) obtained in method 400 based on method 1.2 is 0 is very small, and is almost always greater than 0. In contrast, in method 500, since it is not necessary to determine in advance whether the stylus performs a preset writing operation based on a probability comparison, the number of reported points is determined directly based on the first probability and a proportional discount is made to the first preset number. Therefore, in this case, the value of the first probability used to calculate the number of reported points can be large or small. Therefore, in method 500, the number of reported points obtained by the first probability and the first preset number may be 0, or may be greater than 0 and less than or equal to the first preset number.

[0264] In step S551, when the number of reporting points is greater than 0 and less than or equal to the first preset number, the electronic device predicts the predicted reporting points of the current frame according to the number of reporting points.

[0265] In step S552, when the number of reporting points is equal to 0, the electronic device draws and displays the real reporting points of the current frame.

[0266] For the specific description of step S551 and step S552, please refer to the relevant description of step S351 and step S352 in the above method 300, which will not be repeated here.

[0267] In method 500, since there is no need to determine in advance whether the stylus performs a preset writing operation, the number of reporting points can be determined directly based on the first probability and the preset first preset number. The logic is simple, the code is easy to implement, and the processing time is saved to a certain extent. In addition, since the number of reporting points is determined based on the predicted probability of the actual scene (first probability), the obtained number of reporting points is more consistent with the actual scene, that is, the accuracy of the obtained number of reporting points is higher, and the flexibility of the reporting point prediction is improved.

[0268] Case 2: The prediction result is used to indicate whether the stylus performs the preset writing operation

[0269] Fig.14 6 is a schematic flow chart of a prediction method 600 for reporting points provided in an embodiment of the present application. In method 600 and method 400, although both can determine whether the stylus performs a preset writing operation, the difference between method 600 and method 400 is that, due to the different contents indicated by the prediction results, method 600 does not need to further determine whether the stylus performs a preset writing operation based on the first probability indicated by the prediction results of method 400, but can directly determine whether the stylus performs a preset writing operation based on the prediction results. Therefore, the number of reporting points can be directly determined based on the contents indicated by the prediction results.

[0270] In step S610, the electronic device stylus pen writes on the electronic device to form N historical reporting point information.

[0271] In step S620, the electronic device determines characteristic information according to the reporting point information.

[0272] For the specific description of step S610 and step S620, please refer to the above description of step S310 and step S320, which will not be repeated here.

[0273] In step S630, the electronic device processes the feature information using a gesture prediction model to obtain a prediction result, wherein the prediction result is used to indicate whether the stylus pen performs a preset writing operation.

[0274] In this step, any method may be used to indicate whether the stylus pen performs the preset writing operation, and no limitation is made here, as long as it is possible to distinguish whether the stylus pen performs the preset writing operation or does not perform the preset writing operation.

[0275] Exemplarily, "1" and "0" may be used to indicate whether the stylus performs a preset writing operation. For example, "0" is used to indicate that the stylus does not perform a preset writing operation, and "1" is used to indicate that the stylus performs a preset writing operation. For another example, "1" is used to indicate that the stylus does not perform a preset writing operation, and "0" is used to indicate that the stylus performs a preset writing operation.

[0276] When the prediction result indicates that the stylus pen has not performed the preset writing operation, the electronic device executes step S642, that is, the electronic device determines that the number of reporting points is a first preset number, based on which, the electronic device executes step S652, that is, the electronic device predicts the predicted reporting points of the current frame according to the first preset number, and then displays and draws the real reporting points and predicted reporting points of the current frame.

[0277] When the prediction result indicates that the stylus performs a preset writing operation, the electronic device may perform step S6411 or step S6412, depending on the design of the electronic device. For ease of description, the method in which the electronic device performs step S6411 to determine the number of reported points is recorded as method 2.1, and the method in which the electronic device performs step S6412 to determine the number of reported points is recorded as method 2.2. Methods 2.1 and 2.2 are described in detail below.

[0278] Method 2.1

[0279] In some embodiments, in step S6411, the electronic device determines that the number of reporting points is a second preset number, and the second preset number is less than the first preset number. Based on this, the electronic device executes step S6511, that is, the electronic device predicts the predicted reporting points of the current frame according to the second preset number, and then displays and draws the real reporting points and predicted reporting points of the current frame. The second preset number is a preset fixed value.

[0280] It should be understood that the second preset number is any preset value greater than 0 and less than the first preset number. For example, the second preset number may be 6, 10, 12, 15, 18, etc.

[0281] In the implementation, a first preset number and a second preset number are set inside the electronic device. When the prediction result indicates that the stylus performs a preset writing operation, the electronic device determines the second preset number as the reporting point number and determines the predicted reporting points of the second preset number.

[0282] Method 2.1 is the same as method 1.1 of method 400, and both methods determine the second preset number as the reporting point number.

[0283] Method 2.2

[0284] The difference between method 2.2 and method 2.1 is that in method 2.2, when the prediction result indicates that the stylus performs a preset writing operation, in step S6412, the electronic device determines that the number of reporting points is 0, which is equivalent to canceling the prediction of the predicted reporting points, and then in step S6512, the electronic device only draws and displays the actual reporting points of the current frame.

[0285] It should be understood that the prediction method of the reporting points in the above example is only for illustrative purposes and should not be construed as limiting the embodiments of the present application. Any method of determining the number of reporting points through writing characteristics during the writing process is within the protection scope of the embodiments of the present application.

[0286] In other embodiments, a comprehensive judgment may be made on at least one writing feature in the feature information. For example, when the results of each writing feature satisfy respective preset conditions, it is determined that the stylus pen performs a preset writing operation. When it is determined that the stylus pen performs a preset writing operation, the first number as the number of reported points is determined using method 1.1 or method 1.2. When it is determined that the stylus pen does not perform the preset writing operation, the first preset number is determined as the number of reported points. Exemplarily, when the characteristic information includes trajectory curvature, the preset condition of the trajectory curvature is that the trajectory curvature is less than a numerical value, and when the preset condition is met, it can be considered that the stylus performs a preset writing operation; when the characteristic information includes pressure change rate, the preset condition of the pressure change rate is that the pressure change rate is greater than a numerical value, and when the preset condition is met, it can be considered that the stylus performs a preset writing operation; when the characteristic information includes speed change rate, the preset condition of the speed change rate is that the speed change rate is greater than a numerical value or less than a numerical value, and when the preset condition is met, it can be considered that the stylus performs a preset writing operation; when the characteristic information includes trajectory change rate, the preset condition of the trajectory change rate is that the trajectory change rate is greater than a numerical value or less than a numerical value, and when the preset condition is met, it can be considered that the stylus performs a preset writing operation; when the characteristic information includes trajectory change rate, the preset condition of the trajectory change rate is that the trajectory change rate is greater than a numerical value or less than a numerical value, and when the preset condition is met, it can be considered that the stylus performs the preset writing operation; when the characteristic information includes trajectory curvature, pressure change rate, speed change rate and trajectory change rate, only when the four writing characteristics meet their respective preset conditions can it be determined that the stylus performs the preset writing operation.

[0287] The above is a detailed description of the prediction method of the reporting point in the embodiment of the present application. The following is a description of the training process of the posture prediction model.

[0288] Fig.15 7 is a schematic flow chart of a training process 700 of a posture prediction model provided in an embodiment of the present application. Continuing to use an electronic device as an example for description.

[0289] In step S710, the electronic device obtains a plurality of sample point reporting data, the sample point reporting data being generated based on a plurality of historical point reporting data generated when the stylus pen is writing on the electronic device, the plurality of sample point reporting data comprising a first category of sample point reporting data and a second category of sample point reporting data, the first category of sample point reporting data being data collected when the stylus pen performs a normal writing operation, the second category of sample point reporting data being data collected when the stylus pen performs a preset writing operation.

[0290] It should be understood that each sample reporting point data here can correspond to the reporting point information in the above-mentioned method 300, and the multiple historical reporting points of each sample reporting point data can correspond to the N historical reporting points in method 300. For the specific description of multiple historical reporting points and the reporting point information of multiple historical reporting points, please refer to the relevant description above and will not be repeated here.

[0291] It should also be understood that the normal writing operation may be any writing operation other than the preset writing operation, for example, a straight line writing operation.

[0292] In step S720, the electronic device processes each sample reporting point data to obtain characteristic information of each sample reporting point data.

[0293] For a detailed description of the process in which the electronic device processes each sample reporting point data to obtain characteristic information, reference may be made to the relevant description of step S320 in the above method 300 and will not be repeated here.

[0294] In step S730, the electronic device uses the feature information of each sample reporting data to train the neural network model to obtain a posture prediction model.

[0295] In the implementation, when the fit index (e.g., R2 score) of the neural network model is greater than a threshold (e.g., 0.95), it means that the neural network model converges and the model meets the accuracy requirements. Therefore, there is no need to train the neural network model again to obtain a posture prediction model.

[0296] It should be understood that since the multiple sample point reporting data include the first type of sample point reporting data collected when the stylus performs normal writing operations and the second type of sample point reporting data collected when the stylus performs normal writing operations, the posture prediction model obtained after training the neural network model based on the above multiple sample point reporting data is equivalent to establishing a relationship between the writing operation and the prediction result. Therefore, when using the posture prediction model for prediction, the feature information of the stylus when writing is used as input, and after processing the feature information through the posture prediction model, the prediction result related to the preset writing operation can be output.

[0297] The posture prediction model in the embodiment of the present application is obtained based on a network model in any form and is not limited here.

[0298] In some embodiments, the posture prediction model is obtained based on a deep neural network (DNN) model, wherein the DNN model includes an input layer, a hidden layer, and an output layer, and each layer includes a plurality of neurons.

[0299] The number of neurons in the input layer is the same as the number of writing features. The number of neurons in the output layer is the same as the number of contents included in the prediction result. For example, when the prediction result indicates a first probability, the number of neurons in the output layer is 1; for another example, when the prediction result indicates a first probability and a second probability, the number of neurons in the output layer is 2; for another example, when the prediction result indicates whether the stylus performs a preset writing operation, the number of neurons in the output layer is 1. The number of hidden layers is multiple, and each hidden layer includes multiple neurons.

[0300] In addition, the DNN model is also called a fully connected neural network. One of its characteristics is that the layers are fully connected, that is, all neuron nodes in the layers are connected. In other words, any neuron in the i-th layer must be connected to any neuron in the i+1-th layer.

[0301] The embodiment of the present application does not limit the number of hidden layers and the number of neurons contained in each layer.

[0302] In some embodiments, the number of hidden layers of the posture prediction model is two, denoted as a first hidden layer and a second hidden layer, and each hidden layer includes a plurality of neurons.

[0303] Fig.16 is a network structure diagram of the posture prediction model provided in the embodiment of the present application. Fig.16 , the input layer of the DNN model includes i (for example, 4 or 6) neurons, one neuron corresponds to one writing feature, x1 to xi represent the i writing features of the input layer. z10 represents the first hidden layer, and z20 represents the second hidden layer.

[0304] For the output layer of the DNN model, in an embodiment where the prediction result is used to indicate a first probability and a second probability, the output layer includes 2 neurons, one neuron corresponding to the first probability, and the other neuron corresponding to the second probability, for example, y1 represents the first probability, and y2 represents the second probability.

[0305] In an embodiment where the prediction result is used to indicate the first probability, the output layer includes 1 neuron, which corresponds to the first probability. For example, y1 represents the first probability.

[0306] In an embodiment in which the prediction result is used to predict whether the stylus performs a preset writing operation, the output layer includes 1 neuron, which corresponds to an identifier indicating whether the stylus performs the preset writing operation. For example, y1 represents an identifier of "yes" or "no". When y1 represents "yes", it means that the stylus performs the preset writing operation. When y1 represents "no", it means that the stylus does not perform the preset writing operation.

[0307] Fig.17 is another network structure diagram of the posture prediction model provided in the embodiment of the present application. As a specific example, refer to Fig.17 , the input layer includes 6 neurons, corresponding to x1 to x6, which represent the 6 writing features of the input layer, such as trajectory curvature, pressure change rate, speed change rate, trajectory change rate, moving distance and moving trajectory length; z11 to z1128 represent 128 neurons in the first hidden layer z10. It should be understood that only z11, z12, z1127, and z1128 are shown in the figure, and the remaining neurons in the middle are represented by ellipsis. Taking the neuron corresponding to x1 as an example, this neuron is connected to each neuron from z11 to z1128. Fig.17 Only the connection between this neuron and the four neurons in z10 is shown. z21 to z264 represent the 64 neurons in the second hidden layer z20. It should be understood that only z21, z22, z263, and z264 are shown in the figure, and the remaining neurons in the middle are represented by ellipsis. The output layer includes 2 neurons, one neuron corresponding to the first probability y1, and the other neuron corresponding to the second probability y2.

[0308] For a general DNN model, the input layer corresponds to multiple input features (for example, 6 writing features). The input features do not need to be processed nonlinearly by the activation function. Each feature is input to the next layer of neural network through different connections, that is, different weights w. Each neuron in the next layer takes a weighted sum of all inputs, adds a bias term, and obtains an output through the activation function, which is also the input of the next layer of neural network... and so on until it is input to the output layer. Similarly, each neuron in the output layer takes a weighted sum of all inputs, adds a bias term, and obtains an output through the activation function. The number of neurons in the input layer is determined by the number of features input to the actual problem (for example, 6 writing features), while the number of neurons in the output layer is determined by the actual problem to be solved (for example, the first probability and the second probability, or the first probability, or indicating whether the stylus performs a writing operation). The number of hidden layers in the middle and the number of neurons in each layer need to be flexibly adjusted according to the complexity of the problem to be solved. The more complex the problem, the more layers and neurons are required.

[0309] The above describes in detail the prediction method of the report point provided by the embodiment of the present application. Figure 18 to Figure 19, describing a time prediction device and an electronic device according to an embodiment of the present application.

[0310] Fig.18 800 is an exemplary block diagram of a prediction device 800 for reporting time provided in an embodiment of the present application. The prediction device 800 may be an electronic device or a chip in an electronic device. The prediction device 800 is used to execute the various processes and steps corresponding to the electronic device in the above-mentioned methods 300 to 600. The prediction device 800 includes: a processing unit 1810.

[0311] The processing unit 810 is used to perform the following steps: obtaining the reporting point information of N historical reporting points formed by the stylus pen writing on the electronic device, where N is an integer greater than 1; determining feature information based on the reporting point information, where the feature information includes at least one of the following writing features: trajectory curvature, pressure change rate, speed change rate, and trajectory change rate, wherein the trajectory curvature is used to indicate the degree of curvature of the trajectory formed based on the N historical reporting points, the pressure change rate is used to indicate the degree of pressure change in the process of the stylus pen writing on the electronic device to form the N historical reporting points, the speed change rate is used to indicate the degree of speed change in the process of the stylus pen writing on the electronic device to form the N historical reporting points, and the trajectory change rate is used to indicate the degree of speed change in the process of the stylus pen writing on the electronic device to form the N historical reporting points, and the trajectory change rate is used to indicate the degree of speed change in the process of the stylus pen writing on the electronic device to form the N historical reporting points. the degree of change of the trajectory formed by the historical reporting points compared to the trajectory formed by the multiple historical reporting points obtained in the last prediction process; using the posture prediction model to process the feature information to obtain a prediction result, the prediction result is used to indicate the first probability of the stylus performing a preset writing operation or to indicate whether the stylus performs a preset writing operation, the preset writing operation is a pen lifting operation or a turning operation; according to the prediction result, the number of reporting points is determined, and the number of reporting points is greater than or equal to 0 and less than or equal to the first preset number; when the number of reporting points is greater than 0 and less than or equal to the first preset number, predict the predicted reporting points of the current frame according to the number of reporting points; or, when the number of reporting points is equal to 0, control the display screen to draw and display the real reporting points of the current frame.

[0312] Optionally, the prediction result is used to indicate a first probability; and the processing unit 810 is specifically used to: determine whether the stylus performs a preset writing operation based on the first probability; when it is determined that the stylus performs a preset writing operation, determine the number of reported points to be a first number or 0; or, when it is determined that the stylus does not perform a preset writing operation, determine the number of reported points to be a first preset number.

[0313] Optionally, the first quantity is a second preset quantity.

[0314] Optionally, the processing unit 810 is specifically configured to: when it is determined that the stylus pen performs a preset writing operation, determine that the number of reported points is a first number according to a first preset number and a first probability.

[0315] Optionally, the prediction result is also used to indicate a second probability that the stylus pen does not perform a preset writing operation; and the processing unit 810 is specifically used to: determine that the stylus pen performs a preset writing operation when the first probability is greater than the second probability; or determine that the stylus pen does not perform the preset writing operation when the first probability is less than the second probability.

[0316] Optionally, the prediction result is used to indicate a first probability; and the processing unit 810 is specifically used to: determine the number of reporting points according to a first preset number and the first probability.

[0317] Optionally, the prediction result is used to indicate whether the stylus performs a preset writing operation; and the processing unit 810 is specifically used to: when the prediction result indicates that the stylus performs a preset writing operation, determine the number of reported points to be a second preset number or 0, and the second preset number is less than the first preset number; or, when the prediction result indicates that the stylus does not perform a preset writing operation, determine the number of reported points to be the first preset number.

[0318] Optionally, the reporting point information includes coordinate information of each historical reporting point, and the characteristic information includes the curvature of the trajectory; and the processing unit 810 is specifically used to: determine the moving distance and the length of the moving trajectory according to the coordinate information of each historical reporting point, the moving distance is used to represent the straight-line distance between the 1st historical reporting point and the Nth historical reporting point among N historical reporting points, the time of the 1st historical reporting point is earlier than the time of the Nth historical reporting point, and the moving trajectory length is used to represent the length of the trajectory formed based on the N historical reporting points; determine the curvature of the trajectory according to the moving distance and the moving trajectory length.

[0319] Optionally, the characteristic information also includes the moving distance and / or the moving track length.

[0320] Optionally, the reporting point information includes pressure information of each historical reporting point, and the characteristic information includes a pressure change rate; and the processing unit 810 is specifically used to: determine the pressure change rate according to the pressure information of at least part of the historical reporting points.

[0321] Optionally, the processing unit 810 is specifically used to: determine the ratio between the maximum pressure among the pressures of N historical reporting points and the pressure at the Nth historical reporting point as the pressure change rate, and the Nth historical reporting point is the latest reporting point among the N historical reporting points.

[0322] Optionally, the reporting point information includes coordinate information and time information of each historical reporting point, and the characteristic information includes speed change rate;

[0323] And, the processing unit 810 is specifically used to determine the speed change rate according to the coordinate information and time information of at least part of the historical reported points.

[0324] Optionally, the processing unit 810 is specifically used to: determine N-1 speeds based on the coordinate information and time information of two adjacent historical reporting points among N historical reporting points; determine the ratio between the maximum speed among the N-1 speeds and the N-1th speed as the speed change rate, and the N-1th speed is the latest speed among the N-1 speeds.

[0325] Optionally, the reporting point information includes coordinate information of each historical reporting point, and the characteristic information includes a trajectory change rate; and the processing unit 810 is specifically used to: determine a first moving trajectory length according to the coordinate information of each historical reporting point, the first moving trajectory length is used to represent the length of a trajectory formed based on N historical reporting points; determine the trajectory change rate according to the first moving trajectory length and the second moving trajectory length, the second moving trajectory length is used to represent the length of a trajectory formed based on multiple historical reporting points obtained in the last prediction process.

[0326] Optionally, the processing unit 810 is also used to: obtain multiple sample point reporting data, the sample point reporting data is generated based on multiple historical point reporting data generated when the stylus is writing on the electronic device, the multiple sample point reporting data include first-category sample point reporting data and second-category sample point reporting data, the first-category sample point reporting data is data collected when the stylus performs normal writing operations, and the second-category sample point reporting data is data collected when the stylus performs preset writing operations; process each sample point reporting data to obtain feature information of each sample point reporting data; use the feature information of each sample point reporting data to train a neural network model to obtain a posture prediction model.

[0327] It should be understood that the processing unit 810 can be used for various steps executed by the electronic device. For specific descriptions, please refer to the relevant descriptions above and will not be repeated here.

[0328] In the embodiments of the present application, Fig.18 The device may also be a chip or a chip system, such as a system on chip (SoC).

[0329] Fig.19 1 is a schematic structural diagram of an electronic device 900 provided in an embodiment of the present application. The electronic device 900 is used to execute the corresponding steps and / or processes in the above method embodiment.

[0330] The electronic device 900 includes a processor 910, a transceiver 920, and a memory 930. The processor 910, the transceiver 920, and the memory 930 communicate with each other through an internal connection path, and the processor 910 can implement the functions of the processor 910 in various possible implementations of the electronic device 900. The memory 930 is used to store instructions, and the processor 910 is used to execute the instructions stored in the memory 930, or in other words, the processor 910 can call these stored instructions to implement the functions of the processor 910 in the electronic device 900.

[0331] Optionally, the memory 930 may include a read-only memory and a random access memory, and provide instructions and data to the processor. A portion of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the device type. The processor 910 may be used to execute instructions stored in the memory, and when the processor 910 executes instructions stored in the memory, the processor 910 is used to execute the various steps and / or processes of the above-mentioned method embodiment corresponding to the network device or the terminal device.

[0332] also, Fig.19 The electronic device 900 shown may correspond to Figure 6 In the electronic device 100 shown, the processor 910 in the electronic device 900 may correspond to the processor 110 in the electronic device 100, and the transceiver 920 in the electronic device 900 may correspond to the antenna 1 and the antenna 2 in the electronic device 100. The memory 930 in the electronic device 900 may correspond to the internal memory 121 in the electronic device 100.

[0333] The electronic device 900 is used to execute the various processes and steps corresponding to the electronic devices in the above-mentioned methods 300 to method 600, and the steps executed by the processor 910 can correspond to the various steps executed by the processing unit 810 in the device 800. The specific description refers to the above description and will not be repeated here.

[0334] It should be understood that in the embodiments of the present application, the processor of the above-mentioned device may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0335] In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in a processor or an instruction in the form of software. The steps of the method disclosed in conjunction with the embodiment of the present application can be directly embodied as a hardware processor for execution, or a combination of hardware and software units in a processor for execution. The software unit can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor executes the instructions in the memory, and completes the steps of the above method in conjunction with its hardware. To avoid repetition, it is not described in detail here.

[0336] The embodiment of the present application provides a computer program product, when the computer program product is run on a terminal device, the terminal device executes the technical solution in the above embodiment. Its implementation principle and technical effect are similar to those of the above method-related embodiments, and will not be repeated here.

[0337] The embodiment of the present application provides a readable storage medium, the readable storage medium contains instructions, when the instructions are executed in a terminal device, the terminal device executes the technical solution of the above embodiment. The implementation principle and technical effect are similar, and will not be repeated here.

[0338] The embodiment of the present application provides a chip, the chip is used to execute instructions, when the chip is running, the technical solution in the above embodiment is executed. The implementation principle and technical effect are similar, and will not be repeated here.

[0339] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a high-density digital video disc (DVD)), or a semiconductor medium (eg, a solid state disk (SSD)).

[0340] It should be understood that the "embodiment" mentioned throughout the specification means that the specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, the various embodiments in the entire specification do not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the size of the sequence number of the above-mentioned processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

[0341] It should also be understood that in the present application, "when", "if" and "if" all mean that the UE or base station will take corresponding actions under certain objective circumstances. It is not a time limit, and it does not require the UE or base station to make judgments when implementing it, nor does it mean that there are other limitations.

[0342] A person skilled in the art may understand that the various numerical numbers such as first and second involved in the present application are only used for the convenience of description and are not used to limit the scope of the embodiments of the present application, and also indicate the order of precedence.

[0343] In this application, elements expressed in the singular are intended to mean "one or more" rather than "one and only one", unless otherwise specified. In this application, "at least one" is intended to mean "one or more", and "more than one" is intended to mean "two or more", unless otherwise specified.

[0344] The term "and / or" in this article is merely a description of the association relationship of associated objects, indicating that there may be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. A can be singular or plural, and B can be singular or plural.

[0345] In this document, the term "at least one of..." or "at least one of..." means all or any combination of the listed items. For example, "at least one of A, B, and C" may mean: A exists alone, B exists alone, C exists alone, A and B exist at the same time, B and C exist at the same time, and A, B, and C exist at the same time, where A may be singular or plural, B may be singular or plural, and C may be singular or plural.

[0346] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0347] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0348] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0349] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0350] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0351] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0352] The same or similar parts between the various embodiments in this application can refer to each other. In the various embodiments in this application, and the various implementation methods / implementation methods / implementation methods in each embodiment, if there is no special explanation and logical conflict, the terms and / or descriptions between different embodiments and the various implementation methods / implementation methods / implementation methods in each embodiment are consistent and can be referenced to each other. The technical features in different embodiments and the various implementation methods / implementation methods / implementation methods in each embodiment can be combined to form new embodiments, implementation methods, implementation methods, or implementation methods according to their inherent logical relationships. The above-described implementation methods of this application do not constitute a limitation on the scope of protection of this application.

[0353] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims. In short, the above is only a preferred embodiment of the technical solution of the present application, and is not used to limit the protection scope of the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A method for predicting a time signal, applied to an electronic device, characterized in that: include: Acquire N historical reporting point information formed by writing on the electronic device with the stylus pen, where N is an integer greater than 1; Determine characteristic information according to the reporting point information, wherein the characteristic information includes at least one of the following writing characteristics: trajectory curvature, pressure change rate, speed change rate, and trajectory change rate, wherein the trajectory curvature is used to indicate the degree of curvature of the trajectory formed based on the N historical reporting points, the pressure change rate is used to indicate the degree of pressure change in the process of the stylus pen writing on the electronic device to form the N historical reporting points, the speed change rate is used to indicate the degree of speed change in the process of the stylus pen writing on the electronic device to form the N historical reporting points, and the trajectory change rate is used to indicate the degree of change of the trajectory formed based on the N historical reporting points compared to the trajectory formed by multiple historical reporting points acquired in the last prediction process; Processing the feature information using a posture prediction model to obtain a prediction result, wherein the prediction result is used to indicate a first probability that the stylus pen performs a preset writing operation or to indicate whether the stylus pen performs the preset writing operation, wherein the preset writing operation is a pen-lifting operation or a turning operation; Determine the number of reported points according to the prediction result, where the number of reported points is greater than or equal to 0 and less than or equal to a first preset number; When the number of reporting points is greater than 0 and less than or equal to the first preset number, the predicted reporting points of the current frame are predicted according to the number of reporting points; or, when the number of reporting points is equal to 0, the real reporting points of the current frame are drawn and displayed.

2. The prediction method according to claim 1, characterized in that: The prediction result is used to indicate the first probability; and the step of determining the number of reported points according to the prediction result includes: Determining, according to the first probability, whether the stylus pen performs the preset writing operation; In the case where it is determined that the stylus pen performs the preset writing operation, determining that the number of reported points is a first number or 0; or, When it is determined that the stylus pen has not performed the preset writing operation, the number of reported points is determined to be the first preset number.

3. The prediction method according to claim 2, characterized in that: The first quantity is a second preset quantity.

4. The prediction method according to claim 2, characterized in that: In the case where it is determined that the stylus pen performs the preset writing operation, determining that the number of reported points is the first number includes: When it is determined that the stylus pen performs the preset writing operation, the number of reported points is determined to be the first number according to the first preset number and the first probability.

5. The prediction method according to any one of claims 2 to 4, characterized in that: The prediction result is further used to indicate a second probability that the stylus pen does not perform the preset writing operation; and determining whether the stylus pen performs the preset writing operation according to the first probability includes: When the first probability is greater than the second probability, determining that the stylus pen performs the preset writing operation; or, When the first probability is less than the second probability, it is determined that the stylus pen does not perform the preset writing operation.

6. The prediction method according to claim 1, characterized in that: The prediction result is used to indicate the first probability; and the step of determining the number of reported points according to the prediction result includes: The number of reported points is determined according to the first preset number and the first probability.

7. The prediction method according to claim 1, characterized in that: The prediction result is used to indicate whether the stylus pen performs the preset writing operation; And, determining the number of reporting points according to the prediction result includes: In a case where the prediction result indicates that the stylus pen performs the preset writing operation, determining that the number of reported points is a second preset number or 0, and the second preset number is less than the first preset number; or, When the prediction result indicates that the stylus pen has not performed the preset writing operation, the number of reported points is determined to be the first preset number.

8. The prediction method according to any one of claims 1 to 7, characterized in that: The reporting point information includes coordinate information of each historical reporting point, and the characteristic information includes the curvature of the track; And, determining characteristic information according to the reported point information includes: Determine a moving distance and a moving track length according to the coordinate information of each historical reporting point, wherein the moving distance is used to represent a straight-line distance between a first historical reporting point and an Nth historical reporting point among the N historical reporting points, the time of the first historical reporting point being earlier than the time of the Nth historical reporting point, and the moving track length is used to represent a length of a track formed based on the N historical reporting points; The track curvature is determined according to the moving distance and the moving track length.

9. The prediction method according to claim 8, characterized in that: The characteristic information also includes the moving distance and / or the moving track length.

10. The prediction method according to any one of claims 1 to 9, characterized in that: The reporting point information includes pressure information of each historical reporting point, and the characteristic information includes the pressure change rate; And, determining characteristic information according to the reported point information includes: The pressure change rate is determined based on pressure information of at least part of the historical reporting points.

11. The prediction method according to claim 10, characterized in that: The step of determining the pressure change value based on at least part of the pressure information of the historical reporting points includes: The pressure change rate is determined as the ratio between the maximum pressure among the pressures of the N historical reporting points and the pressure at the Nth historical reporting point, where the Nth historical reporting point is the latest reporting point among the N historical reporting points.

12. The prediction method according to any one of claims 1 to 11, characterized in that: The reporting point information includes coordinate information and time information of each historical reporting point, and the characteristic information includes the speed change rate; And, determining characteristic information according to the reported point information includes: The speed change rate is determined based on coordinate information and time information of at least part of the historical reported points.

13. The prediction method according to claim 12, characterized in that: The determining the speed change rate according to the coordinate information and time information of at least part of the historical reported points includes: Determine N-1 speeds according to the coordinate information and time information of two adjacent historical reporting points among the N historical reporting points; The speed change rate is determined as a ratio between a maximum speed among the N-1 speeds and an N-1th speed, wherein the N-1th speed is the latest speed among the N-1 speeds.

14. The prediction method according to any one of claims 1 to 13, characterized in that: The reporting point information includes coordinate information of each historical reporting point, and the characteristic information includes the trajectory change rate; And, determining characteristic information according to the reported point information includes: Determine a first moving track length according to the coordinate information of each of the historical reporting points, where the first moving track length is used to represent the length of a track formed based on the N historical reporting points; The trajectory change rate is determined according to the first moving trajectory length and the second moving trajectory length, wherein the second moving trajectory length is used to represent the length of a trajectory formed based on a plurality of historical reporting points obtained in the last prediction process.

15. The prediction method according to any one of claims 1 to 14, characterized in that: Before acquiring the reporting point information of N historical reporting points formed by writing on the electronic device by the stylus pen, the method further includes: Acquire a plurality of sample point reporting data, wherein the sample point reporting data is generated based on a plurality of historical point reporting data generated when the stylus pen writes on the electronic device, and the plurality of sample point reporting data includes a first type of sample point reporting data and a second type of sample point reporting data, wherein the first type of sample point reporting data is data collected when the stylus pen performs a normal writing operation, and the second type of sample point reporting data is data collected when the stylus pen performs the preset writing operation; Processing each of the sample point reporting data to obtain characteristic information of each of the sample point reporting data; The neural network model is trained using the characteristic information of each of the sample point reporting data to obtain the posture prediction model.

16. An electronic device, characterized in that: include: Memory, for storing computer instructions; A processor, configured to call the computer instructions stored in the memory to execute the method according to any one of claims 1 to 15.

17. A computer-readable storage medium, characterized in that: Used to store computer instructions, wherein the computer instructions are used to implement the method according to any one of claims 1 to 15.

18. A chip, characterized in that: The chip comprises: Memory: used to store instructions; A processor is used to call and execute the instructions from the memory so that an electronic device equipped with the chip system executes the method as described in any one of claims 1 to 15.

Images