A method, device, and electronic equipment for predicting reporting points.

Abstract

This application provides a method, device, and electronic device for predicting call points. By processing the call point information of multiple historical call points, feature information is obtained. A posture prediction model is used to process the feature information and output a prediction result related to the scenario in which the stylus performs a preset writing operation (lifting the pen or turning the pen). Then, the number of call points is determined based on the prediction result. When the number of call points is less than a first preset number or equal to 0, the predicted call points that originally needed to be predicted are reduced or canceled. For scenarios in which the stylus is likely to perform a preset writing operation (lifting the pen or turning the pen), the phenomenon of flying lines or stroke retraction caused by the prediction of call points can be reduced, thereby improving the user's writing experience.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence (AI) technology, and more specifically, to a method, device, and electronic device for predicting reporting points. Background Technology

[0002] With the development of touch technology, more and more electronic devices (such as tablets, laptops, and mobile phones) are adopting touch for human-computer interaction. In addition to being operated by fingers, the display screens of terminal devices can also be operated by styluses for tasks such as note-taking and drawing in the office.

[0003] When writing with a stylus on electronic devices such as tablets, the touch response, point calculation, and drawing of these points require processing and computation by the device. This process inevitably involves a certain delay. If the delay is too long, there will be a distance and delay between the position of the drawing point on the screen (commonly known as the ink outlet position) and the actual position of the stylus's touch point on the screen (i.e., the stylus tip position). This results in a lag in the writing process, severely impacting the user experience. Based on this, various prediction algorithms have been proposed. These algorithms can predict multiple points in advance and draw these predicted points, reducing the lag in the writing process. However, if the predicted points are drawn in scenarios such as lifting the pen or turning, phenomena such as stray lines (the ink outlet position exceeds the pen tip position) or stroke retraction may occur, affecting the user's writing experience.

[0004] Therefore, there is a need to provide a technology that can reduce stray lines or stroke retraction during the writing process in order to improve the user experience. Summary of the Invention

[0005] This application provides a method, device, and electronic device for predicting writing points. It processes multiple historical writing point information generated during stylus writing to obtain feature information including at least one writing feature. A posture prediction model is used to process the feature information, outputting a prediction result related to the scenario where the stylus performs a preset writing operation (lifting the pen or turning). The number of writing points is then determined based on the prediction result. When the determined number of writing points is less than a first preset number or equal to 0, the number of writing points that originally needed to be predicted (i.e., predicted writing points) can be reduced or canceled. This reduces stylus stylus prediction-induced stylus stylus errors or stroke retraction in scenarios where the stylus is highly likely to perform the preset writing operation (lifting the pen or turning), improving the user's writing experience. Furthermore, by associating the number of writing points with the preset writing operation, a number of writing points related to the preset writing operation can be obtained, increasing the flexibility of writing point prediction.

[0006] Firstly, a method for predicting writing points is provided, applied to electronic devices. The method includes: acquiring writing point information for N historical writing points formed by a stylus on the electronic device, where N is an integer greater than 1; determining feature information based on the writing point information, the feature information including at least one of the following writing features: trajectory curvature, pressure change rate, speed change rate, and trajectory change rate, wherein trajectory curvature represents the degree of curvature of the trajectory formed based on the N historical writing points, pressure change rate represents the degree of pressure change during the process of the stylus writing on the electronic device to form the N historical writing points, speed change rate represents the degree of speed change during the process of the stylus writing on the electronic device to form the N historical writing points, and trajectory change rate... The rate is used to represent the degree of change of the trajectory formed based on N historical reporting points compared to the trajectory formed by multiple historical reporting points obtained in the previous prediction process; the feature information is processed by an attitude prediction model to obtain the prediction result, which is used to indicate the first probability of the stylus performing a preset writing operation or to indicate whether the stylus performs a preset writing operation, where the preset writing operation is a pen lifting operation or a turning operation; based on the prediction result, the number of reporting points is determined, which is greater than or equal to 0 and less than or equal to a first preset number; when the number of reporting points is greater than 0 and less than or equal to the first preset number, the predicted reporting points of the current frame are predicted according to the number of reporting points; or, when the number of reporting points is equal to 0, the actual reporting points of the current frame are drawn and displayed.

[0007] Here, N historical call points are the actual call points generated before the current frame is drawn (or rendered). The current frame is the frame that the electronic device is about to draw based on the writing content of the stylus. For example, if the electronic device has just finished drawing the m-th frame, and the stylus continues to touch the display screen, the call points generated by the electronic device based on the touch will continue to be drawn in the m+1-th frame. The current frame here is the m+1-th frame.

[0008] For example, N historical points are the most recent N historical points that were generated before the current frame (e.g., frame m+1) was drawn (or rendered).

[0009] When the number of reported points is greater than 0 and less than or equal to a first preset number, the predicted reported points of the current frame are predicted according to the number of reported points, and the actual reported points and predicted reported points of the current frame are drawn and displayed. Specifically, when the number of reported points is greater than 0 and less than the first preset number, the predicted reported points of the current frame are predicted according to the number of reported points less than the first preset number; when the number of reported points is equal to the first preset number, the predicted reported points of the current frame are predicted according to the number of reported points of the first preset number.

[0010] The first preset quantity is a fixed value set in advance for predicting the reporting points. In the prior art, the electronic device always predicts the reporting points according to the first preset quantity. Therefore, the phenomenon of flying lines or stroke retraction may occur in scenarios such as turning or lifting the pen.

[0011] Of the four writing features in the example above, trajectory curvature represents the degree of curvature of the trajectory formed based on N historical call points. In implementation, if the stylus performs a turning operation (i.e., writing a curve), the trajectory curvature will be greater; conversely, if the stylus does not perform a turning operation, for example, if the stylus is writing a straight line, the trajectory curvature will be smaller. Ideally, the trajectory curvature will be almost negligible. Therefore, trajectory curvature can, to some extent, represent the probability (or trend) of the stylus performing a turning operation. Theoretically, if the stylus performs a turning operation, the number of predicted call points can be reduced or the prediction of predicted call points can be canceled to reduce flying lines or stroke retraction caused by prediction errors.

[0012] The pressure change rate is used to represent the degree of pressure change during the process of a stylus writing on an electronic device to form N historical writing points. In practice, if the stylus performs a lift-off operation, the pressure will change significantly; specifically, the pressure will decrease. Therefore, the pressure change rate can, to some extent, represent the probability (or trend) of the stylus performing a lift-off operation. Theoretically, if the stylus performs a lift-off operation, the number of predicted writing points can be reduced or the prediction of predicted writing points can be canceled, thereby reducing flying lines or stroke retraction caused by prediction errors.

[0013] The rate of change of speed is used to represent the degree of speed change of a stylus during the writing process on an electronic device, forming N historical writing points. In practice, if the stylus performs a turning or lifting operation, the writing speed will change significantly; specifically, whether the writing speed increases or decreases, the rate of change of speed will change significantly. Therefore, the rate of change of speed can, to some extent, represent the probability (or trend) of the stylus performing a turning or lifting operation. Theoretically, a significant change in the rate of change of speed likely means a significant change in writing speed, which can reduce the number of predicted writing points or cancel the prediction of predicted writing points, reducing stray lines or stroke retraction caused by prediction errors.

[0014] The trajectory change rate represents the degree of change in the trajectory formed based on N historical points compared to the trajectory formed from multiple historical points obtained during the previous prediction process. In implementation, if the stylus performs a turning or lifting operation, the writing speed will change significantly. Specifically, whether the writing speed increases or decreases, the trajectory change rate will change significantly. Therefore, the trajectory change rate can, to some extent, represent the probability (or trend) of the stylus performing a turning or lifting operation. Thus, theoretically, a significant change in the trajectory change rate likely means a significant change in writing speed, which can reduce the number of predicted points or cancel the prediction of predicted points, reducing stray lines or stroke retraction caused by prediction errors.

[0015] It should be understood that although both the rate of change of speed and the rate of change of trajectory can reflect writing speed, they have different focuses. The rate of change of speed reflects changes in writing speed from a microscopic perspective, while the rate of change of trajectory reflects changes in writing speed from a macroscopic perspective. In some scenarios (such as scenarios where writing speed changes frequently), the rate of change of trajectory can better reflect changes in writing speed from a macroscopic perspective.

[0016] The reporting point prediction method provided in this application process the reporting point information of multiple (e.g., N) historical reporting points formed when the stylus is writing to obtain feature information including at least one writing feature. These writing features can effectively characterize some state changes of the stylus when it performs a preset writing operation (lifting the pen or turning the pen). Therefore, the feature information is used as the input of the posture prediction model. The posture prediction model processes the feature information and outputs the prediction result. The output prediction result can be related to the scenario of the stylus performing the preset writing operation (lifting the pen or turning the pen). That is, the prediction result is used to indicate the first probability of the stylus performing the preset writing operation or to indicate whether the stylus performs the preset writing operation. Based on this, the number of reporting points determined according to the prediction result is related to the actual writing scenario. Thus, on the one hand, when the number of predicted points is less than or equal to the first preset number, it means that the predicted points that were originally required to be predicted have been reduced or canceled. In this case, the stylus is likely to execute the preset writing operation. Therefore, reducing or canceling the predicted points that were originally required to be predicted can effectively reduce the phenomenon of flying lines or stroke retraction caused by the prediction of points, thus improving the user's writing experience. On the other hand, when the number of predicted points is equal to the first preset number, it means that the stylus is likely not to execute the preset writing operation. The possibility of flying lines or stroke retraction is very small. Therefore, using the first preset number for predicting points can effectively reduce the responsiveness of the writing process and improve the responsiveness. In addition, since the number of predicted points is associated with the preset writing operation, the number of predicted points related to the preset writing operation can be obtained, which can improve the flexibility of point prediction.

[0017] Before acquiring the reporting information of N historical reporting points generated by the stylus writing on the electronic device, i.e., before executing the reporting point prediction method, a neural network model can be trained to obtain a posture prediction model. The training process is as follows: In some embodiments, multiple sample reporting point data are acquired. The sample reporting point data is generated based on multiple historical reporting points generated when the stylus writes on the electronic device. The multiple sample reporting point data includes a first type of sample reporting point data and a second type of sample reporting point data. The first type of sample reporting point data is the data collected when the stylus performs a normal writing operation, and the second type of sample reporting point data is the data collected when the stylus performs a preset writing operation. Each sample reporting point data is processed to obtain the feature information of each sample reporting point data. The feature information of each sample reporting point data is used to train the neural network model to obtain the posture prediction model.

[0018] The reporting prediction method provided in this application training method trains a neural network model with multiple sample reporting data. The training result is equivalent to establishing the relationship between writing operation and prediction result. Therefore, when using the posture prediction model for prediction, the feature information of the stylus writing is used as input. After processing the feature information, the posture prediction model can output the prediction result related to the preset writing operation.

[0019] The process of determining the number of reporting points based on the prediction results will also differ depending on the prediction results.

[0020] In embodiments where the prediction result is used to indicate a first probability, in the step of determining the number of reporting points based on the prediction result, in some embodiments, it is determined whether the stylus performs a preset writing operation based on the first probability; if it is determined that the stylus performs a preset writing operation, the number of reporting points is determined to be a first number or 0; or, if it is determined that the stylus does not perform a preset writing operation, the number of reporting points is determined to be a first preset number.

[0021] The point prediction method provided in this application embodiment can be understood as follows: under normal circumstances, it is only possible to determine whether the stylus will perform a preset writing operation when the value of the first probability is relatively large. Therefore, it is first determined whether the stylus will perform a preset writing operation based on the first probability. Only when it is determined that the stylus will perform a preset writing operation is the number of points reported reduced or canceled (i.e., the number of points reported is the first number or 0). This is equivalent to filtering out the first probability with a smaller value in advance. Therefore, the result of determining whether the stylus will perform a preset writing operation is more consistent with the actual scenario, and the judgment result is more reliable and accurate. Therefore, the number of points reported based on this situation is also more consistent with the actual scenario. On the one hand, it can better reduce the phenomenon of flying lines or stroke retraction caused by point prediction, and improve the user's writing experience. On the other hand, it can also better reduce the responsiveness delay in the writing process and improve the responsiveness experience.

[0022] In the above steps of determining the number of reporting points as the first number, in some embodiments, the first number is a second preset number.

[0023] The reporting prediction method provided in this application directly determines the second preset quantity as the first quantity of reporting points. The logic is simple, the code is easy to implement, the practicality is stronger, and the processing time is also saved to a certain extent.

[0024] In the above-described step of determining the number of reporting points as the first number, in some other embodiments, when it is determined that the stylus is performing a preset writing operation, the number of reporting points is determined as the first number based on the first preset number and the first probability.

[0025] The reporting prediction method provided in this application, based on the first probability indicated by the prediction result, determines a first quantity as the reporting quantity according to the first probability and a first preset quantity when it is determined that the stylus will perform a preset writing operation. Since the reporting quantity is determined based on the probability of the predicted actual scenario, the obtained reporting quantity is more consistent with the actual scenario, that is, the accuracy of the obtained reporting quantity is higher, and the flexibility of reporting prediction is improved.

[0026] In some embodiments, the prediction result is further used to indicate a second probability that the stylus does not perform a preset writing operation; and, in the step of determining whether the stylus performs a preset writing operation based on the first probability, if the first probability is greater than the second probability, it is determined that the stylus performs a preset writing operation; or, if the first probability is less than the second probability, it is determined that the stylus does not perform a preset writing operation.

[0027] The reporting prediction method provided in this application uses the prediction results output by the posture prediction model to indicate not only the first probability but also the second probability. The multi-dimensional output can improve the accuracy of machine learning to a certain extent. Furthermore, based on the comparison between the first probability and the second probability, it can determine whether the stylus performs a preset writing operation. This can more accurately predict whether the actual writing operation is the preset writing operation. Thus, when it is determined that the stylus performs a preset writing operation, the number of reporting points that can be determined to be less than the first preset number can better avoid the phenomenon of flying lines or stroke retraction.

[0028] In embodiments where the prediction result is used to indicate a first probability, in the step of determining the number of reporting points based on the prediction result, in some embodiments, the number of reporting points is determined based on a first preset number and a first probability.

[0029] The point prediction method provided in this application does not require prior determination of whether the stylus will perform a preset writing operation. Instead, it can directly determine the number of points to be reported based on the first probability and the preset first number. The logic is simple and the code is easy to implement, which also saves processing time to a certain extent. In addition, since the number of points to be reported is determined based on the probability (first probability) of the predicted actual scenario, the number of points to be reported is more consistent with the actual scenario, that is, the accuracy of the number of points to be reported is high, and the flexibility of point prediction is improved.

[0030] In embodiments where the prediction result is used to indicate whether the stylus performs a preset writing operation, in the step of determining the number of reporting points based on the prediction result, in some embodiments, if the prediction result indicates that the stylus performs a preset writing operation, the number of reporting points is determined to be a second preset number or 0, where the second preset number is less than the first preset number; or, if the prediction result indicates that the stylus does not perform a preset writing operation, the number of reporting points is determined to be the first preset number.

[0031] The reporting prediction method provided in this application embodiment allows the electronic device to directly determine whether the stylus performs a preset writing operation based on the prediction results output by the attitude prediction model. Since it is not necessary to further determine whether the stylus performs a preset writing operation based on the prediction results, the logic is simple and the code is easy to implement. Furthermore, when the stylus performs a preset writing operation, the preset second preset quantity or 0 is determined as the reporting quantity, and no other additional formula is needed to calculate the reporting quantity. This process is also easy to implement and saves processing time to a certain extent.

[0032] In this embodiment of the application, the reporting information of the above N historical reporting points may include at least one of the following: time information of each historical reporting point, coordinate information of each historical reporting point, and pressure information of each historical reporting point, which is related to the determined writing characteristics.

[0033] In some embodiments, the reporting information includes the coordinate information of each historical reporting point, and the feature information includes the trajectory curvature; and, in the step of determining the feature information based on the reporting information, for example, the moving distance and the moving trajectory length are determined based on the coordinate information of each historical reporting point, the moving distance is used to represent the straight-line distance between the first historical reporting point and the Nth historical reporting point among N historical reporting points, the time of the first historical reporting point is earlier than the time of the Nth historical reporting point, and the moving trajectory length is used to represent the length of the trajectory formed based on the N historical reporting points; the trajectory curvature is determined based on the moving distance and the moving trajectory length.

[0034] The reporting point prediction method provided in this application uses the trajectory curvature determined by the above-mentioned moving distance and trajectory length, which can better characterize the curvature of the trajectory formed based on N historical reporting points, and is beneficial to the accuracy of model training and the accuracy of the prediction results output during use.

[0035] In embodiments where the aforementioned feature information includes trajectory curvature and trajectory curvature obtained based on the moving distance and / or the moving trajectory length, in some embodiments, the feature information further includes the moving distance and / or the moving trajectory length.

[0036] The point prediction method provided in this application, in addition to considering the main features in the neural network algorithm, also needs to consider some secondary features to consider the influencing factors on the result from multiple dimensions, thereby obtaining relatively accurate data. It can be understood that the trajectory curvature is calculated through the moving distance and trajectory length, and is input into the attitude prediction model as an absolute explicit feature. However, considering that the moving distance and trajectory length may be affected by other factors (e.g., adding, subtracting, multiplying, or dividing other values ​​to obtain the moving distance or trajectory length), the trajectory curvature obtained based on the moving distance and trajectory length may also be affected by other factors. Therefore, inputting these relatively primitive parameters, such as the moving distance and / or trajectory length, as writing features into the attitude prediction model can improve the accuracy of the prediction results by considering more writing features.

[0037] In some embodiments, the reporting information includes pressure information of each historical reporting point, and the characteristic information includes the pressure change rate; and, in the step of determining the characteristic information based on the reporting information described above, the pressure change rate is, for example, determined based on the pressure information of at least some of the historical reporting points.

[0038] In the step of determining the pressure change value based on the pressure information of at least some historical reporting points, in some embodiments, the ratio between the maximum pressure among the pressures of N historical reporting points and the pressure of the Nth historical reporting point is determined as the pressure change rate, where the Nth historical reporting point is the latest reporting point among the N historical reporting points.

[0039] The reporting point prediction method provided in this application uses the ratio between the maximum pressure among N historical reporting points and the pressure of the Nth historical reporting point to determine the pressure change rate. That is, the pressure of the latest reporting point is compared with the maximum pressure among N reporting points to obtain the pressure change rate. This can better characterize the degree of pressure change during the process of writing on an electronic device with a stylus to form N historical reporting points, which is beneficial to the accuracy of model training and the accuracy of the prediction results output during use.

[0040] In some embodiments, the reporting information includes coordinate information and time information of each historical reporting point, and the feature information includes the rate of change of velocity; and, in the step of determining the feature information based on the reporting information described above, the rate of change of velocity is, by way of example, determined based on the coordinate information and time information of at least some of the historical reporting points.

[0041] In the above step of determining the rate of change of speed based on the coordinate and time information of at least some historical reporting points, in some embodiments, N-1 speeds are determined based on the coordinate and time information of two adjacent historical reporting points among N historical reporting points; the ratio between the maximum speed among the N-1 speeds and the N-1th speed is determined as the rate of change of speed, and the N-1th speed is the latest speed among the N-1 speeds.

[0042] The reporting prediction method provided in this application uses the ratio between the maximum speed and the (N-1)th speed among N-1 speeds to determine the speed change rate. That is, the speed change rate is obtained by comparing the latest speed with the maximum speed. This method can better characterize the degree of speed change during the process of the stylus writing on the electronic device to form N historical reporting points, which is beneficial to the accuracy of model training and the accuracy of the prediction results output during use.

[0043] In some embodiments, the reporting point information includes the coordinate information of each historical reporting point, and the feature information includes the trajectory change rate; and, in the step of determining the feature information based on the reporting point information, for example, a first moving trajectory length is determined based on the coordinate information of each historical reporting point, the first moving trajectory length being used to represent the length of the trajectory formed based on N historical reporting points; and a trajectory change rate is determined based on the first moving trajectory length and the second moving trajectory length, the second moving trajectory length being used to represent the length of the trajectory formed based on multiple historical reporting points obtained in the previous prediction process.

[0044] The prediction method for reporting points provided in this application uses a trajectory change rate to represent the degree of change of the trajectory formed based on N historical reporting points compared to the trajectory formed by multiple historical reporting points obtained in the previous prediction process. In fact, it reflects the change in writing speed corresponding to the two segments of the trajectory. The trajectory change rate determined by the first and second movement trajectory lengths can better characterize the change in writing speed corresponding to the two segments of the trajectory, which is beneficial to the accuracy of model training and the accuracy of the prediction results output during use.

[0045] In a second aspect, an electronic device is provided for performing the method provided in the first aspect. Specifically, the electronic device may include modules for performing any possible implementation of the first aspect.

[0046] Thirdly, an electronic device is provided, including a processor. The processor is coupled to a memory and can be used to execute instructions in the memory to implement the methods in any possible implementation of the first aspect described above. Optionally, the electronic device further includes a memory. Optionally, the device further includes a communication interface, to which the processor is coupled.

[0047] Fourthly, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a device, causes the device to implement the method in any possible implementation of the first aspect described above.

[0048] Fifthly, a computer program product comprising instructions, which, when executed by a computer, cause a device to implement the method in any of the possible implementations of the first aspect described above.

[0049] In a sixth aspect, a chip is provided, comprising: an input interface, an output interface, a processor, and a memory, wherein the input interface, the output interface, the processor, and the memory are connected via an internal connection path, and the processor is used to execute code in the memory, wherein when the code is executed, the processor is used to execute a method in any possible implementation of the first aspect described above. Attached Figure Description

[0050] Figure 1 This application provides a communication scenario for use between a stylus and an electronic device.

[0051] Figure 2 This is a scene diagram provided in the embodiments of this application, showing a user writing on the display screen of an electronic device using a stylus.

[0052] Figure 3 This is a schematic diagram illustrating the processing of an electronic device when a stylus is used for writing, as provided in the embodiments of this application.

[0053] Figure 4 This is a schematic diagram illustrating the relationship between the stylus and the reporting point when the stylus is writing, as provided in the embodiments of this application.

[0054] Figure 5 This is a schematic diagram of the stylus writing phenomenon when lifting the pen or turning during writing, provided in the embodiments of this application.

[0055] Figure 6 This is a schematic diagram of the structure of the electronic device 100 provided in the embodiments of this application.

[0056] Figure 7 This is a software structure block diagram of the electronic device 100 provided in the embodiments of this application.

[0057] Figure 8This is a schematic flowchart of the point prediction method 300 provided in the embodiments of this application.

[0058] Figure 9 This is a schematic diagram of the writing content when the stylus performs a turning operation according to an embodiment of this application.

[0059] Figure 10 This is a schematic diagram of the pressure change curve of the stylus provided in the embodiments of this application during the writing process.

[0060] Figure 11 This is a schematic diagram of the content written by the stylus provided in this application embodiment at different writing speeds.

[0061] Figure 12 This is a schematic flowchart of the point prediction method 400 provided in the embodiments of this application.

[0062] Figure 13 This is a schematic flowchart of the point prediction method 500 provided in the embodiments of this application.

[0063] Figure 14 This is a schematic flowchart of the point prediction method 600 provided in the embodiments of this application.

[0064] Figure 15 This is a schematic flowchart of the training process 700 of the pose prediction model provided in the embodiments of this application.

[0065] Figure 16 This is a network structure diagram of the attitude prediction model provided in the embodiments of this application.

[0066] Figure 17 This is another network structure diagram of the attitude prediction model provided in the embodiments of this application.

[0067] Figure 18 This is an exemplary block diagram of the reporting prediction device 800 provided in the embodiments of this application.

[0068] Figure 19 This is a schematic structural diagram of the electronic device 900 provided in the embodiments of this application. Detailed Implementation

[0069] The technical solutions in this application will now be described with reference to the accompanying drawings.

[0070] This application relates to a scenario where a user uses a stylus to write on an electronic device. Figure 1 This application provides a communication scenario for use between a stylus and an electronic device. Figure 2 This is a scene diagram provided in the embodiments of this application, showing a user writing on the display screen of an electronic device using a stylus.

[0071] refer to Figure 1 The stylus 200 and the electronic device 100 can be connected wirelessly or via a wired connection. Wireless connection can be established using methods such as Bluetooth or wireless local area networks (WLAN) (e.g., Wireless Fidelity, Wi-Fi). After the connection is established, refer to... Figure 2 When a user uses a stylus 200 to write on the display screen 100a of an electronic device, the point where the stylus 200 contacts the display screen 100a is called a contact point. After calculating based on the contact point of the stylus 200 on the display screen 100a, the electronic device 100 can draw the path that the stylus 200 travels on the display screen 100a to display the corresponding line, which is the content drawn by the stylus 200.

[0072] For example, the electronic device 100 can be various devices with a touch screen, such as mobile phones, tablets, laptops, and wearable devices. This application embodiment does not limit the form of the electronic device.

[0073] The stylus 200 includes passive styluses and active styluses; this application mainly relates to active styluses. Unless otherwise specified, the styluses in the embodiments of this application are all active styluses. During implementation, when the stylus 200 contacts the display screen 100a of the electronic device 100, it transmits a drive signal to the electronic device 100. The electronic device 100 can determine the position of the touch point based on the change in the parameter value generated by the drive signal, and thus draw the content written by the user on the display screen 100a based on the touch point. In some embodiments, the stylus 200 is an active capacitive stylus; in this case, the drive signal sent by the stylus 200 is a voltage signal. The larger the voltage signal received by the electronic device 100, the larger the capacitance value change, and the position of the touch point can be determined based on the increased capacitance value. In other embodiments, the stylus 200 is an active electromagnetic stylus; in this case, the drive signal sent by the stylus 200 is an electromagnetic signal. The larger the electromagnetic signal received by the electronic device 100, the larger the magnetic flux at the touch point, and the position of the touch point can be determined based on the increased magnetic flux.

[0074] When the stylus 200 writes on the display screen 100a, unlike writing on paper with a normal pen which can be displayed immediately, the electronic device needs to calculate the reporting point based on the touch point of the stylus 200 on the display screen 100a, and draw (or render) the reporting point before it can draw the corresponding line.

[0075] Figure 3 This is a schematic diagram illustrating the processing of an electronic device when a stylus is used for writing, as provided in an embodiment of this application. Specifically, refer to... Figure 3When a stylus touches the display screen, the screen needs a certain amount of time to respond. After the display screen responds, the physical layer (or bottom layer) of the electronic device calculates the reported point based on the touch point. The physical layer then reports the reported point to the upper layer (such as the application framework layer). This process also involves a certain delay, known as the reporting delay. The upper layer renders the reported point to obtain the drawn reported point (denoted as the drawing point), which also requires a certain amount of processing time. Finally, the drawing point is sent to the display screen for display. It can be seen that in the above process, since the touch response, reporting calculation, and drawing of the reported point all require processing and calculation by the device, the above process inevitably has a certain delay. It can be understood that when the line is drawn, the actual touch point of the stylus may have moved further back, resulting in a certain distance and delay between the position of the drawing point on the display screen (commonly known as the ink outlet position of the display screen) and the position of the actual touch point of the stylus on the display screen (i.e., the position of the stylus tip). This causes a delay in the writing process, seriously affecting the user experience.

[0076] Figure 4 This is a schematic diagram illustrating the relationship between the stylus and the reporting point when the stylus is used for writing, as provided in an embodiment of this application. (Reference) Figure 4 The stylus has moved to the actual touch point a position. The position of the drawing point c on the display screen does not overlap with the position of the actual touch point a position. There is a certain distance between the two, which causes the actual touch point of the stylus to move backward when the drawing point is displayed, resulting in a serious lack of responsiveness.

[0077] Based on this, various prediction algorithms have been proposed. These algorithms can predict multiple call points (hereinafter referred to as predicted call points) in advance and draw them, reducing the responsiveness of the writing process. In implementation, during each frame drawing, the electronic device erases the predicted call points of the previous frame and simultaneously calculates and draws both the actual and predicted call points for the current frame. The actual call points are calculated based on the touch point, equivalent to the actual point of contact between the stylus and the display screen. This avoids display problems caused by discrepancies between the predicted call points of the previous frame and the actual call points of the next frame. Drawing the predicted call points is only to solve the responsiveness problem of the current frame. When the next frame is drawn, the actual call points are used for drawing, without needing the predicted call points of the previous frame. This ensures that the display screen shows only the trajectory of the actual call points, thus avoiding display problems caused by discrepancies between the predicted call points of the previous frame and the actual call points of the next frame.

[0078] Continue to refer to Figure 4 For the current frame, call points b1, b2, and b3 are predicted call points, with call point b1 being the last predicted call point. During rendering and display, not only are the actual call points calculated based on the touch points drawn and displayed, but also the predicted call points are drawn and displayed. Figure 4In the image, markers b1, b2, and b3 are drawn and displayed. It can be seen that the distance between the position of marker b1 and the actual touch point is reduced, which can reduce the lag in the writing process. When drawing the next frame, the electronic device erases the predicted markers b1, b2, and b3 from the previous frame and redraws them based on the actual markers of the next frame and the predicted markers of the next frame.

[0079] In the above-mentioned solution of reducing the writing latency by predicting the stroke points, if the predicted stroke points are drawn in scenarios such as lifting the pen or turning, phenomena such as flying lines (the ink outlet position exceeds the pen tip position) or stroke retraction will occur, which will affect the user's writing experience. Figure 5 This is a schematic diagram illustrating the stylus writing phenomenon when lifting the pen or turning during writing, as provided in the embodiments of this application. (Reference) Figure 5 In examples (a) and (b), the stylus performs a lift-up or turn operation, but the current frame has already drawn and displayed the predicted marker (the trajectory of the dotted line is formed by the predicted marker). It can be seen that the position of the drawing point on the display (the ink outlet position) exceeds the actual touch point position (the stylus tip position), resulting in a flying line phenomenon. Furthermore, since the next frame erases the predicted marker of the previous frame, if a flying line phenomenon occurs in the previous frame, then a brief stroke retraction phenomenon will also occur in the next frame (which is actually the process of erasing the predicted marker of the previous frame). These flying line or stroke retraction phenomena significantly affect the user's writing experience.

[0080] To address the aforementioned issues, this application provides a method for predicting call points. The method processes multiple historical call point information generated during stylus writing to obtain feature information including at least one writing feature. These features effectively characterize state changes during a pre-defined writing operation (lifting or turning). A posture prediction model is used to process the feature information and output prediction results related to the scenario of the stylus performing the pre-defined writing operation. For example, the prediction results indicate a first probability of the stylus performing the pre-defined writing operation or whether the stylus performs the pre-defined writing operation. Based on this, the number of call points is determined according to the prediction results. When the determined number of call points is less than a first preset number or equal to 0, the number of call points that originally needed to be predicted (i.e., predicted call points) can be reduced or canceled. This reduces the phenomenon of stray lines or stroke retraction caused by call point prediction in scenarios where the stylus is highly likely to perform the pre-defined writing operation (lifting or turning), improving the user's writing experience. Furthermore, by associating the number of call points with the pre-defined writing operation, a number of call points related to the pre-defined writing operation can be obtained, improving the flexibility of call point prediction.

[0081] Before describing the embodiments of this application, a summary of the relevant terms used throughout the text is provided.

[0082] Contact point: The point where the stylus actually contacts the display screen.

[0083] Reported points: Points used for drawing and displaying on the screen, including actual reported points and predicted reported points.

[0084] Actual contact reporting: The reporting points corresponding to each contact point are calculated based on the contact points.

[0085] Predicted call points: Used to predict the trajectory of the stylus after a real call point. Predicted call points can be calculated from real call points. In the current frame's rendering, there is no corresponding touch point for the predicted call point.

[0086] Historical report points: These are real report points generated before the current frame was drawn. Unless otherwise specified, all historical report points in this application embodiment are real report points.

[0087] The following, combined with Figures 6 to 17 The technical methods of the embodiments of this application will be described in detail below.

[0088] Figure 6 This is a schematic diagram of the structure of an electronic device 100 provided in an embodiment of this application. Exemplarily, the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, antenna 1, antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a proximity sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc. It can be understood that when the electronic device 100 is a mobile phone, the electronic device 100 includes a mobile communication module 150 and a SIM card interface 195. Furthermore, the display screen 194 here can correspond to the above... Figure 1 The display screen is 100a.

[0089] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0090] Processor 110 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0091] The controller can be the nerve center and command center of the electronic device 100. The controller can generate operation control signals according to the instruction opcode and timing signals to complete the control of fetching and executing instructions.

[0092] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0093] In some embodiments, the processor 110 can determine the position of the touch point based on parameter values ​​(such as voltage signals or electromagnetic signals) generated by the driving signals sent by the stylus, and then draw the content written by the user on the display screen 194 based on the touch point.

[0094] In some embodiments, the processor 110 can process the acquired historical reporting information to obtain at least one writing feature, and use a posture prediction model to process the at least one writing feature to obtain a prediction result. The prediction result is related to the scenario in which the stylus performs a preset writing operation (pen lifting operation or turning operation). Then, the number of reporting points is determined based on the prediction result, so as to reduce or cancel the reporting points that originally need to be predicted (i.e., predicted reporting points) when the stylus is likely to perform the preset writing operation. In this way, the phenomenon of flying lines or stroke retraction caused by the prediction of reporting points can be reduced, and the user's writing experience can be improved. In addition, when the number of reporting points is determined to be greater than 0 (i.e., when reporting points need to be predicted), the predicted reporting points are determined using historical reporting points. For example, the predicted reporting points are obtained by processing historical reporting points based on the reporting point prediction model, and the predicted reporting points are displayed on the display screen 194.

[0095] The wireless communication function of electronic device 100 can be implemented through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor, and baseband processor. Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals.

[0096] The mobile communication module 150 can provide solutions for wireless communication, including 2G / 3G / 4G / 5G, applied to the electronic device 100. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via antenna 1. In some embodiments, at least some functional modules of the mobile communication module 150 may be housed in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 may be housed in the same device.

[0097] The wireless communication module 160 can provide solutions for wireless communication applications on the electronic device 100, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to processor 110. The wireless communication module 160 can also receive signals to be transmitted from processor 110, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2.

[0098] In this embodiment, the wireless communication module 160 can receive a drive signal sent by the stylus. The processor 110 can determine the position of the touch point based on the parameter value generated by the drive signal, and then draw the content written by the user on the display screen 194 based on the touch point. In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time-Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technologies, etc. The GNSS may include the Global Positioning System (GPS), the Global Navigation Satellite System (GLONASS), the BeiDou Navigation Satellite System (BDS), the Quasi-Zenith Satellite System (QZSS), and / or satellite-based augmentation systems (SBAS).

[0099] Electronic device 100 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0100] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a miniature LED, a microLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, electronic device 100 may include one or N displays 194, where N is a positive integer greater than 1.

[0101] The display screen 194 also includes a display driver integrated circuit (DDIC). The DDIC is one of the main control components of the display panel. Its main function is to send functional signals and data to the display panel in the form of electrical signals. By controlling the screen brightness and color, it enables image information such as letters and pictures to be displayed on the screen. As a key component connecting the processor and the display screen, the DDIC plays a very important role in image display.

[0102] In embodiments where the driving signal sent by the stylus is a voltage signal, the voltage signal can change the electric field at the touch point, thereby changing the electrode capacitance at the touch point. For example, the DDIC of the display screen 194 can determine the position of the touch point by detecting the change in electrode capacitance. In embodiments where the driving signal sent by the stylus is an electromagnetic signal, for example, the electromagnetic signal interacts with the electromagnetic induction plate behind the display screen 194. When the stylus approaches the display screen 194, the induction lines under the electromagnetic induction plate behind the display screen 194 change. The DDIC of the display screen 194 receives signals from the horizontal and vertical antenna arrays and calculates the coordinate position of the stylus by the change in magnetic flux.

[0103] Electronic device 100 can perform shooting functions through ISP, camera 193, video codec, GPU, display 194 and application processor.

[0104] The external storage interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, music, video, and other files can be saved on the external memory card.

[0105] Internal memory 121 can be used to store computer executable program code, which includes instructions. Processor 110 executes various functional applications and data processing of electronic device 100 by running the instructions stored in internal memory 121. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback, image playback, etc.), etc. The data storage area may store data created during the use of electronic device 100 (such as audio data, phonebook, etc.). Furthermore, internal memory 121 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0106] In this embodiment of the application, specifically, the code for executing the attitude prediction model is stored in the internal memory 121, and the code for processing the historical reporting information to obtain at least one writing feature and predicting the reporting based on the prediction results of the attitude prediction model is also stored in the internal memory 121.

[0107] Electronic device 100 can implement audio functions, such as music playback and recording, through audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor.

[0108] Among the various types of sensors in sensor module 180, this application embodiment mainly relates to touch sensor 180K, also known as a "touch panel". Touch sensor 180K can be disposed on display screen 194, and the touch sensor 180K and display screen 194 together form a touchscreen, also known as a "touch screen". Touch sensor 180K is used to detect touch operations applied to or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through display screen 194. In other embodiments, touch sensor 180K may also be disposed on the surface of electronic device 100, in a different location than display screen 194.

[0109] The software system of electronic device 100 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application embodiment uses the layered architecture Android system as an example to exemplify the software structure of electronic device 100.

[0110] Figure 7 This is a software structure block diagram of the electronic device 100 provided in this application embodiment. The layered architecture divides the software into several layers, each with a clear role and division of labor. Layers communicate with each other through software interfaces. In some embodiments, the Android system includes a native framework layer, an application framework, and applications (APPs).

[0111] The local framework layer mainly includes some local services and some linked libraries. This layer can be implemented in C and C++ and is used for driver interaction with the underlying hardware. For example, the local framework layer includes a point acquisition module, which can respond to touch operations when the stylus touches the display screen and acquire point information of the actual points collected by the display screen.

[0112] In one example, the real time report obtained by the time report acquisition module can include the coordinates and time information of the time report. The time information can indicate absolute time, which refers to the internationally recognized Coordinated Universal Time (UTC), also known as Greenwich Mean Time. After acquiring the time report, the time report acquisition module can also perform noise reduction processing on the acquired time report and send the noise-reduced time report to the application framework layer.

[0113] The application framework layer includes system servers, which provide various services that the application framework layer may use, such as touch events (input flinger). Touch events provide service support for events that occur on the display screen when the stylus moves on the screen.

[0114] The application framework layer also includes several application programming interfaces (APIs), which allow the application layer to retrieve relevant data from underlying layers (such as the local framework layer) by calling these APIs. For example, the application framework layer includes a reporting API, which allows applications to obtain actual reporting information from the reporting module of the local framework layer.

[0115] The application framework layer can also include algorithms to perform any possible functions, such as uploading data to other applications in the application layer to complete a specific task.

[0116] The application layer can include a series of application packages. Application packages can include applications such as writing applications, WLAN, and Bluetooth as shown in the figure, or applications not shown in the figure such as camera, gallery, calendar, call, map, navigation, music, video, and SMS.

[0117] For example, the writing application could be a memo app, drawing app, or similar application.

[0118] The writing application can render and display call points (including real call points and predicted call points) on the screen.

[0119] In this embodiment, the electronic device can determine the number of reporting points based on the attitude prediction model. When the number of reporting points is greater than 0, the device predicts the reporting points for the current frame based on the prediction reporting model according to the number of reporting points. Here, the attitude prediction model and / or the reporting point prediction model can be configured in the writing application or in the application framework layer; this embodiment does not impose any limitations. Furthermore, at the software level, the reporting point prediction method of this embodiment can be implemented in one or more layers of the electronic device. The specific software layer at which the reporting point prediction method of this application is implemented is not specifically limited in this embodiment.

[0120] In some embodiments, such as Figure 7 As shown, the attitude prediction model and the reporting prediction model are configured in the writing application.

[0121] In one example, the writing application can cache the reporting information of multiple (e.g., N) recent historical reporting points from the acquired real reporting point information. Based on this information, the writing application obtains at least one writing feature, processes this feature using a pose prediction model, and outputs a prediction result to determine the number of reporting points to be predicted. The writing application then uses the reporting point prediction model to predict the predicted reporting points based on the cached historical reporting information, and outputs the predicted number of reporting points.

[0122] In another example, the local framework layer's reporting module can obtain reporting information from multiple (e.g., N) recent historical reporting points. The writing application can obtain this information by calling an API. Alternatively, the application framework layer can obtain reporting information from multiple (e.g., N) recent historical reporting points from the real reporting information from the local framework layer, and the writing application can obtain this information from the application framework layer. Based on this information, the writing application obtains at least one writing feature, processes this feature using a pose prediction model, and outputs a prediction result to determine the number of predicted reporting points. Finally, based on the cached reporting information from multiple historical reporting points, the writing application uses a reporting point prediction model to predict the predicted reporting points and outputs the predicted number of reporting points.

[0123] In other embodiments, the pose prediction model and the predicted reporting model can also be configured in the application framework layer (not shown in the figure). For example, the application framework layer can obtain the reporting information of multiple (e.g., N) recent historical reporting points from the real reporting point information reported by the local framework layer. Based on this information, the application framework layer obtains at least one writing feature, processes this feature using the pose prediction model, and outputs a prediction result to determine the number of predicted reporting points to be predicted. The application framework layer can then use the predicted reporting model to predict the predicted reporting points based on the obtained information from the multiple historical reporting points, outputting the predicted number of reporting points. Furthermore, the writing application can obtain the predicted reporting points from the application framework layer for rendering.

[0124] In other embodiments, the attitude prediction model and the reporting prediction model may also be located in different layers, and this application embodiment does not impose any limitations.

[0125] It should be understood that the embodiments of this application are only illustrated using the Android system as an example. In other operating systems (such as Windows system, iOS system, etc.), as long as the functions implemented by each functional module are similar to those in the embodiments of this application, the solution of this application can also be implemented.

[0126] Figure 8 This is a schematic flowchart of the point prediction method 300 provided in the embodiments of this application. It should be understood that the executing entity of method 300 can be an electronic device, or a processor or chip within the electronic device. For ease of description, an electronic device is used as the executing entity in the description of the embodiments of this application.

[0127] In step S310, the electronic device acquires the reporting information of N historical reporting points formed by the stylus writing on the electronic device.

[0128] In implementation, the writing application is activated, and the stylus touches and slides on the electronic device's display screen, indicating that the stylus is writing on the electronic device. Based on the stylus's writing operation on the display screen, the electronic device can obtain the reporting information of N historical reporting points. For example, after the stylus touches the electronic device's display screen, the electronic device drives the display screen and obtains the touch information of the display screen. It then determines the reporting information of the reporting points based on the touch information, and from the determined reporting points, the electronic device obtains the reporting information of N historical reporting points.

[0129] It should be understood that the N historical call points here are the actual call points generated before the current frame is drawn (or rendered). These N historical call points are used to determine the number of call points. The current frame is the frame that the electronic device is about to draw based on the writing content of the stylus. For example, if the electronic device has just finished drawing the m-th frame, and the stylus continues to touch the display screen, in the m+1-th frame, the call points generated by the electronic device based on the touch will continue to be drawn. Here, the current frame is the m+1-th frame.

[0130] In some embodiments, N can be preset. For example, N can be any number such as 6, 8, 10, 15, 18, 20, 30, etc., and this application does not impose any limitation.

[0131] In some embodiments, the N historical report points are the most recent N historical report points generated before the current frame (e.g., frame m+1) is drawn (or rendered). These N historical report points may include actual report points to be drawn that have been generated in the current frame (e.g., frame m+1), or actual report points that have been drawn in at least one frame before the current frame (e.g., frame m), depending on the specific circumstances. When the writing speed is fast, the N historical report points may only include at least a portion of the actual report points to be drawn in the current frame (e.g., frame m+1). When the writing speed is slow, the N historical report points may include not only the actual report points to be drawn in the current frame (e.g., frame m+1), but also at least a portion of the actual report points that have been drawn in at least one frame before the current frame (e.g., frame m). Suppose the current frame is frame 5, and N=10, meaning the 10 historical call points are the 10 most recent historical call points generated before frame 5 was drawn. For example, if the writing speed is fast, frame 5 may have generated 15 real call points but not yet drawn. In this case, the 10 historical call points include the 10 real call points generated in frame 5 but not yet drawn, and these 10 real call points are later than the other 5 real call points in frame 5. As another example, if the writing speed is slow, frame 5 may have generated only 6 real call points. In this case, the 10 historical call points include the 6 real call points generated in frame 5 but not yet drawn, as well as the 4 real call points drawn in frame 4.

[0132] In one example, when the stylus begins writing, the number of historical reporting points is relatively small. If the number of historical reporting points is less than N, the stylus posture can be disregarded initially, and subsequent steps can be executed only when the number of historical reporting points reaches N. In other examples, if the number of historical reporting points is less than N, subsequent steps can be executed based on the currently acquired number of historical reporting points.

[0133] In other embodiments, N may not be preset, but may change as needed based on actual conditions. For example, all real call points generated but not yet drawn in the current frame may be used as N historical call points to perform subsequent steps. In this case, N is not a preset fixed value, but a variable.

[0134] In some embodiments, the reporting information of the above N historical reporting points may include at least one of the time information, coordinate information, and pressure information of each historical reporting point, which is related to the writing features that need to be determined later, and will be described in detail below.

[0135] In step S320, the electronic device determines the feature information based on the reported information.

[0136] The feature information is used to represent the state changes of the stylus when writing on the electronic device. The feature information includes at least one writing feature, and it should be understood that the writing feature in the feature information is any feature (or parameter) that can characterize the state changes of the stylus when writing.

[0137] In some embodiments, the feature information includes at least one of the following writing features: trajectory curvature, pressure change rate, velocity change rate, and trajectory change rate.

[0138] In other embodiments, the feature information may also include other writing features besides the four writing features mentioned above, such as at least one of the following: movement distance, movement trajectory length, the (N-1)th speed out of N-1 speeds, the maximum speed out of N-1 speeds, the length of the first movement trajectory, and the length of the second movement trajectory. Thus, the prediction results obtained through more writing features are more accurate.

[0139] The following is a detailed description of the various writing features involved in the embodiments of this application and the process of determining each writing process based on the reporting information.

[0140] I. Trajectory curvature

[0141] Trajectory curvature is used to represent the degree of curvature of a trajectory formed based on N historical reporting points.

[0142] In practice, if the stylus performs a turning operation (i.e., writing a curve), the curvature of the trajectory will be greater; conversely, if the stylus does not perform a turning operation, for example, when writing a straight line, the curvature of the trajectory will be smaller, ideally almost negligible. Therefore, the curvature of the trajectory can, to some extent, represent the probability (or trend) of the stylus performing a turning operation. Theoretically, if the stylus is writing a straight line, more prediction points can be predicted to improve writing tracking performance; if the stylus is writing a curve (i.e., performing a turning operation), the number of prediction points can be reduced or prediction points can be canceled to reduce stray lines or stroke retraction caused by prediction errors.

[0143] Figure 9 This is a schematic diagram illustrating the writing content when the stylus performs a turning operation, as provided in an embodiment of this application. (Reference) Figure 9 The implementation part is based on the trajectory formed by N historical reporting points. Since the stylus performs a turning operation, the trajectory formed by the N historical reporting points is a curve with a significant degree of curvature.

[0144] In cases where the feature information includes trajectory curvature, the reporting point information includes the coordinate information of each historical reporting point. Correspondingly, the electronic device can determine the trajectory curvature based on the coordinate information of each historical reporting point.

[0145] In some embodiments, the electronic device determines the moving distance and the moving trajectory length based on the coordinate information of each historical reporting point. The moving distance represents the straight-line distance between the first historical reporting point and the Nth historical reporting point among N historical reporting points, and the time of the first historical reporting point is earlier than the time of the Nth historical reporting point. The moving trajectory length represents the length of the trajectory formed based on the N historical reporting points. The electronic device determines the trajectory curvature based on the moving distance and the moving trajectory length.

[0146] It should be understood that the time of historical reporting points mentioned in the embodiments of this application refers to the time when the historical reporting points were generated.

[0147] Regarding the movement distance, in one example, the movement distance can be generated based on the coordinate information of the first historical reporting point and the coordinate information of the Nth historical reporting point, and the movement distance satisfies Formula 1:

[0148]

[0149] in, Indicates the distance traveled. This represents the coordinates (x-coordinate) of the Nth historical reporting point in the X direction. This represents the coordinate (vertical coordinate) of the Nth historical reporting point in the Y direction. This represents the x-coordinate of the first historical reporting point. This represents the ordinate of the first historical reporting point.

[0150] Regarding the length of the movement trajectory, in one example, the length of the movement trajectory can be generated based on the coordinate information of each historical reporting point, and the length of the movement trajectory satisfies Formula 2:

[0151]

[0152] in, Indicates the length of the movement trajectory. This represents the x-coordinate of the i-th historical reporting point. This represents the y-coordinate of the i-th historical reporting point, where i iterates from 1 to N. That is , , ... , , That is , , ... , .

[0153] Regarding trajectory curvature, in one example, trajectory curvature satisfies Formula 3: ,in, Indicates the curvature of the trajectory. Less than or equal to 1.

[0154] Based on formulas 1 to 3 above, it can be seen that, The smaller the value, the greater the curvature of the trajectory, and the greater the probability (or tendency) of the stylus to perform a lift-off operation.

[0155] In embodiments where the aforementioned feature information includes trajectory curvature, in some embodiments, the feature information also includes the travel distance and / or the length of the travel trajectory. This can improve the accuracy of the prediction results.

[0156] In this embodiment, the writing features in the feature information are the input to the pose prediction model. Theoretically, in neural network algorithms, in addition to considering the main features, some secondary features also need to be considered to account for the influencing factors on the results from multiple dimensions, thereby obtaining relatively accurate data. Ideally, the original data (e.g., the reporting information of N historical reporting points) would be used as the input to the pose recognition model, but this would generate a lot of noisy data. Therefore, considering all factors, feature extraction is performed on the original data (e.g., the reporting information of N historical reporting points), and the extracted writing features are used as the input to the pose prediction model. This reduces noisy data and also yields relatively accurate results. Based on the above considerations, it can be understood that the trajectory curvature is calculated using the travel distance and trajectory length, and is input into the attitude prediction model as an absolute explicit feature. However, considering that the travel distance and trajectory length may be affected by other factors (e.g., adding, subtracting, multiplying, or dividing other values ​​to obtain the travel distance or trajectory length), the trajectory curvature obtained based on the travel distance and trajectory length may also be affected by other factors. Therefore, inputting the relatively primitive parameters such as travel distance and / or trajectory length as writing features into the attitude prediction model can improve the accuracy of the prediction results by considering more writing features.

[0157] II. Pressure Change Rate

[0158] The pressure change rate is used to represent the degree of pressure change during the process of a stylus writing on an electronic device to form N historical data points.

[0159] In practice, when the stylus lifts off the pen, the pressure changes significantly; specifically, the pressure decreases. With slow lifting, the pressure change is relatively smooth, gradually decreasing from a large value to zero. With rapid lifting, the pressure change may be precipitous, dropping abruptly from its maximum value to zero. Therefore, the rate of pressure change can, to some extent, represent the probability (or trend) of the stylus lifting off the pen. Theoretically, lifting off the pen can reduce the number of predicted points or cancel predicted points altogether, thus reducing flying lines or stroke retraction caused by prediction errors.

[0160] Figure 10 This is a schematic diagram of the pressure change curve of the stylus provided in the embodiments of this application during the writing process. Figure 10 (a) in the figure shows the pressure change curve when the pen is lifted slowly. It can be seen that, due to the slow lifting of the pen, the pressure gradually changes from the 10th reporting point. The pressure change curve is relatively smooth. The pressure slowly changes from the maximum value to 0. The pressure becomes 0 at about the 22nd reporting point. The entire change process lasts for about 12 reporting points. Figure 10(b) shows the pressure change curve when the pen is lifted quickly. It can be seen that, due to the quick lifting of the pen, the pressure drops sharply to 0 starting from about the 26th reporting point. The change process lasts for a maximum of only one reporting point.

[0161] When the characteristic information includes the rate of pressure change, the reporting information includes the pressure information of each historical reporting point. Correspondingly,

[0162] Electronic devices can determine the rate of pressure change based on pressure information from at least some historical data points.

[0163] The pressure information of at least some historical reporting points refers to the pressure information of some or all of the N historical reporting points. Furthermore, the pressure information of historical reporting points represents the pressure applied by the stylus when it contacts the display screen at the corresponding touch point of the historical reporting point.

[0164] In the step of determining the pressure change value based on pressure information from at least some historical reporting points, in some embodiments, the electronic device determines the pressure change rate as the ratio between the maximum pressure among the pressures of N historical reporting points and the pressure of the Nth historical reporting point, where the Nth historical reporting point is the latest reporting point among the N historical reporting points. In this embodiment, the pressure change value is determined based on the pressure information from all historical reporting points among the N historical reporting points.

[0165] In other words, the electronic device determines the pressure with the largest value (i.e., the maximum pressure) from the pressure of N historical reporting points, and compares the maximum pressure with the pressure of the Nth historical reporting point, which is the latest one, to obtain the pressure change rate.

[0166] The rate of change of pressure can be obtained from formula 4: , This represents the rate of change of pressure, and is a value less than or equal to 1. This represents the pressure at the i-th historical reporting point, where i ranges from 1 to N. That is , , ... .

[0167] Based on the above formula 4, it can be seen that... The larger the pressure, the greater the probability (or tendency) that the stylus will perform a lift-off operation.

[0168] It should be understood that the method of determining the pressure change rate in the above examples is merely illustrative and should not be construed as limiting the embodiments of this application.

[0169] For example, the electronic device can also compare the pressure of any two adjacent historical reporting points to obtain N-1 pressure change rates, and determine the pressure change rate with the largest value among these N-1 pressure change rates (i.e., the maximum pressure change rate) as the pressure change rate input to the posture prediction model. In this embodiment, the electronic device determines the pressure change rate based on the pressure information of all historical reporting points out of N historical reporting points.

[0170] For example, the electronic device can also determine the pressure change rate as the ratio between the pressure of the first historical reporting point and the pressure of the Nth historical reporting point. In this embodiment, the electronic device determines the pressure change rate based on the pressure information of a portion of the N historical reporting points.

[0171] III. Rate of change of velocity

[0172] The rate of change of speed is used to represent the degree of speed change of a stylus during the process of writing on an electronic device to form N historical data points.

[0173] In practice, if the stylus performs a turning or lifting operation, the writing speed will change significantly. Specifically, both increasing and decreasing the writing speed will cause a change in the rate of change of speed. Therefore, the rate of change of speed can, to some extent, represent the probability (or trend) of the stylus performing a turning or lifting operation. It should be noted that the spacing between the predicted points varies with the writing speed; a faster writing speed results in a larger spacing between the points, and a slower writing speed results in a smaller spacing. Therefore, theoretically, a significant change in the rate of change of speed likely means a significant change in writing speed. If there are many predicted points, even a single incorrect predicted point can produce noticeable stray lines or stroke retraction. Therefore, the number of predicted points can be reduced or the prediction of predicted points can be canceled.

[0174] When the feature information includes the rate of change of velocity, the reporting information includes the coordinate and time information of each historical reporting point. Correspondingly, the electronic device can determine the rate of change of velocity based on the coordinate and time information of at least some of the historical reporting points. The time information of the historical reporting points indicates the time when the historical reporting point was generated.

[0175] In the above step of determining the rate of change of speed based on the coordinate and time information of at least some historical reporting points, in some embodiments, the electronic device determines N-1 speeds based on the coordinate and time information of two adjacent historical reporting points among N historical reporting points; the electronic device determines the rate of change of speed as the ratio between the maximum speed among the N-1 speeds and the N-1th speed, where the N-1th speed is the latest speed among the N-1 speeds.

[0176] Specifically, the electronic device obtains the i-th speed based on the coordinate and time information of the i-th and (i+1)-th historical reporting points. The speed is iterated from 1 to N-1, ultimately yielding N-1 speed values. For example, if i=1, the first speed is obtained based on the coordinate and time information of the first and second historical reporting points; if i=2, the second speed is obtained based on the coordinate and time information of the second and third historical reporting points; and so on, until i=N-1, where the N-1-th speed is obtained based on the coordinate and time information of the (N-1)-th and N-th historical reporting points. The electronic device then determines the speed with the largest value (i.e., the maximum speed) from these N-1 speeds, and the ratio between the maximum speed and the (N-1)-th speed is defined as the rate of change of speed.

[0177] For example, the i-th speed among N-1 speeds satisfies Formula 5:

[0178]

[0179] in, Represents the i-th velocity. This represents the velocity of the i-th velocity in the x-direction. This represents the velocity of the i-th velocity in the y-direction. , , This represents the coordinates of the (i+1)th historical reporting point in the X direction. This represents the coordinates of the (i+1)th historical reporting point in the Y direction. This represents the time when the (i+1)th historical report was generated. This represents the coordinates of the i-th historical reporting point in the X direction. This represents the coordinates of the i-th historical reporting point in the Y direction. This represents the time when the i-th historical reporting point was generated, where i ranges from 1 to N-1.

[0180] Regarding the rate of change of velocity, based on the above formula 5, N-1 velocities are obtained, and the rate of change of velocity satisfies formula 6:

[0181]

[0182] in, This represents the (N-1)th velocity. This represents the maximum speed among N-1 speeds.

[0183] It should be understood that the method of determining the rate of change of velocity in the above examples is merely illustrative and should not be construed as limiting the embodiments of this application.

[0184] For example, the electronic device can also compare the velocity forces of any two adjacent historical reporting points to obtain N-1 velocity change rates, and determine the velocity change rate with the largest value among these N-1 velocity change rates (i.e., the maximum velocity change rate) as the velocity change rate input to the posture prediction model. In this embodiment, the electronic device determines the velocity change rate based on the coordinate and time information of all historical reporting points in the N historical reporting points.

[0185] For example, the electronic device can also determine the rate of change of speed as the ratio between the speed of the first historical reporting point and the speed of the Nth historical reporting point. In this embodiment, the electronic device determines the rate of change of speed based on the coordinate and time information of a portion of the N historical reporting points.

[0186] In embodiments where the aforementioned feature information includes the rate of change of velocity, in some embodiments, the feature information may further include the (N-1)th velocity and / or the maximum velocity among the (N-1)th velocities. It is understood that the rate of change of velocity is calculated using the (N-1)th velocity and the maximum velocity among the (N-1)th velocities, and is input into the attitude prediction model as an absolute explicit feature. However, considering that the (N-1)th velocity and the maximum velocity among the (N-1)th velocities may be affected by other factors, and consequently the rate of change of velocity obtained based on the (N-1)th velocity and the maximum velocity among the (N-1)th velocities may also be affected by other factors, it is appropriate to also input these relatively primitive parameters, such as the (N-1)th velocity and the maximum velocity among the (N-1)th velocities, as writing features into the attitude prediction model. By considering more writing features, the accuracy of the prediction results can be improved.

[0187] IV. Rate of change of trajectory

[0188] The trajectory change rate is used to represent the degree of change of the trajectory formed based on N historical reporting points compared to the trajectory formed by multiple historical reporting points obtained in the previous prediction process.

[0189] The multiple historical reporting points acquired in the previous prediction process are N historical reporting points obtained before the previous frame was drawn, used to determine the number of predicted reporting points. Specifically, the feature information determined by the N historical reporting points acquired in the previous prediction process is used to predict whether the stylus will perform a pen lift-off or turning operation. The number of reporting points is determined based on the prediction results. It should be understood that since the time interval between adjacent reporting points is the same, the duration corresponding to the N historical reporting points acquired in each prediction process is the same. For example, if the time interval between adjacent reporting points is 1ms and N=10, then the duration corresponding to the 10 historical reporting points acquired in the previous prediction process is 10ms, and the duration corresponding to the 10 historical reporting points acquired in the current prediction process is also 10ms.

[0190] It should be understood that for the same number (N) of historical reporting points, if the writing speed is fast and the spacing between reporting points is large, the length of the trajectory formed by N historical reporting points will be longer. If the writing speed is slow and the spacing between reporting points is small, the length of the trajectory formed by N historical reporting points will be shorter. Figure 11 This is a schematic diagram illustrating the writing content of the stylus provided in this application at different writing speeds. Assume N=6. Figure 11 (a) in the diagram illustrates the trajectory formed by six historical reporting points under conditions of high writing speed. Figure 11 (b) in the diagram illustrates the trajectory formed by the six historical reporting points under slow writing speed. Clearly, Figure 11 The length of the trajectory corresponding to (a) in the middle is greater than that of the trajectory in the middle. Figure 11 The trajectory corresponding to (b) in the diagram is longer. If Figure 11 (a) in the image shows the trajectory formed by the six historical data points obtained during the previous prediction process, and... Figure 11 In the diagram, (b) represents the trajectory formed by the 6 historical points obtained in the current prediction process. The length of the trajectory formed by the 6 historical points obtained in the current prediction process is longer than the length of the trajectory formed by the 6 historical points obtained in the previous prediction process, indicating that the writing speed corresponding to the 6 historical points obtained in the current prediction process is generally faster. Therefore, the degree of change in the trajectory formed by the N historical points obtained in the two prediction processes actually reflects the change in the writing speed corresponding to the N historical points obtained in the two prediction processes, that is, the change in the writing speed corresponding to the two trajectories. In the implementation, if the stylus performs a turning operation or a lifting operation, the writing speed will change significantly. Specifically, whether the writing speed increases or decreases, it will cause a change in the trajectory change rate. Therefore, the trajectory change rate can, to some extent, represent the probability (or trend) of the stylus performing a turning operation or a lifting operation.

[0191] It should be noted that although both the rate of change of speed and the rate of change of trajectory can reflect writing speed, they have different focuses. The rate of change of speed reflects changes in writing speed from a microscopic perspective, while the rate of change of trajectory reflects changes in writing speed from a macroscopic perspective. In some scenarios (such as scenarios where writing speed changes frequently), the rate of change of trajectory can better reflect changes in writing speed from a macroscopic perspective. For example, the writing speed corresponding to the 10 historical points obtained in the previous prediction process is constantly changing, such as fast-slow-fast-slow, forming a trajectory with a length of 10mm. The writing speed corresponding to the 10 historical points obtained in the current prediction process is also constantly changing, such as fast-slow-fast, forming a trajectory with a length of 12mm. If the rate of change is used to represent the writing speed, the frequently changing speeds corresponding to the two predictions are not easy or objectively reflect the writing speed. However, if the trajectory change rate is used to represent the writing speed, regardless of the frequency of change of the writing speed corresponding to the 10 historical points obtained in the two prediction processes, the degree of change of the trajectory formed by the 10 historical points obtained in the two prediction processes can determine from the overall perspective that the writing speed corresponding to the 10 historical points obtained in the current prediction process is fast. This judgment result is very macroscopic and relatively objective.

[0192] In cases where the feature information includes the trajectory change rate, the reporting point information includes the coordinate information of each historical reporting point. Correspondingly, the electronic device determines the trajectory change rate based on the coordinate information of each historical reporting point.

[0193] In the above steps of determining the trajectory change rate based on the coordinate information of each historical reporting point, in some embodiments, the electronic device determines the first moving trajectory length based on the coordinate information of each historical reporting point. The first moving trajectory length is used to represent the length of the trajectory formed based on N historical reporting points. The electronic device determines the trajectory change rate based on the first moving trajectory length and the second moving trajectory length. The second moving trajectory length is used to represent the length of the trajectory formed based on multiple historical reporting points obtained in the previous prediction process.

[0194] The rate of change of the trajectory satisfies Formula 7: , Indicates the rate of change of the trajectory. Indicates the length of the second movement trajectory Indicates the length of the first movement trajectory. and It can be obtained from Formula 2, which is used to determine the length of the movement trajectory, and will not be repeated here.

[0195] In embodiments where the aforementioned feature information includes the trajectory change rate, in some embodiments, the feature information may further include the first trajectory length and / or the second trajectory length. It is understood that the trajectory change rate is calculated using the first and second trajectory lengths and is input into the attitude prediction model as an absolute explicit feature. However, considering that the first and second trajectory lengths may be affected by other factors, and consequently the trajectory change rate obtained based on these lengths may also be affected by other factors, it is appropriate to also input these relatively primitive parameters as writing features into the attitude prediction model. By considering more writing features, the accuracy of the prediction results can be improved.

[0196] In step S330, the electronic device uses an attitude prediction model to process the feature information and obtain the prediction result.

[0197] The prediction result is used to indicate the first probability of the stylus performing a preset writing operation or to indicate whether the stylus performs a preset writing operation, which is a pen lifting operation or a turning operation.

[0198] The posture prediction model can be obtained by pre-training a neural network model with the feature information of each sample data in multiple sample reporting data. Each sample data is generated based on multiple (e.g., N) historical reporting points generated when a stylus is writing on an electronic device. The feature information of each sample reporting data is obtained by processing each sample reporting data. The neural network model is trained using the feature information of each sample reporting data to obtain the posture prediction model.

[0199] It should be noted that the aforementioned sample reporting data includes two types of sample reporting data. The first type includes data collected when the stylus performs normal writing operations, and the second type includes data collected when the stylus performs a preset writing operation. In this way, the trained posture prediction model establishes a relationship between writing operations and prediction results. Therefore, when using the posture prediction model for prediction, inputting the feature information of the stylus during writing can output the relevant results of the stylus performing a preset writing operation (e.g., the first probability of performing the preset writing operation or whether the preset writing operation is performed). It should be understood that normal writing operations can be any writing operation other than the preset writing operation, such as writing a straight line.

[0200] Furthermore, the neural network model used for training can be any form of neural network model, and this application embodiment does not limit it in any way. For example, the neural network model can be a deep neural network (DNN) model.

[0201] In implementation, an electronic device or other device (such as a server) can be used to train a neural network model using feature information from each sample reporting point data to obtain a pose prediction model. This pose prediction model is then deployed on the electronic device. During use, the user opens a writing application installed on the electronic device. Upon detecting the user's writing action or the opening of the writing application, the electronic device starts a thread to load the pose prediction model. After obtaining feature information from N historical reporting points, the electronic device inputs this feature information into the loaded pose prediction model. The pose prediction model processes the feature information and outputs the prediction result.

[0202] In step S340, the electronic device determines the number of reporting points based on the prediction results.

[0203] The number of reported points is greater than or equal to 0 and less than or equal to a first preset number, which is a pre-set fixed value used for reporting point prediction. In the prior art, the electronic device continuously predicts reported points according to the first preset number, that is, determines the predicted reported points of the first preset number. It should be understood that the first preset number can be any value greater than 0, for example, the first preset number can be 6, 10, 12, 15, 18, 20, 30, etc.

[0204] If the number of reported points is greater than 0, the electronic device executes step S351; if the number of reported points is equal to 0, the electronic device executes step S352.

[0205] In this step, the process by which the electronic device determines the number of reporting points varies depending on the content indicated by the prediction result. When the prediction result indicates a first probability that the stylus will perform a preset writing operation, the electronic device determines the number of reporting points based on this first probability. For details, please refer to the relevant descriptions of methods 400 and 500 below, which will not be repeated here. When the prediction result indicates whether the stylus will perform a preset writing operation, the electronic device can determine the number of reporting points according to preset rules. For details, please refer to the relevant description of method 600 below, which will not be repeated here.

[0206] In step S351, when the number of reported points is greater than 0 and less than or equal to the first preset number, the electronic device predicts the predicted reported points of the current frame according to the number of reported points.

[0207] It should be understood that the current frame refers to the frame currently to be drawn.

[0208] In the process of predicting the predicted reporting points for the current frame, in some embodiments, the electronic device can continue to use the reporting point information of the aforementioned N historical reporting points to predict the predicted reporting points, thereby obtaining the predicted number of reporting points. Currently, in other embodiments, the electronic device can also use the reporting point information of fewer or more than N historical reporting points for reporting point prediction, and this application embodiment does not impose any limitations.

[0209] After determining the predicted number of call points, the electronic device draws and displays the actual and predicted call points for the current frame. Here, "drawing and displaying the actual and predicted call points for the current frame" means drawing the actual and predicted call points for the current frame and displaying the drawn actual and predicted call points.

[0210] When the number of reported points equals the first preset number, it means that the probability of the stylus executing the preset writing operation (lifting the pen or turning the pen) is very small. Therefore, the electronic device can predict the predicted reported points of the current frame according to the first preset number, that is, determine the first preset number of predicted reported points. In this way, when the stylus is unlikely to execute the preset writing operation (lifting the pen or turning the pen), there will be virtually no flying lines or stroke retraction, and it can effectively reduce the responsiveness of the writing process and improve the responsiveness experience.

[0211] When the number of reported points is greater than 0 and less than the first preset number, it means that the probability of the stylus performing the preset writing operation (lifting the pen or turning the pen) is relatively high. The number of reported points can be reduced, that is, the predicted reported points of the current frame are predicted according to the number of reported points less than the first preset number. Since the number of reported points is reduced, the flying lines or stroke retraction caused by the stylus performing the preset writing operation (lifting the pen or turning the pen) can be effectively reduced.

[0212] In step S352, when the number of reported points is equal to 0, the electronic device draws and displays the actual reported points of the current frame.

[0213] When the number of reporting points is equal to 0, it is equivalent to canceling the prediction of the predicted reporting points. That is, the electronic device no longer performs the prediction of the predicted reporting points, but draws and displays the actual reporting points of the current frame. This can effectively reduce the flying lines or stroke retraction caused by the stylus performing preset writing operations (pen lifting operation or turning operation).

[0214] As mentioned earlier, the process by which the electronic device determines the number of reporting points based on the prediction result varies depending on the content indicated by the prediction result. Below, we will describe in detail the process of determining the number of reporting points based on the prediction result, categorized into two cases (case 1 and case 2), and methods 400 to 600. Case 1 represents the scenario where the prediction result is used to instruct the stylus to perform a preset writing operation with a first probability; methods 400 and 500 primarily describe the process of determining the number of reporting points in case 1. Case 2 represents the scenario where the prediction result is used to instruct whether the stylus performs a preset writing operation; method 600 primarily describes the process of determining the number of reporting points in case 2. Similarly, an electronic device will be used as the executing entity for illustrative purposes.

[0215] Scenario 1: The prediction result is used to indicate the first probability of the stylus performing a preset writing operation.

[0216] Figure 12 This is a schematic flowchart of the reporting prediction method 400 provided in the embodiments of this application. In the process of determining the number of reporting points based on the prediction result in method 400, the key point is that the electronic device first determines whether the stylus performs a preset writing operation based on the first probability indicated by the prediction result. Only when it is determined that the stylus performs a preset writing operation will the number of reporting points be determined.

[0217] In step S410, the electronic device stylus writes on the electronic device to form N historical reporting points.

[0218] In step S420, the electronic device determines the feature information based on the reported information.

[0219] In step S430, the electronic device uses an attitude prediction model to process the feature information and obtain a prediction result, wherein the prediction result is used to indicate the first probability of the stylus performing a preset writing operation.

[0220] For a detailed description of steps S410 to S430, please refer to the relevant descriptions of steps S310 to S330 above, which will not be repeated here.

[0221] In step S441, the electronic device determines whether the stylus should perform a preset writing operation based on a first probability.

[0222] That is, the electronic device first determines whether the stylus will perform a preset writing operation based on a first probability. Only if it is determined that the stylus will perform the preset writing operation will the number of points be determined.

[0223] In some embodiments, the electronic device may compare a first probability with a preset probability. If the first probability is greater than the preset probability, it may determine that the stylus performs a preset writing operation. If the first probability is less than the preset probability, it may determine that the stylus does not perform a preset writing operation.

[0224] The preset probability can be a pre-set value, which serves as the benchmark for judging the preset writing operation. For example, if the preset probability is 0.5, the stylus can be considered to have performed the preset writing operation when the first probability is greater than 0.5.

[0225] It should be noted that when the first probability equals the preset probability, it can be classified as either the result of the stylus performing a preset writing operation or the result of the stylus not performing a preset writing operation; no limitation is made here.

[0226] In other embodiments, the electronic device may compare a first probability with a second probability indicated by the prediction result, the second probability being a variable determined based on the prediction results for different writing situations.

[0227] Specifically, the prediction result is also used to indicate a second probability that the stylus has not performed a preset writing operation, and in step S3412, if the first probability is greater than the second probability, the electronic device determines that the stylus has performed a preset writing operation; if the first probability is less than the second probability, it determines that the stylus has not performed a preset writing operation.

[0228] In other words, after the electronic device processes the feature information through the posture prediction model, the output prediction result is used to indicate a first probability and a second probability. The electronic device compares the first probability and the second probability to determine whether the electronic device should perform a preset writing operation. It can be understood that the prediction result is used not only to indicate the first probability but also the second probability. The multi-dimensional output can improve the accuracy of machine learning to a certain extent. Furthermore, determining whether the stylus should perform a preset writing operation based on the comparison of the first probability and the second probability can more accurately predict whether the actual writing operation is the preset writing operation. Therefore, when determining that the stylus should perform a preset writing operation, the number of reported points less than the first preset number can better avoid the phenomenon of flying lines or stroke retraction.

[0229] It should be noted that the case where the first probability equals the second probability can be categorized as either the result of the stylus performing a preset writing operation or the result of the stylus not performing a preset writing operation; no limitation is made here. Based on the judgment in step S441, if it is determined that the stylus did not perform a preset writing operation, the electronic device executes step S4423, that is, the electronic device determines the number of reported points to be the first preset number. Based on this, the electronic device executes step S452, that is, the electronic device predicts the predicted reported points for the current frame according to the first preset number, and then displays and draws the actual reported points and predicted reported points for the current frame.

[0230] After the judgment in step S441, if it is determined that the stylus will perform a preset writing operation, the electronic device executes either step S4421 or step S4422, depending on the internal design of the electronic device. For ease of description, the method by which the electronic device executes step S4421 to determine the number of reporting points is denoted as Method 1, and the method by which the electronic device executes step S4422 to determine the number of reporting points is denoted as Method 2. Method 1 and Method 2 will be described in detail below.

[0231] Method 1

[0232] In step S4421, when it is determined that the stylus is performing a preset writing operation, the electronic device determines that the number of reporting points is a first number, which is less than a first preset number. Based on this, the electronic device executes step S4511, that is, the electronic device predicts the predicted reporting points of the current frame according to the first number, and then displays and draws the actual reporting points and predicted reporting points of the current frame.

[0233] In Method 1, for example, the electronic device can determine the first quantity as the number of reporting points in two ways (denoted as Method 1.1 and Method 1.2), which are described in detail below.

[0234] Method 1.1

[0235] In some embodiments, the electronic device determines the second preset quantity as the first quantity, that is, the first quantity is the second preset quantity, and the second preset quantity is less than the first preset quantity. In other words, the first quantity is a preset value.

[0236] It should be understood that the second preset quantity is any value that is greater than 0 and less than the first preset quantity. For example, the second preset quantity can be 6, 10, 12, 15, 18, etc.

[0237] In implementation, the electronic device has a first preset number and a second preset number. When it is determined that the stylus will perform a preset writing operation, the electronic device will determine the second preset number as the reporting number and determine the predicted reporting number of the second preset number.

[0238] In the above method, the second preset quantity is directly determined as the first quantity of the reporting quantity. The logic is simple, the code is easy to implement, the practicality is stronger, and the processing time is saved to a certain extent.

[0239] Method 1.2

[0240] In some embodiments, the electronic device determines the number of reporting points as a first quantity based on a first preset quantity and a first probability.

[0241] In this approach, the first quantity is not a fixed value, but a value that changes with the first probability.

[0242] In implementation, the electronic device can obtain the first quantity as the reporting quantity by multiplying the first preset quantity and the first probability. That is, the electronic device determines the first quantity as the reporting quantity by proportionally discounting the first preset quantity according to the first probability. For example, if the first probability is 0.8 and the first preset quantity is 10, then the first quantity is 10 * 0.8 = 8.

[0243] When the product of the first preset quantity and the first probability is not an integer, in one example, the product of the first preset quantity and the first probability is rounded to obtain the first quantity. For example, if the first probability is 0.7 and the first preset quantity is 15, 15 * 0.7 = 10.5, and using rounding, the first quantity is 11. As another example, if the first probability is 0.7 and the first preset quantity is 6, 6 * 0.7 = 4.2, and using rounding, the first quantity is 4.

[0244] In another example, the product of the first preset quantity and the first probability is rounded down. For example, if the first probability is 0.7 and the first preset quantity is 15, 15 * 0.7 = 10.5, and by rounding down, the first quantity is 10. As another example, if the first probability is 0.7 and the first preset quantity is 6, 6 * 0.7 = 4.2, and by rounding down, the first quantity is 4.

[0245] In other examples, the product of the first preset quantity and the first probability is rounded up. For example, if the first probability is 0.7 and the first preset quantity is 15, 15 * 0.7 = 10.5, and by rounding up, the first quantity is 11.

[0246] In the above method, based on the first probability indicated by the prediction result, when it is determined that the stylus will perform a preset writing operation, the first quantity as the reporting quantity is determined according to the first probability and the first preset quantity. Since the reporting quantity is determined based on the probability of the actual scenario predicted, the obtained reporting quantity is more consistent with the actual scenario, that is, the accuracy of the obtained reporting quantity is higher, and the flexibility of reporting prediction is improved.

[0247] Method 2

[0248] The difference between Method 2 and Method 1 is that in Method 2, when it is determined that the stylus is performing a preset writing operation, in step S4422, the electronic device determines that the number of reporting points is 0, which is equivalent to canceling the prediction of the predicted reporting points. Then, in step S4512, the electronic device only draws and displays the actual reporting points of the current frame.

[0249] In method 400, it can be understood that under normal circumstances, the stylus can only be determined to perform a preset writing operation when the value of the first probability is relatively large. Therefore, the electronic device first determines whether the stylus performs a preset writing operation based on the first probability. Only when it is determined that the stylus performs a preset writing operation is the number of reported points reduced or canceled (i.e., the number of reported points is the first number or 0). This is equivalent to filtering out the first probability with a smaller value in advance. Therefore, the result of determining whether the stylus performs a preset writing operation is more consistent with the actual scenario, and the judgment result is more reliable and accurate. Therefore, the number of reported points determined based on this situation is also more consistent with the actual scenario. On the one hand, it can better reduce the phenomenon of flying lines or stroke retraction caused by the prediction of reported points, thus improving the user's writing experience. On the other hand, it can also better reduce the responsiveness delay in the writing process, thus improving the responsiveness experience.

[0250] Figure 13 This is a schematic flowchart of the reporting prediction method 500 provided in the embodiments of this application. The main difference between method 500 and method 400 is that in method 500, the electronic device does not need to determine in advance whether the stylus will perform a preset writing operation, but directly determines the number of reporting points based on the first probability indicated by the prediction result and the preset first preset number.

[0251] In step S510, the electronic device stylus writes on the electronic device to form N historical reporting points.

[0252] In step S520, the electronic device determines the feature information based on the reported information.

[0253] In step S530, the electronic device uses an attitude prediction model to process the feature information and obtain a prediction result, wherein the prediction result is used to indicate the first probability of the stylus performing a preset writing operation.

[0254] For a detailed description of steps S510 to S530, please refer to the relevant descriptions of steps S310 to S330 above, which will not be repeated here.

[0255] In step S540, the electronic device determines the number of reporting points based on the first probability and the first preset quantity.

[0256] There are three possible results for the number of reported points obtained from this step: the number of reported points is equal to 0, the number of reported points is greater than 0 and less than the first preset number, or the number of reported points is equal to the first preset number.

[0257] If the number of reported points is greater than 0 and less than or equal to the first preset number, the electronic device executes step S551; if the number of reported points is equal to 0, the electronic device executes step S552.

[0258] As can be seen, in the process of determining the number of reported points based on the first probability indicated by the prediction result, step S540 does not need to determine in advance whether the stylus will perform a preset writing operation. Instead, it directly determines the number of reported points based on the first probability and the preset first number. In other words, the electronic device directly determines the first number of reported points by proportionally discounting the first preset number based on the first probability.

[0259] In implementation, the electronic device can obtain the number of reported points by multiplying a first preset quantity and a first probability. For example, if the first probability is 0.8 and the first preset quantity is 10, then the number of reported points is 10 * 0.8 = 8. As another example, if the first probability is 0.2 and the first preset quantity is 10, then the number of reported points is 10 * 0.2 = 2.

[0260] For cases where the product of the first preset quantity and the first probability is not an integer, in one example, the product of the first preset quantity and the first probability is rounded to obtain the reported number. For example, if the first probability is 0.7 and the first preset quantity is 15, 15 * 0.7 = 10.5, and the reported number is 11 after rounding. Another example: if the first probability is 0.7 and the first preset quantity is 6, 6 * 0.7 = 4.2, and the reported number is 4 after rounding. Yet another example: if the first probability is 0.04 and the first preset quantity is 10, 10 * 0.04 = 0.4, and the reported number is 0 after rounding.

[0261] In another example, the product of the first preset quantity and the first probability is rounded down. For example, if the first probability is 0.7 and the first preset quantity is 15, 15 * 0.7 = 10.5, and by rounding down, the reported number of points is 10. As another example, if the first probability is 0.1 and the first preset quantity is 6, 6 * 0.1 = 0.6, and by rounding down, the reported number of points is 0.

[0262] It should be understood that in the above step S540, where the electronic device directly determines the number of reported points based on the first probability and the first preset quantity, when the number of reported points is equal to 0, it means that the probability of the stylus performing the preset writing operation (lifting the pen or turning the pen) is the highest (for example, the first probability is 0.1, the first preset quantity is 6, 6*0.1=0.6, and by rounding down, the number of reported points is 0). Ideally, it can be assumed that the stylus has indeed performed the preset writing operation (lifting the pen or turning the pen). Therefore, when the determined number of reported points is equal to 0, it is equivalent to canceling the prediction of the reported points. That is, the electronic device no longer performs the prediction of the reported points, but instead draws and displays the actual reported points of the current frame, which can very effectively reduce the flying lines or stroke retraction caused by the stylus performing the preset writing operation (lifting the pen or turning the pen). When the number of reported points is greater than 0 but less than the first preset number, it means that the probability of the stylus performing the preset writing operation (lifting the pen or turning the pen) is relatively high. The number of reported points can be appropriately reduced; that is, the predicted reported points for the current frame are predicted according to a number less than the first preset number. Because the number of reported points is reduced, the phenomenon of flying lines or stroke retraction caused by the stylus performing the preset writing operation (lifting the pen or turning the pen) can also be effectively reduced. When the number of reported points is equal to the first preset number, it means that the probability of the stylus performing the preset writing operation (lifting the pen or turning the pen) is very low. The electronic device can predict the predicted reported points for the current frame according to the first preset number, that is, determine the first preset number of predicted reported points.

[0263] It should be noted that although the method of determining the number of reported points based on the first preset quantity and the first probability in step S540 is similar to method 1.2 in method 400, the output results of the two are different. Because method 1.2 needs to determine in advance whether the stylus will perform the preset writing operation based on the comparison of the two probabilities, the number of reported points (i.e., the first quantity) will only be determined by the first probability and the first preset quantity if it is determined that the stylus will perform the preset writing operation. In this case, it is equivalent to filtering out the first probability with smaller values. Therefore, the values ​​of the first probabilities used to calculate the number of reported points (i.e., the first quantity) are all large. Therefore, the possibility that the number of reported points (i.e., the first quantity) obtained by method 400 based on method 1.2 is 0 is very small, and it is almost always greater than 0. In contrast, in method 500, since it is not necessary to determine in advance whether the stylus will perform the preset writing operation based on the probability comparison, but instead directly determines the number of reported points by proportionally discounting the first preset quantity based on the first probability, the value of the first probability used to calculate the number of reported points can be large or small. Therefore, in method 500, the number of reported points obtained by the first probability and the first preset quantity may be 0, or it may be greater than 0 and less than or equal to the first preset quantity.

[0264] In step S551, when the number of reported points is greater than 0 and less than or equal to the first preset number, the electronic device predicts the predicted reported points of the current frame according to the number of reported points.

[0265] In step S552, when the number of reported points is equal to 0, the electronic device draws and displays the actual reported points of the current frame.

[0266] For a detailed description of steps S551 and S552, please refer to the relevant descriptions of steps S351 and S352 in Method 300 above, which will not be repeated here.

[0267] In method 500, since it is not necessary to determine in advance whether the stylus will perform the preset writing operation, but the number of reported points can be determined directly based on the first probability and the preset first number, the logic is simple and the code is easy to implement, which also saves processing time to a certain extent. In addition, since the number of reported points is determined based on the probability (first probability) of the predicted actual scene, the number of reported points obtained is more consistent with the actual scene, that is, the accuracy of the number of reported points obtained is high, and the flexibility of reported point prediction is improved.

[0268] Scenario 2: The prediction result is used to indicate whether the stylus should perform the preset writing operation.

[0269] Figure 14 This is a schematic flowchart of the reporting prediction method 600 provided in the embodiments of this application. While both method 600 and method 400 can determine whether the stylus performs a preset writing operation, the difference between method 600 and method 400 lies in the fact that, due to the different content indicated by the prediction result, method 600 does not require a first probability based on the prediction result indication of method 400 to further determine whether the stylus performs a preset writing operation. Instead, it can directly determine whether the stylus performs a preset writing operation based on the prediction result. Therefore, the number of reported points can be directly determined based on the content indicated by the prediction result.

[0270] In step S610, the electronic device stylus writes on the electronic device to form N historical reporting points.

[0271] In step S620, the electronic device determines the feature information based on the reported information.

[0272] For a detailed description of steps S610 and S620, please refer to the relevant descriptions of steps S310 and S320 above, which will not be repeated here.

[0273] In step S630, the electronic device uses an attitude prediction model to process the feature information and obtain a prediction result, wherein the prediction result is used to indicate whether the stylus performs a preset writing operation.

[0274] In this step, any method can be used to indicate whether the stylus performs a preset writing operation. There are no restrictions here, as long as it is possible to distinguish whether the stylus performs a preset writing operation or not.

[0275] For example, "1" and "0" can be used to indicate whether the stylus performs a preset writing operation. For instance, "0" indicates that the stylus did not perform a preset writing operation, and "1" indicates that the stylus performed a preset writing operation. As another example, "1" indicates that the stylus did not perform a preset writing operation, and "0" indicates that the stylus performed a preset writing operation.

[0276] If the prediction result indicates that the stylus has not performed the preset writing operation, the electronic device executes step S642, that is, the electronic device determines that the number of reporting points is a first preset number. Based on this, the electronic device executes step S652, that is, the electronic device predicts the predicted reporting points of the current frame according to the first preset number, and then displays and draws the actual reporting points and predicted reporting points of the current frame.

[0277] When the prediction result instructs the stylus to perform a preset writing operation, the electronic device can execute either step S6411 or step S6412, depending on the internal design of the electronic device. For ease of description, the method by which the electronic device executes step S6411 to determine the number of reporting points is referred to as method 2.1, and the method by which the electronic device executes step S6412 to determine the number of reporting points is referred to as method 2.2. Method 2.1 and method 2.2 will be described in detail below.

[0278] Method 2.1

[0279] In some embodiments, in step S6411, the electronic device determines the number of reporting points to be a second preset number, which is less than a first preset number. Based on this, the electronic device executes step S6511, that is, the electronic device predicts the predicted reporting points of the current frame according to the second preset number, and then displays and draws the actual reporting points and predicted reporting points of the current frame. The second preset number is a pre-set value.

[0280] It should be understood that the second preset quantity is any preset value that is greater than 0 and less than the first preset quantity. For example, the second preset quantity can be 6, 10, 12, 15, 18, etc.

[0281] In implementation, the electronic device has a first preset number and a second preset number. When the prediction result instructs the stylus to perform a preset writing operation, the electronic device determines the second preset number as the reporting number and determines the predicted reporting number of the second preset number.

[0282] Method 2.1 is the same as Method 1.1 of Method 400, both of which determine the second preset quantity as the reporting quantity.

[0283] Method 2.2

[0284] The difference between Method 2.2 and Method 2.1 is that in Method 2.2, when the prediction result instructs the stylus to perform a preset writing operation, in step S6412, the electronic device determines that the number of reporting points is 0, which is equivalent to canceling the prediction of the reporting points. Then, in step S6512, the electronic device only draws and displays the actual reporting points of the current frame.

[0285] It should be understood that the above-described method for predicting reporting points is merely illustrative and should not be construed as limiting the embodiments of this application. Any method of determining the number of reporting points through the writing characteristics of the writing process is within the protection scope of the embodiments of this application.

[0286] In other embodiments, at least one writing feature in the feature information can be comprehensively judged. For example, when the result of each writing feature meets its respective preset condition, it is determined that the stylus performs a preset writing operation. When it is determined that the stylus performs a preset writing operation, a first quantity is determined as the reporting quantity using method 1.1 or method 1.2. When it is determined that the stylus does not perform a preset writing operation, the first preset quantity is determined as the reporting quantity. For example, when the feature information includes trajectory curvature, the preset condition for trajectory curvature is that the trajectory curvature is less than a certain value. When this preset condition is met, the stylus can be considered to have performed a preset writing operation. When the feature information includes pressure change rate, the preset condition for pressure change rate is that the pressure change rate is greater than a certain value. When this preset condition is met, the stylus can be considered to have performed a preset writing operation. When the feature information includes speed change rate, the preset condition for speed change rate is that the speed change rate is greater than or less than a certain value. When this preset condition is met, the stylus can be considered to have performed a preset writing operation. When the feature information includes trajectory change rate, the preset condition for trajectory change rate is that the trajectory change rate is greater than or less than a certain value. When this preset condition is met, the stylus can be considered to have performed a preset writing operation. When the feature information includes trajectory curvature, pressure change rate, speed change rate, and trajectory change rate, the stylus can only be considered to have performed a preset writing operation when all four writing features meet their respective preset conditions.

[0287] The above provides a detailed description of the point prediction method in the embodiments of this application. The following describes the training process of the attitude prediction model.

[0288] Figure 15 This is a schematic flowchart of the training process 700 of the attitude prediction model provided in this application embodiment. The explanation will continue using an electronic device as an example.

[0289] In step S710, the electronic device acquires multiple sample reporting data. The sample reporting data is generated based on multiple historical reporting data generated when the stylus is writing on the electronic device. The multiple sample reporting data includes a first type of sample reporting data and a second type of sample reporting data. The first type of sample reporting data is the data collected when the stylus performs a normal writing operation, and the second type of sample reporting data is the data collected when the stylus performs a preset writing operation.

[0290] It should be understood that each sample reporting point data here can correspond to the reporting point information in method 300 above, and multiple historical reporting points of each sample reporting point data can correspond to N historical reporting points in method 300. For a detailed description of multiple historical reporting points and the reporting point information of multiple historical reporting points, please refer to the relevant description above, which will not be repeated here.

[0291] It should also be understood that normal writing operations can be any writing operation other than the preset writing operations, such as writing a straight line.

[0292] In step S720, the electronic device processes the data of each sample reporting point to obtain the feature information of each sample reporting point.

[0293] For a detailed description of the process by which the electronic device processes the data of each sample reporting point to obtain feature information, please refer to the relevant description of step S320 in method 300 above, which will not be repeated here.

[0294] In step S730, the electronic device uses the feature information of each sample reporting point data to train the neural network model and obtain the attitude prediction model.

[0295] In practice, when the fit index (e.g., R2 score) of the neural network model is greater than the threshold (e.g., 0.95), it means that the neural network model has converged and meets the accuracy requirements. Therefore, it is not necessary to train the neural network model again, thus obtaining the pose prediction model.

[0296] It should be understood that since the multiple sample reporting data includes the first type of sample reporting data collected when the stylus performs a normal writing operation and the second type of sample reporting data collected when the stylus performs a normal writing operation, the posture prediction model obtained after training the neural network model based on the above multiple sample reporting data is equivalent to establishing the relationship between the writing operation and the prediction result. Therefore, when using the posture prediction model for prediction, the feature information of the stylus writing is used as input, and the posture prediction model can output the prediction result related to the preset writing operation after processing the feature information.

[0297] The attitude prediction model in this application is based on an arbitrary form of network model, and no limitations are made here.

[0298] In some embodiments, the pose prediction model is based on a deep neural network (DNN) model. The DNN model includes an input layer, hidden layers, and an output layer, each containing multiple neurons.

[0299] The number of neurons in the input layer is the same as the number of writing features. The number of neurons in the output layer is the same as the number of elements included in the prediction result. For example, if the prediction result indicates a first probability, the number of neurons in the output layer is 1; if the prediction result indicates a first probability and a second probability, the number of neurons in the output layer is 2; and if the prediction result indicates whether the stylus will perform a preset writing operation, the number of neurons in the output layer is 1. There are multiple hidden layers, each containing multiple neurons.

[0300] In addition, the DNN model is also called a fully connected neural network. One of its characteristics is that the layers are fully connected, that is, all neurons in the layers are connected. In other words, any neuron in the i-th layer is connected to any neuron in the (i+1)-th layer.

[0301] The embodiments of this application do not limit the number of hidden layers or the number of neurons contained in each layer.

[0302] In some embodiments, the pose prediction model has two hidden layers, referred to as the first hidden layer and the second hidden layer, with each hidden layer including multiple neurons.

[0303] Figure 16 This is a network structure diagram of the attitude prediction model provided in an embodiment of this application. (Reference) Figure 16 The input layer of the DNN model consists of i neurons (e.g., 4 or 6), with each neuron corresponding to a writing feature. x1 to xi represent the i writing features of the input layer. z10 represents the first hidden layer, and z20 represents the second hidden layer.

[0304] For the output layer of the DNN model, in an embodiment where the prediction result is used to indicate a first probability and a second probability, the output layer includes two neurons, one neuron corresponding to the first probability and the other neuron corresponding to the second probability. For example, y1 represents the first probability and y2 represents the second probability.

[0305] In an embodiment where the prediction result is used to indicate a first probability, the output layer includes one neuron corresponding to the first probability, for example, y1 represents the first probability.

[0306] In an embodiment where the prediction result is used to determine whether the stylus performs a preset writing operation, the output layer includes one neuron that is applied to an identifier indicating whether the stylus performs a preset writing operation. For example, y1 represents an identifier of "yes" or "no". When y1 represents "yes", it means that the stylus performs the preset writing operation. When y1 represents "no", it means that the stylus does not perform the preset writing operation.

[0307] Figure 17 This is another network structure diagram of the pose prediction model provided in the embodiments of this application. As a specific example, refer to... Figure 17 The input layer consists of 6 neurons, corresponding to x1 to x6, which represent the 6 writing features of the input layer, such as trajectory curvature, pressure change rate, velocity change rate, trajectory change rate, movement distance, and movement trajectory length. z11 to z1128 represent the 128 neurons in the first hidden layer z10. It should be understood that only z11, z12, z1127, and z1128 are shown in the diagram; the remaining neurons are represented by ellipses. Taking the neuron corresponding to x1 as an example, this neuron is connected to every neuron in z11 to z1128. Figure 17 Only the connection between this neuron and the four neurons in z10 is shown. z21 to z264 represent the 64 neurons in the second hidden layer z20. It should be understood that only z21, z22, z263, and z264 are shown in the figure; the remaining neurons are represented by ellipses. The output layer consists of two neurons, one corresponding to the first probability y1 and the other corresponding to the second probability y2.

[0308] For a general DNN model, the input layer corresponds to multiple input features (e.g., 6 writing features). These input features do not require non-linear processing by activation functions; each feature is input to the next layer via different connections, i.e., different weights *w*. Each neuron in the next layer sums all inputs with weights, adds a bias term, and then passes through an activation function to obtain the output, which also becomes the input to the next layer... This process continues until the output layer is reached. Similarly, each neuron in the output layer sums all inputs with weights, adds a bias term, and then passes through an activation function to obtain the output. The number of neurons in the input layer is determined by the number of features input to the actual problem (e.g., 6 writing features), while the number of neurons in the output layer is determined by the actual problem to be solved (e.g., first probability and second probability, or first probability, or instructing the stylus whether to perform a writing operation). The number of hidden layers and the number of neurons in each layer need to be flexibly adjusted according to the complexity of the problem; the more complex the problem, the more layers and neurons are required.

[0309] The above details the prediction method for reporting points provided in the embodiments of this application. The following will combine... Figures 18 to 19This application describes a prediction apparatus and electronic device for reporting points according to embodiments thereof.

[0310] Figure 18 This is an exemplary block diagram of the reporting prediction device 800 provided in this application embodiment. The prediction device 800 may be an electronic device or a chip in an electronic device. The prediction device 800 is used to execute the various processes and steps corresponding to the electronic device in the methods 300 to 600 described above. The prediction device 800 includes: a processing unit 810.

[0311] Processing unit 810 performs the following steps: acquiring point information of N historical points formed by the stylus writing on the electronic device, where N is an integer greater than 1; determining feature information based on the point information, the feature information including at least one of the following writing features: trajectory curvature, pressure change rate, speed change rate, and trajectory change rate, wherein trajectory curvature is used to represent the degree of curvature of the trajectory formed based on the N historical points, pressure change rate is used to represent the degree of pressure change during the process of the stylus writing on the electronic device to form N historical points, speed change rate is used to represent the degree of speed change during the process of the stylus writing on the electronic device to form N historical points, and trajectory change rate is used to represent the degree of curvature of the trajectory formed based on the N historical points. The degree of change of the trajectory formed by historical reporting points compared to the trajectories formed by multiple historical reporting points obtained in the previous prediction process; the feature information is processed using a posture prediction model to obtain prediction results, which are used to indicate the first probability of the stylus performing a preset writing operation or to indicate whether the stylus performs a preset writing operation, which is a pen lifting operation or a turning operation; based on the prediction results, the number of reporting points is determined, which is greater than or equal to 0 and less than or equal to a first preset number; if the number of reporting points is greater than 0 and less than or equal to the first preset number, the predicted reporting points of the current frame are predicted according to the number of reporting points; or, if the number of reporting points is equal to 0, the display screen is controlled to draw and display the actual reporting points of the current frame.

[0312] Optionally, the prediction result is used to indicate a first probability; and the processing unit 810 is specifically used to: determine whether the stylus performs a preset writing operation based on the first probability; if it is determined that the stylus performs a preset writing operation, determine the number of reported points to be a first number or 0; or, if it is determined that the stylus does not perform a preset writing operation, determine the number of reported points to be a first preset number.

[0313] Optionally, the first quantity is the second preset quantity.

[0314] Optionally, the processing unit 810 is specifically used to: when it is determined that the stylus is performing a preset writing operation, determine the number of reported points as a first number based on a first preset number and a first probability.

[0315] Optionally, the prediction result is also used to indicate a second probability that the stylus has not performed a preset writing operation; and the processing unit 810 is specifically used to: determine that the stylus has performed a preset writing operation when the first probability is greater than the second probability; or, determine that the stylus has not performed a preset writing operation when the first probability is less than the second probability.

[0316] Optionally, the prediction result is used to indicate a first probability; and the processing unit 810 is specifically used to: determine the number of reporting points based on a first preset number and a first probability.

[0317] Optionally, the prediction result is used to indicate whether the stylus performs a preset writing operation; and the processing unit 810 is specifically used to: determine the number of reporting points as a second preset number or 0 when the prediction result indicates that the stylus performs a preset writing operation, wherein the second preset number is less than the first preset number; or, determine the number of reporting points as the first preset number when the prediction result indicates that the stylus does not perform a preset writing operation.

[0318] Optionally, the reporting information includes the coordinate information of each historical reporting point, and the feature information includes the trajectory curvature; and the processing unit 810 is specifically used to: determine the moving distance and the moving trajectory length based on the coordinate information of each historical reporting point, wherein the moving distance is used to represent the straight-line distance between the first historical reporting point and the Nth historical reporting point among N historical reporting points, the time of the first historical reporting point is earlier than the time of the Nth historical reporting point, and the moving trajectory length is used to represent the length of the trajectory formed based on the N historical reporting points; and determine the trajectory curvature based on the moving distance and the moving trajectory length.

[0319] Optionally, the feature information may also include the distance traveled and / or the length of the travel trajectory.

[0320] Optionally, the reporting information includes pressure information of each historical reporting point, and the feature information includes the pressure change rate; and the processing unit 810 is specifically used to: determine the pressure change rate based on the pressure information of at least some of the historical reporting points.

[0321] Optionally, the processing unit 810 is specifically used to: determine the ratio between the maximum pressure among the pressures of N historical reporting points and the pressure of the Nth historical reporting point as the pressure change rate, wherein the Nth historical reporting point is the latest reporting point among the N historical reporting points.

[0322] Optionally, the reporting information includes the coordinates and time information of each historical reporting point, and the feature information includes the rate of change of velocity;

[0323] In addition, the processing unit 810 is specifically used to: determine the rate of change of velocity based on the coordinate information and time information of at least some historical reporting points.

[0324] Optionally, the processing unit 810 is specifically used to: determine N-1 speeds based on the coordinate information and time information of two adjacent historical reporting points among N historical reporting points; determine the ratio between the maximum speed among the N-1 speeds and the N-1th speed as the speed change rate, where the N-1th speed is the latest speed among the N-1 speeds.

[0325] Optionally, the reporting point information includes the coordinate information of each historical reporting point, and the feature information includes the trajectory change rate; and the processing unit 810 is specifically used to: determine the length of a first moving trajectory based on the coordinate information of each historical reporting point, the first moving trajectory length being used to represent the length of the trajectory formed based on N historical reporting points; and determine the trajectory change rate based on the first moving trajectory length and the second moving trajectory length, the second moving trajectory length being used to represent the length of the trajectory formed based on multiple historical reporting points obtained in the previous prediction process.

[0326] Optionally, the processing unit 810 is further configured to: acquire multiple sample reporting data, the sample reporting data being generated based on multiple historical reporting points generated when the stylus is writing on the electronic device, the multiple sample reporting data including a first type of sample reporting data and a second type of sample reporting data, the first type of sample reporting data being data collected when the stylus performs a normal writing operation, and the second type of sample reporting data being data collected when the stylus performs a preset writing operation; process each sample reporting data to obtain feature information of each sample reporting data; and use the feature information of each sample reporting data to train a neural network model to obtain a posture prediction model.

[0327] It should be understood that the processing unit 810 can be used for various steps executed by the electronic device. For a detailed description, please refer to the relevant description above, which will not be repeated here.

[0328] In the embodiments of this application, Figure 18 The device in the middle can also be a chip or a chip system, such as a system on chip (SoC).

[0329] Figure 19 This is a schematic structural diagram of the electronic device 900 provided in an embodiment of this application. The electronic device 900 is used to execute the corresponding steps and / or processes in the above method embodiments.

[0330] Electronic device 900 includes a processor 910, a transceiver 920, a memory 930, and a display screen 940. The processor 910, transceiver 920, and memory 930 communicate with each other via internal connections. The processor 910 can perform the functions of a processor 910 in various possible implementations of electronic device 900. The memory 930 stores instructions, and the processor 910 executes the instructions stored in the memory 930; in other words, the processor 910 can call these stored instructions to implement the functions of the processor 910 in electronic device 900. The display screen 940 is used to display images, videos, etc., for example, to display content written by the user.

[0331] Optionally, the memory 930 may include read-only memory and random access memory, and provide instructions and data to the processor. A portion of the memory may also include non-volatile random access memory. For example, the memory may also store device type information. The processor 910 may be used to execute instructions stored in the memory, and when the processor 910 executes instructions stored in the memory, the processor 910 is used to perform the various steps and / or processes of the method embodiments corresponding to the network device or terminal device described above.

[0332] also, Figure 19 The electronic device 900 shown can correspond to Figure 6 The processor 910 in the illustrated electronic device 100 can correspond to the processor 110 in the electronic device 100, and the transceiver 920 in the electronic device 900 can correspond to antenna 1 and antenna 2 in the electronic device 100. The memory 930 in the electronic device 900 can correspond to the internal memory 121 in the electronic device 100.

[0333] The electronic device 900 is used to execute the various processes and steps corresponding to the electronic device in methods 300 to 600. The steps executed by the processor 910 can correspond to the various steps executed by the processing unit 810 in the device 800. For details, please refer to the above description and will not be repeated here.

[0334] It should be understood that, in the embodiments of this application, the processor of the above-described device can be a central processing unit (CPU), which can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0335] In implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software. The steps of the method disclosed in the embodiments of this application can be directly manifested as execution by a hardware processor, or as a combination of hardware and software units within the processor. The software units can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory, and the processor executes the instructions in the memory, combining them with its hardware to complete the steps of the above method. To avoid repetition, detailed descriptions are omitted here.

[0336] This application provides a computer program product that, when run on a terminal device, causes the terminal device to execute the technical solutions described in the above embodiments. Its implementation principle and technical effects are similar to those of the related embodiments described above, and will not be repeated here.

[0337] This application provides a readable storage medium containing instructions that, when executed by a terminal device, cause the terminal device to perform the technical solution described in the above embodiments. The implementation principle and technical effects are similar and will not be repeated here.

[0338] This application provides a chip for executing instructions. When the chip is running, it executes the technical solutions described in the above embodiments. Its implementation principle and technical effects are similar and will not be repeated here.

[0339] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., high-density digital video discs (DVDs)), or semiconductor media (e.g., solid-state disks (SSDs)).

[0340] It should be understood that the term "embodiment" used throughout the specification means that a specific feature, structure, or characteristic related to an embodiment is included in at least one embodiment of this application. Therefore, various embodiments throughout the specification do not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0341] It should also be understood that in this application, “when…”, “if” and “if” all refer to the UE or base station taking corresponding actions under certain objective circumstances, and are not time-limited, nor do they require the UE or base station to perform a judgment action, nor do they imply any other limitations.

[0342] Those skilled in the art will understand that the various numerical designations such as "first," "second," etc., involved in this application are merely for the convenience of description and are not intended to limit the scope of the embodiments of this application, nor do they indicate the order of sequence.

[0343] In this application, the use of singular pronouns to denote "one or more" rather than "one and only one," unless otherwise specified. In this application, unless otherwise specified, "at least one" is intended to mean "one or more," and "more than" is intended to mean "two or more."

[0344] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three cases: A exists alone, A and B exist simultaneously, and B exists alone. Here, A can be singular or plural, and B can be singular or plural.

[0345] In this document, the terms "at least one of..." or "at least one of..." refer to all or any combination of the listed items. For example, "at least one of A, B, and C" can mean: A exists alone, B exists alone, C exists alone, A and B exist simultaneously, B and C exist simultaneously, and A, B, and C exist simultaneously. A can be singular or plural, B can be singular or plural, and C can be singular or plural.

[0346] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0347] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0348] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0349] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0350] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0351] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0352] The same or similar parts between the various embodiments in this application can be referred to mutually. In the various embodiments of this application, and in the various implementation methods / methods / implementations within each embodiment, unless otherwise specified or logically conflicting, the terminology and / or descriptions between different embodiments and between the various implementation methods / methods / implementations within each embodiment are consistent and can be mutually referenced. The technical features in different embodiments and the various implementation methods / methods / implementations within each embodiment can be combined according to their inherent logical relationships to form new embodiments, implementation methods, methods, or implementation approaches. The above-described embodiments of this application do not constitute a limitation on the scope of protection of this application.

[0353] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of protection of the claims. In conclusion, the above description is merely a preferred embodiment of the technical solution of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for predicting a point report, applied in an electronic device, characterized in that, The method comprises: obtaining point information of N historical points formed by a stylus writing on an electronic device, N being an integer greater than 1; determining feature information according to the point information, the feature information comprising at least one writing feature: trajectory curvature, pressure change rate, speed change rate, and trajectory change rate, wherein the trajectory curvature is used to represent the bending degree of a trajectory formed based on the N historical points, the pressure change rate is used to represent the pressure change degree in the process of the stylus writing on the electronic device to form the N historical points, the speed change rate is used to represent the speed change degree in the process of the stylus writing on the electronic device to form the N historical points, and the trajectory change rate is used to represent the change degree of the trajectory formed based on the N historical points compared with the trajectory formed based on a plurality of historical points obtained in a last prediction process; processing the feature information by using a posture prediction model to obtain a prediction result, the prediction result being used to indicate a first probability that the stylus performs a preset writing operation, the preset writing operation being a lifting operation or a turning operation; determining whether the stylus performs the preset writing operation according to the first probability; in a case where it is determined that the stylus performs the preset writing operation, determining that the number of points is a first number or 0; or in a case where it is determined that the stylus does not perform the preset writing operation, determining that the number of points is a first preset number, the number of points being greater than or equal to 0 and less than or equal to the first preset number; in a case where the number of points is greater than 0 and less than or equal to the first preset number, predicting a predicted point of a current frame according to the number of points; or in a case where the number of points is equal to 0, drawing and displaying a real point of the current frame.

2. The prediction method of claim 1, wherein, The first number is a second preset number.

3. The prediction method of claim 1, wherein, The determining, in a case where it is determined that the stylus performs the preset writing operation, that the number of points is the first number comprises: in a case where it is determined that the stylus performs the preset writing operation, determining, according to the first preset number and the first probability, that the number of points is the first number.

4. The prediction method of any one of claims 1 to 3, characterized in that, The prediction result is further used to indicate a second probability that the stylus does not perform the preset writing operation; and the determining, according to the first probability, whether the stylus performs the preset writing operation comprises: in a case where the first probability is greater than the second probability, determining that the stylus performs the preset writing operation; or in a case where the first probability is less than the second probability, determining that the stylus does not perform the preset writing operation.

5. The prediction method of claim 2, wherein, The second preset number is less than the first preset number.

6. The prediction method of any one of claims 1 to 3, characterized in that, The point information comprises coordinate information of each historical point, and the feature information comprises the trajectory curvature. The determining, according to the point information, the feature information comprises: According to the coordinate information of the respective historical report points, a moving distance and a moving track length are determined, the moving distance is used to represent a straight-line distance between a first historical report point and an Nth historical report point in the N historical report points, the time of the first historical report point is earlier than the time of the Nth historical report point, and the moving track length is used to represent a length of a track formed based on the N historical report points; According to the moving distance and the moving track length, the track curvature is determined.

7. The prediction method of claim 6, wherein, The feature information further includes the moving distance and / or the moving track length.

8. The prediction method of any one of claims 1 to 3, characterized in that, The report point information includes pressure information of the respective historical report points, and the feature information includes the pressure change rate; Furthermore, the determining of the feature information according to the report point information includes: According to the pressure information of at least part of the historical report points, the pressure change rate is determined.

9. The prediction method of claim 8, wherein, The determining of the pressure change rate according to the pressure information of at least part of the historical report points includes: A ratio between a maximum pressure in the pressures of the N historical report points and a pressure of an Nth historical report point is determined as the pressure change rate, the Nth historical report point being a report point with the latest time in the N historical report points.

10. The prediction method of any one of claims 1 to 3, characterized in that, The report point information includes coordinate information and time information of the respective historical report points, and the feature information includes the speed change rate; Furthermore, the determining of the feature information according to the report point information includes: According to the coordinate information and time information of at least part of the historical report points, the speed change rate is determined.

11. The prediction method of claim 10, wherein, The determining of the speed change rate according to the coordinate information and time information of at least part of the historical report points includes: According to the coordinate information and time information of adjacent two historical report points in the N historical report points, N-1 speeds are determined; A ratio between a maximum speed in the N-1 speeds and an N-1th speed is determined as the speed change rate, the N-1th speed being a speed with the latest time in the N-1 speeds. The report point information includes coordinate information of the respective historical report points, and the feature information includes the track change rate; 12. The prediction method of any one of claims 1 to 3, characterized in that, Furthermore, the determining of the feature information according to the report point information includes: According to the coordinate information of the respective historical report points, a first moving track length is determined, the first moving track length being used to represent a length of a track formed based on the N historical report points; According to the first moving track length and a second moving track length, the track change rate is determined, the second moving track length being used to represent a length of a track formed based on a plurality of historical report points acquired in a last prediction process. Before the report point information of N historical report points formed by the stylus writing on the electronic device is acquired, the method further includes:

13. The prediction method of any one of claims 1 to 3, characterized in that, A plurality of sample report point data is acquired, the sample report point data being generated based on a plurality of historical report points generated when the stylus writes on the electronic device, the plurality of sample report point data including first-type sample report point data and second-type sample report point data, the first-type sample report point data being data collected when the stylus performs a normal writing operation, and the second-type sample report point data being data collected when the stylus performs the preset writing operation; ​ Processing each of the sample report point data to obtain feature information of each of the sample report point data; Training a neural network model using the feature information of each of the sample report point data to obtain the posture prediction model.

14. An electronic device, comprising: Comprise: a memory for storing computer instructions; a processor for calling the computer instructions stored in the memory to execute the method of any one of claims 1 to 13.

15. A computer-readable storage medium, characterized in that, For storing computer instructions, the computer instructions are used to implement the method of any one of claims 1 to 13.

16. A chip, characterized by The chip comprises: a memory for storing instructions; a processor for calling and running the instructions from the memory, so that the electronic device installed with the chip system executes the method of any one of claims 1 to 13.

Images