Training method of point prediction model and touch point prediction method and device

Abstract

The invention provides a training method of a point prediction model and a touch point prediction method and device. The method comprises the steps of obtaining a first candidate point prediction model and a first sample point sequence set of the ith training round; inputting a first candidate point prediction model one by one according to a first training sequence of each first sample point sequence so as to obtain a first hidden state of each first sample point one by one in sequence; training according to the first hidden state and a second hidden state sequence in the first candidate point prediction model to obtain a second candidate point prediction model; carrying out hidden state zero clearing on the second candidate point prediction model to obtain a third candidate point prediction model; and obtaining a second sample point sequence set of the (i + 1)-th training round to continuously train the third candidate point prediction model to obtain a target point prediction model. The consumption degree and the consumption cost of algorithm resources are reduced, the operation efficiency and the operation precision of a target point prediction model are improved, and the hand-following delay degree between a user side and a human hand is reduced.

Description

Technical Field

[0001] The present disclosure relates to the technical field of data processing, and in particular to artificial intelligence technologies such as deep learning. Background Art

[0002] With the development of technology, touch devices play an increasingly important role in people's work and life. During the use of touch devices, in response to human touch operations, the touch devices may experience delays. Summary of the Invention

[0003] The present disclosure aims to solve at least one of the technical problems in the related art to some extent.

[0004] To this end, a first aspect of the present disclosure proposes a training method for a point prediction model.

[0005] A second aspect of the present disclosure proposes a touch point prediction method.

[0006] A third aspect of the present disclosure proposes a training device for a point prediction model.

[0007] A fourth aspect of the present disclosure proposes a touch point prediction device.

[0008] A fifth aspect of the present disclosure proposes an electronic device.

[0009] A sixth aspect of the present disclosure proposes a computer-readable storage medium.

[0010] A seventh aspect of the present disclosure proposes a chip.

[0011] A first aspect of the present disclosure proposes a training method for a point prediction model. The method includes: obtaining a first candidate point prediction model to be trained and a first sample point sequence set in the current i-th training round; obtaining a first training sorting of each first sample point sequence in the first sample point sequence set, and inputting each first sample point sequence into the first candidate point prediction model one by one according to the first training sorting, so as to obtain a first hidden state of each first sample point in any sample point sequence one by one in order through the first candidate point prediction model; obtaining a second hidden state sequence cached in history in the first candidate point prediction model, so as to perform model training on the first candidate point prediction model in the i-th training round according to the first hidden state and the second hidden state sequence, and obtaining a second candidate point prediction model after the i-th training round ends; clearing the hidden state of the second candidate point prediction model to obtain a third candidate point prediction model; obtaining a second sample point sequence set in the (i + 1)-th training round, and continuing to perform model training on the third candidate point prediction model according to the second sample point sequence set until the training ends, and obtaining a trained target point prediction model.

[0012] The second aspect of the present disclosure provides a touch point prediction method, including: obtaining a trained target point prediction model, where the target point prediction model is obtained based on the point prediction model training method proposed in the first aspect above; obtaining the candidate touch points currently received on the user side, and inputting the candidate touch point information of the candidate touch points into the target point prediction model, to obtain the candidate hidden state of the candidate touch points through the target hidden layer in the target point prediction model; obtaining the hidden state cache sequence in the target point prediction model, where the hidden state cache sequence includes the candidate hidden state; in response to the hidden state cache sequence satisfying a preset model prediction condition, inputting the hidden state cache sequence into the downstream model layer of the target hidden layer in the target point prediction model to predict the touch points, and outputting the target prediction touch point sequence corresponding to the candidate touch points, where the target prediction touch point sequence is used to represent the touch sequence received by the user side based on the candidate touch points within a future time range.

[0013] The third aspect of the present disclosure provides a training device for a point prediction model. The device includes: a first acquisition module, configured to acquire a first candidate point prediction model to be trained and a first sample point sequence set of the current i-th training round; a first training module, configured to obtain the first training sorting of each first sample point sequence in the first sample point sequence set, and input each first sample point sequence into the first candidate point prediction model one by one according to the first training sorting, so as to obtain the first hidden state of each first sample point in any sample point sequence one by one through the first candidate point prediction model; a second training module, configured to obtain the second hidden state sequence cached in history in the first candidate point prediction model, and perform model training on the first candidate point prediction model for the i-th training round according to the first hidden state and the second hidden state sequence, to obtain a second candidate point prediction model after the end of the i-th training round; a clearing module, configured to clear the hidden state of the second candidate point prediction model to obtain a third candidate point prediction model; a third training module, configured to acquire a second sample point sequence set of the (i + 1)-th training round, and continue to perform model training on the third candidate point prediction model according to the second sample point sequence set until the training ends, to obtain a trained target point prediction model.

[0014] A fourth aspect of the present disclosure provides a touch point prediction device, which includes: a second acquisition module, configured to acquire a trained target point prediction model, where the target point prediction model is obtained based on the point prediction model training device proposed in the third aspect above; a third acquisition module, configured to acquire candidate touch points currently received on a user terminal, and input candidate touch point information of the candidate touch points into the target point prediction model, and obtain candidate hidden states of the candidate touch points through a target hidden layer in the target point prediction model; a fourth acquisition module, configured to acquire a hidden state cache sequence in the target point prediction model, where the hidden state cache sequence includes the candidate hidden states; a prediction module, configured to, in response to the hidden state cache sequence satisfying a preset model prediction condition, input the hidden state cache sequence into a downstream model layer of the target hidden layer in the target point prediction model to perform prediction of touch points, and output a target prediction touch point sequence corresponding to the candidate touch points, where the target prediction touch point sequence is used to represent a touch sequence received by the user terminal based on the candidate touch points within a future time range.

[0015] A fifth aspect of the present disclosure provides an electronic device, including: a processor; a memory for storing executable instructions of the processor; wherein, the processor is configured to execute the instructions to implement the point prediction model training method proposed in the first aspect above and / or the touch point prediction method proposed in the second aspect above.

[0016] A sixth aspect of the present disclosure provides a computer-readable storage medium, when instructions in the computer-readable storage medium are executed by a processor of an electronic device, enabling the electronic device to execute the point prediction model training method proposed in the first aspect above and / or the touch point prediction method proposed in the second aspect above.

[0017] A seventh aspect of the present disclosure provides a chip, including one or more interface circuits and one or more processors; the interface circuit is configured to receive a signal and send the signal to the processor, the signal includes computer instructions stored in a memory, and when the processor executes the computer instructions, enabling the chip to execute the steps of the point prediction model training method proposed in the first aspect above and / or the touch point prediction method proposed in the second aspect above.

[0018] The training method of the point prediction model and the touch point prediction method proposed by the present disclosure obtain the first sample point sequence set of the i-th training round to train the first candidate point prediction model, obtain the second candidate point prediction model, clear the hidden state of the second candidate point prediction model to obtain the third candidate point prediction model, and return to obtain the second sample point sequence set of the (i + 1)-th training round to continue training the third candidate point prediction model until the training ends to obtain the trained target point prediction model. In the present disclosure, each first sample point sequence is sequentially input into the first candidate point prediction model, realizing the training of multiple sub-rounds within the i-th training round based on each first sample point sequence by the first candidate point prediction model, optimizing the training effect of the first candidate point prediction model. For each first sample point in any first sample point sequence, the first hidden state of each first sample point is sequentially obtained through the first candidate point prediction model, and the first candidate point prediction model is trained according to the second hidden state sequence and the first hidden state cached in the model, reducing the possibility of repeated calculation of multiple historical touch points, reducing the consumption degree and consumption cost of algorithm resources, improving the operation efficiency and operation accuracy of the trained target point prediction model, and improving the prediction efficiency and prediction accuracy of touch points in the scenario of touch point prediction based on the target point prediction model, thereby reducing the follow-up delay degree between the user side and the human hand, improving the stability of system operation, and optimizing the user experience.

[0019] It should be understood that the content described in the present disclosure is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The above and / or additional aspects and advantages of the present disclosure will become apparent and easily understood from the following description of the embodiments in conjunction with the drawings, where:

[0021] Figure 1 is a schematic flowchart of the training method of the point prediction model according to an embodiment of the present disclosure;

[0022] Figure 2 is a schematic diagram of the point sequence set according to an embodiment of the present disclosure;

[0023] Figure 3 is a schematic flowchart of the training method of the point prediction model according to another embodiment of the present disclosure;

[0024] Figure 4 is a schematic flowchart of the training method of the point prediction model according to another embodiment of the present disclosure;

[0025] Figure 5 is a schematic diagram of the touch point prediction method according to an embodiment of the present disclosure;

[0026] Figure 6 Structural schematic diagram of a training device for a point prediction model according to an embodiment of the present disclosure;

[0027] Figure 7 Structural schematic diagram of a touch point prediction device according to an embodiment of the present disclosure;

[0028] Figure 8 Block diagram of an electronic device according to an embodiment of the present disclosure. Detailed implementation manners

[0029] Embodiments of the present disclosure will be described in detail below. Examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to explain the present disclosure, and should not be construed as a limitation of the present disclosure.

[0030] In the related art, during the calculation process of each touch point prediction, information of multiple historical touch points can be obtained, and the position of future touch points can be predicted based on the obtained information of multiple historical touch points. There is a possibility of repeated calculation of historical touch points, resulting in a certain degree of waste of algorithm resources and poor efficiency.

[0031] A method for training a point prediction model, a method for predicting touch points, and a device therefor proposed in an embodiment of the present disclosure will be described below with reference to the accompanying drawings.

[0032] Figure 1 Flow schematic diagram of a method for training a point prediction model according to an embodiment of the present disclosure, as Figure 1 shown, the method includes:

[0033] S101, obtaining a first candidate point prediction model to be trained and a first sample point sequence set of the current i-th training round.

[0034] During the use of a touch device, a user will draw a line segment through a touch screen provided by the touch device. To reduce the follow-up delay of the user during the drawing process, relevant information of each touch point that may be included in the line segment to be drawn in the future can be predicted based on each touch point in the user's historical drawn line segment.

[0035] In an embodiment of the present disclosure, the prediction of relevant information of each touch point proposed above can be implemented by a trained point prediction model, where the point prediction model to be trained can be marked as the first candidate point prediction model.

[0036] During the process of model training, which may include multiple training rounds, the training round that the first candidate point prediction model currently needs to perform can be marked as i. Then, the i-th training round is the current training round of the first candidate point prediction model.

[0037] The multiple sample point sequences input into the first candidate point prediction model in the i-th training round for its training are marked as the multiple first sample point sequences of the i-th training round, and then a first sample point sequence set composed of multiple first sample point sequences is obtained.

[0038] As an example, the first sample point sequence set can be as Figure 2 shown. In the Figure 2 scenario shown, point sequence 1 is the first sample point sequence 1, point sequence 2 is the first sample point sequence 2, point sequence 3 is the first sample point sequence 3, and point sequence M is the first sample point sequence M. In Figure 2 , the set composed of point sequence 1, point sequence 2, point sequence 3, ……, point sequence M can be determined as the first sample point sequence set.

[0039] S102. Obtain the first training order of each first sample point sequence in the first sample point sequence set, and input each first sample point sequence into the first candidate point prediction model one by one according to the first training order, so as to obtain the first hidden state of each first sample point in any sample point sequence one by one in order through the first candidate point prediction model.

[0040] In the embodiments of the present disclosure, each first sample point sequence in the first sample point sequence set needs to be input into the first candidate point prediction model for training. In this scenario, each first sample point sequence in the first sample point sequence set can be input into the first candidate point prediction model in batches.

[0041] Among them, the batch order in which each first sample point sequence is input into the first candidate point prediction model in batches can be marked as the first training order between each first sample point sequence.

[0042] Furthermore, input each first sample point sequence into the first candidate point prediction model in batches according to the first training order, so as to realize the model training of multiple sub-rounds of the first candidate point prediction model in the i-th training round.

[0043] Optionally, for the first sample point sequence input into the first candidate point prediction model in any batch, the hidden state of each first sample point in this first sample point sequence can be obtained one by one in order through the first candidate point prediction model, and this hidden state is marked as the first hidden state of each first sample point.

[0044] It can be understood that the hidden state of each first sample point can be calculated one by one according to a preset order. Among them, the hidden state algorithm in the related art can be configured in the first candidate point prediction model to calculate the hidden state of the first candidate point prediction model, so as to obtain the first hidden state of each first sample point.

[0045] S103. Obtain the second hidden state sequence cached in history in the first candidate point prediction model, and train the first candidate point prediction model in the i-th training round according to the first hidden state and the second hidden state sequence, so as to obtain the second candidate point prediction model after the end of the i-th training round.

[0046] In the embodiments of the present disclosure, the first candidate point prediction model can cache the hidden states calculated within a historical time range. Among them, the hidden states already cached in the first candidate point prediction model can be marked as the second hidden states, and the sequence composed of each second hidden state is marked as the second hidden state sequence cached in the first candidate point prediction model.

[0047] Optionally, for the first sample point currently calculating the hidden state, the hidden state sequence composed of the first hidden state calculated for this first sample point and the second hidden state sequence already cached in the model can be transmitted to the downstream model layer of the first candidate point prediction model, so as to obtain the model output result output by the first candidate point prediction model.

[0048] Furthermore, based on this model output result, obtain the output result of the first candidate point prediction model in the model training of the i-th training round, and then implement the model training of the first candidate point prediction model according to this output result, and determine the model obtained after the end of the model training in the i-th training round as the second candidate point prediction model.

[0049] It should be noted that for the current training sub-round in the i-th training round, the second hidden state sequence cached in history in the first candidate point prediction model may include the hidden states calculated in the current training sub-round, or may include the hidden states calculated in the previous training sub-round of the current training sub-round in the i-th training round. In this scenario, the hidden states in the second hidden state sequence cached in history in the model can be sorted based on the time sequence from early to late, and the second hidden state sequence can be updated based on the first-in, first-out principle.

[0050] It can be understood that, for any first hidden state in the current training sub-round, the process of obtaining the corresponding second hidden state sequence can be to delete the hidden state with the earliest calculation time from the second hidden state sequence cached in history from the sequence, and add the previous adjacent first hidden state corresponding to this first hidden state to the sequence, so as to obtain the second hidden state sequence corresponding to this first hidden state.

[0051] S104. Clear the hidden states of the second candidate point prediction model to obtain a third candidate point prediction model.

[0052] In the embodiments of the present disclosure, during the process of training the first candidate point prediction model to obtain the second candidate point prediction model, multiple hidden states will be cached in the model during the training process. In this scenario, when the i-th training round ends, it is necessary to perform an operation of clearing the hidden states of the second candidate point prediction model that ends this round of training, and determine the model obtained after clearing as the third candidate point prediction model.

[0053] After the i-th training round ends and before the (i + 1)-th training round starts, the hidden states cached in the second candidate point prediction model obtained at the end of the i-th training round can be obtained, and the state values of the hidden states in this scenario are cleared, so as to clear the hidden states of the second candidate point prediction model in this scenario, and then obtain the third candidate point prediction model, and perform model training for the (i + 1)-th training round based on the third candidate point prediction model.

[0054] Optionally, the state value of each hidden state can be modified to a preset clearing value, or the state value of each hidden state can be modified to other preset values, which are not specifically limited here.

[0055] Among them, the first candidate point prediction model can be constructed based on a recurrent neural network (RNN) in related technologies. In this scenario, the second candidate point prediction model can be cleared according to the method of clearing the hidden states of the RNN model in related technologies, so as to obtain the cleared third candidate point prediction model.

[0056] Moreover, in the scenario where the first candidate point prediction model is constructed based on other types of models, the second candidate point prediction model can be cleared according to the corresponding clearing method of the model to obtain the third candidate point prediction model.

[0057] S105. Obtain the second sample point sequence set for the (i + 1)-th training round, and continue to perform model training on the third candidate point prediction model according to the second sample point sequence set until the training ends to obtain the trained target point prediction model.

[0058] In the embodiments of the present disclosure, after the model training of the i-th training round is completed, the model training of the (i + 1)-th training round of the next round can be started, and the third candidate point prediction model can be continuously trained based on this round.

[0059] Optionally, a set of sample point sequences used in the (i + 1)-th training round can be obtained from the training sample set, determined as the second sample point sequence set, and the third candidate point prediction model can be trained based on the second sample point sequence set until the model training of the (i + 1)-th training round is completed, and the model after the training is completed is obtained.

[0060] Furthermore, it is identified whether the model obtained after the training of the (i + 1)-th training round meets the end condition of the model training. When it is identified that the model meets the end condition of the model training, it can be determined that the current model training is completed, and the model obtained after the training of the last training round is determined as the trained target point prediction model.

[0061] Correspondingly, when it is identified that the model obtained after the training of the (i + 1)-th training round does not meet the preset end condition of the model training, it can return to continue obtaining the next set of sample point sequences from the training sample set and continue to train the model until the end condition of the model training is met, then the model training can be ended, and the model obtained after the training of the last training round is determined as the trained target point prediction model.

[0062] The training method and device for the point prediction model proposed by the present disclosure obtain the first sample point sequence set of the i-th training round to train the first candidate point prediction model, obtain the second candidate point prediction model, clear the hidden state of the second candidate point prediction model to obtain the third candidate point prediction model, and return to obtain the second sample point sequence set of the (i + 1)-th training round to continue training the third candidate point prediction model until the training ends to obtain the trained target point prediction model. In the present disclosure, each first sample point sequence is input into the first candidate point prediction model one by one in order, realizing the training of multiple sub-rounds within the i-th training round based on each first sample point sequence by the first candidate point prediction model, optimizing the training effect of the first candidate point prediction model. For each first sample point in any first sample point sequence, the first hidden state of each first sample point is obtained one by one in order by the first candidate point prediction model, and the first candidate point prediction model is trained according to the second hidden state sequence and the first hidden state cached in the model, reducing the possibility of repeated calculation of multiple historical touch points, reducing the consumption degree and consumption cost of algorithm resources, improving the operation efficiency and operation accuracy of the trained target point prediction model, improving the prediction efficiency and prediction accuracy of touch points in the scenario of predicting touch points based on the target point prediction model, thereby reducing the follow-up latency between the user side and the human hand, improving the stability of system operation, and optimizing the user experience.

[0063] The touch point prediction method and device proposed by the present disclosure obtain the target predicted touch point sequence capable of caching the hidden state according to the target point prediction model obtained by the training method proposed in the above embodiment, without the need to repeatedly calculate the hidden state of historical touch points during the touch point prediction of each round, reducing the consumption degree of computing resources. Only the hidden state of one touch point is calculated in each prediction round, and the target predicted touch point sequence is obtained according to the new hidden state and the hidden state sequence cached historically, improving the prediction efficiency and prediction accuracy of the target predicted touch point sequence, thereby reducing the follow-up latency between the user side and the human hand, improving the stability of system operation, and optimizing the user experience.

[0064] In the above embodiment, regarding the acquisition of the target point prediction model, it can also be combined with Figure 3 Understand that Figure 3 is a schematic flowchart of the training method of the point prediction model according to another embodiment of the present disclosure. As Figure 3 shown, the method includes:

[0065] S301, obtain the first candidate point prediction model to be trained and the first sample point sequence set of the current i-th training round.

[0066] Optionally, obtain the second sample line segment set used in the i-th training round and the sample sampling timestamp set of each second sample line segment.

[0067] In the embodiments of the present disclosure, the line segments sampled for samples at the i-th training round can be marked as second sample line segments, and the set composed of each second sample line segment is marked as the second sample line segment set.

[0068] Optionally, when performing point sampling on each second sample line segment, the timestamp used for point sampling can be marked as the sample sampling timestamp, so as to obtain the sample sampling timestamp set composed of each sample sampling timestamp.

[0069] Optionally, based on the sample sampling timestamp set, perform point sampling on each second sample line segment to obtain the sample candidate line segment point information set of each second sample line segment. Among them, for any sample candidate line segment point information, the sample candidate line segment point information at least includes a point identifier and corresponding metadata.

[0070] In the embodiments of the present disclosure, point sampling can be performed on each second sample line segment based on each sample sampling timestamp in the sample sampling timestamp set according to the point sampling method in the related art. Among them, for any sample sampling timestamp, the touch points that each second sample line segment appears at this sample sampling timestamp can be marked as the touch points collected based on this sample sampling timestamp.

[0071] Further, for any second sample line segment, the touch points obtained by performing point sampling on this second sample line segment according to each sample sampling timestamp can be marked as the sample candidate line segment points on this second sample line segment, and the set composed of the point information of each candidate line segment point is marked as the sample candidate line segment point information set of this second sample line segment.

[0072] It should be noted that for any sample candidate line segment point information, it can include the identification information of this point, marked as the point identifier of this sample candidate line segment point, or can include the descriptive metadata of this point, marked as the corresponding metadata of this sample candidate line segment point, and can also include other attribute information of this point, which is not specifically limited here.

[0073] Among them, the metadata can include the position information of this point, the pressure information when the user touches this point, and the timestamp information generated by this point, etc.

[0074] Optionally, for any sample sampling timestamp, obtain the sample target line segment point information of each second sample line segment at the sample sampling timestamp from the sample candidate line segment point information set of each second sample line segment, and obtain the first sample point sequence corresponding to the sample sampling timestamp according to each sample target line segment point information.

[0075] In the embodiments of the present disclosure, the sample candidate line segment point information includes the timestamp information of the corresponding sample candidate line segment point. In this scenario, for any sample sampling timestamp, the sample candidate line segment point information with the same timestamp information as the sample sampling timestamp information can be obtained from the timestamp information in the sample candidate line segment point information sets of each second sample line segment, and is respectively marked as the sample target line segment point information of each second sample line segment.

[0076] Further, the sequence composed of each sample target line segment point information is marked as the first sample point sequence corresponding to the sample sampling timestamp.

[0077] Optionally, according to the first sample point sequences of each sample sampling timestamp set, the first sample point sequence set of the i-th training round is obtained.

[0078] In the embodiments of the present disclosure, based on the set construction method in the related art, the set construction can be performed on the first sample point sequences of each sample sampling timestamp, so as to obtain the first sample point sequence set used for model training in the i-th training round.

[0079] S302. According to the first training sorting, obtain the k-th first sample point sequence used in the j-th training sub-round in the i-th training round from the first sample point sequence set.

[0080] In the embodiments of the present disclosure, there may be multiple training sub-rounds in the i-th training round, where the training sub-round in which the first candidate point prediction model is currently performing model training can be marked as the j-th training sub-round in the i-th training round.

[0081] Among them, the number of training sub-rounds included in the i-th training round is the same as the number of first sample point sequences included in the first sample point sequence set.

[0082] In this scenario, according to the first training sorting, the first sample point sequence with the input sorting the same as the sorting of the j-th training sub-round among all training sub-rounds can be obtained from each first sample point sequence, and it is determined as the k-th first sample point sequence used in the j-th training sub-round.

[0083] As an example, as Figure 2 shown, the first training sorting among each first sample point sequence is point sequence 1, point sequence 2, point sequence 3,..., point sequence M. Assuming that j takes the value of 2, then in the Figure 2 shown scenario, the point sequence with the input sorting of 2 can be obtained from each first sample point sequence, that is, Figure 2 the point sequence 2 shown, as the k-th first sample point sequence used in the 2nd training sub-round.

[0084] S303. Obtain the hidden layer in the first candidate point prediction model, where the hidden layer is provided with recurrent neural network units.

[0085] As an example, as Figure 4 shown, in the Figure 4 point prediction model shown, it includes an Figure 4 input layer, a hidden layer, and an output layer shown, where the hidden layer is provided with Figure 4 a recurrent neural network cell (RNN cell) shown. Through the Figure 4 RNN cell shown, the hidden state calculation can be performed on each first sample point in the k-th first sample point sequence input to the first candidate prediction model.

[0086] S304. Through the recurrent neural network units in the hidden layer, based on the calculation order of each first sample point in the k-th first sample point sequence, obtain the first hidden state of each first sample point one by one.

[0087] As an example, as Figure 4 shown, in the Figure 4 point prediction model shown, it includes an Figure 4 input layer, a hidden layer, and an output layer shown, where through the Figure 4 RNN cell shown, the hidden state calculation can be performed on each first sample point in the k-th first sample point sequence input to the first candidate prediction model, and the first hidden state of each first sample point.

[0088] In the Figure 4 scenario shown, the first sample point corresponding to the input 5 is the sample point currently input to the first candidate point prediction model. Through the Figure 4 RNN cell shown, the hidden state calculation can be performed on this sample point, and the Figure 4 hidden state 5 shown is output. This hidden state 5 is the first hidden state of the first sample point corresponding to the input 5.

[0089] S305. For the j-th training sub-round, obtain the k-th first sample point sequence of the j-th training sub-round. According to the first hidden state and the second hidden state sequence of each first sample point in the k-th first sample point sequence, obtain the first target output point sequence of the k-th first sample point sequence output by the first candidate point prediction model, so as to obtain the first training loss of the first candidate point prediction model in the j-th training sub-round.

[0090] Optionally, obtain the output sequence length of the first candidate point prediction model.

[0091] In the embodiments of the present disclosure, the number of elements in the data sequence output by the first candidate point prediction model can be marked as the output sequence length of the first candidate point prediction model.

[0092] Optionally, according to the first hidden state and the second hidden state sequence, obtain the sample point sequence feature information of the k-th first sample point sequence, and according to the output sequence length and the sample point sequence feature information, obtain the first target output point sequence corresponding to the k-th first sample point sequence.

[0093] In the embodiments of the present disclosure, for the k-th first sample point sequence used for model training in the j-th training sub-round of the i-th training round, based on the first hidden states of the first sample points in the k-th first sample point sequence and the second hidden states in the second hidden state sequence that has been cached in the model, the feature relationships between the first sample points in the k-th first sample point sequence and the feature information of each first sample point in the k-th first sample point sequence can be extracted. In this scenario, the extracted feature relationships between the first sample points and their respective feature information can be marked as the sample point sequence feature information of the k-th first sample point sequence.

[0094] Optionally, there is a sequence length limit for the output point sequence of the first candidate point prediction model, and this limited sequence length can be determined as the output sequence length of the first candidate point prediction model. Among them, the output sequence length of the first candidate point prediction model can be set based on model performance or determined by other algorithms, and specific limitations are not made here.

[0095] In this scenario, the corresponding first target output point sequence can be obtained according to the extracted sample point sequence feature information and the output sequence length of the first candidate point prediction model. Among them, for any first sample point in the k-th first sample point sequence, the number of output point timestamps corresponding to the first sample point is obtained according to the output sequence length.

[0096] In the embodiments of the present disclosure, for the k-th first sample point sequence, the prediction of the touch point sequence can be performed on each first sample point included in the sequence, and the first target output point sequence corresponding to the k-th first sample point sequence can be formed according to the point sequences predicted for each first sample point.

[0097] Among them, the value of the number of sequence elements output by the model carried in the output sequence length can be obtained, and this value can be determined as the number of prediction timestamps used when predicting the touch point sequence for any first sample point in the k-th first sample point sequence, and this number can be marked as the output point timestamp number of this first sample point.

[0098] Optionally, obtain the first point timestamp of the first sample point, the second point timestamp at the next moment of the first point timestamp, and the third point timestamp corresponding to the number of output point timestamps after the second point timestamp.

[0099] In the embodiments of the present disclosure, for any first sample point, the timestamp included in the point information of the sample point can be marked as the first point timestamp of the first sample point. It can be understood that the first point timestamp of the first sample point is the sample sampling timestamp of the first sample point.

[0100] In this scenario, the timestamp corresponding to the next moment of the first point timestamp is marked as the second timestamp. Among them, the second timestamp at the next moment can be understood as the timestamp corresponding to the first output point in the predicted output of the touch point sequence that may be generated in the future based on the first point timestamp, that is, the next prediction moment timestamp of the first point timestamp.

[0101] And, obtain the timestamp corresponding to the number of output point timestamps after the second point timestamp, and mark it as the third timestamp. As an example, assume that the first point timestamp is t, and the number of output point timestamps is p. Then the second point timestamp is the timestamp corresponding to t + 1, and the third point timestamp is the timestamp corresponding to t + 1 + p.

[0102] Optionally, according to the sample point sequence feature information, obtain the first sample segment to which the first sample point belongs, and extract each candidate output point of the first sample point from the first sample segment according to the second point timestamp and the third point timestamp, so as to obtain the candidate output point sequence corresponding to the first sample point.

[0103] In the embodiments of the present disclosure, the sample point sequence feature information includes the feature relationship between each first sample point in the kth first sample point sequence, and the feature information of each first sample point. In this scenario, the point feature information of the first sample point can be obtained through the sample point sequence feature information, and then the first sample segment to which the first sample point belongs can be determined based on the obtained point feature information.

[0104] Further, on the first sample segment, obtain each candidate output point of the first sample point, so as to obtain the candidate output point sequence corresponding to the first sample point composed of each candidate output point.

[0105] As an example, take Figure 2 the first sample point corresponding to "segment 2, timestamp 2" in the point sequence 2 shown as an example. From Figure 2 it can be seen that the first point timestamp of the first sample point is 2. Assume that the second point timestamp of the first sample point is 3 and the third point timestamp is 5. Then in Figure 2In the shown scenario, the line segment 2 to which the first sample point belongs can be obtained as the first sample line segment of the first sample point.

[0106] Further, according to the interval composed of the second timestamp 3 and the third timestamp 5, a candidate output point of the first sample point on the first sample line segment is obtained. As Figure 2 shown, the candidate output points of the first sample point on the first sample line segment may include Figure 2 the points corresponding to "line segment 2, timestamp 3", "line segment 2, timestamp 4", and "line segment 2, timestamp 5" shown. Then, in this example, these three points are the candidate output points of the first sample point corresponding to "line segment 2, timestamp 2".

[0107] Then, the sequence composed of the candidate output points corresponding to "line segment 2, timestamp 3", the candidate output points corresponding to "line segment 2, timestamp 4", and the candidate output points corresponding to "line segment 2, timestamp 5" can be marked as the candidate output point sequence of the first sample point corresponding to "line segment 2, timestamp 2".

[0108] Optionally, according to the candidate output point sequences of the first sample points in the k-th first sample point sequence, a first target output point sequence of the k-th first sample point sequence is formed.

[0109] Among them, based on the sequence construction method in the related art, the candidate output point sequences of the first sample points can be combined, and the combined sequence is used as the first target output point sequence of the k-th first sample point sequence.

[0110] Optionally, a label point sequence of the k-th first sample point sequence is obtained, and the first target output point sequence and the label point sequence are used to obtain the first training loss of the first candidate point prediction model in the j-th training sub-round.

[0111] In the embodiments of the present disclosure, the sequence carried in the label information of the k-th first sample point sequence can be marked as the label point sequence. In this scenario, based on the loss value acquisition algorithm in the related art, the first target output point sequence and the k-th first sample point sequence can be processed algorithmically, so that the loss value of the first target output point sequence based on the k-th first sample point sequence is marked as the first training loss of the first candidate point prediction model in the j-th training sub-round.

[0112] S306. Adjust the model parameters of the first candidate point prediction model according to the first training loss, and return the first candidate point prediction model after adjusting the model parameters using the (k + 1)-th first sample point sequence obtained in the (j + 1)-th training sub-round from the first sample point sequence set, and continue to train the first candidate point prediction model until the training of the i-th training round ends, and obtain the second candidate point prediction model after the training of the i-th training round ends.

[0113] In the embodiments of the present disclosure, the model parameters of the first candidate point prediction model can be adjusted according to the first training loss to achieve iterative optimization of the first candidate point prediction model, so as to obtain the first candidate point prediction model with adjusted parameters.

[0114] Optionally, it can be identified whether the adjusted first candidate point prediction model meets the model training end condition of the i-th training round. When it is identified that the adjusted first candidate point prediction model obtained in the j-th training sub-round does not meet the model training end condition of the i-th training round, the next (k + 1)-th first sample point sequence can be obtained from the first sample point sequence set, and the first candidate point prediction model with adjusted parameters can be continuously trained in the next (j + 1)-th training sub-round based on the (k + 1)-th first sample point sequence until the model training of the i-th training round ends.

[0115] Further, the model obtained after the last training sub-round in the i-th training round is marked as the second candidate point prediction model obtained after the i-th training round ends.

[0116] S307. Obtain the third hidden state sequence in the second candidate point prediction model, and adjust the third hidden state sequence to a cleared state sequence to obtain a third candidate point prediction model, where the cleared state sequence includes either a zero vector sequence or a preset initial value vector sequence.

[0117] In the embodiments of the present disclosure, in order to optimize the model training effect of the (i + 1)-th training round, an operation of clearing the hidden state of the second candidate point prediction model obtained in the i-th training round can be performed.

[0118] Among them, the sequence composed of the hidden states cached in the second candidate point prediction model can be marked as the third hidden state sequence. In this scenario, each cached data in the third hidden state sequence can be adjusted to a preset zero vector or adjusted to a preset initial value vector.

[0119] Further, the zero vector sequence composed of the adjusted zero vectors or the initial value vector sequence composed of the adjusted initial value vectors is determined as the cleared state sequence obtained after the clearing adjustment of the third hidden state sequence.

[0120] S308. Obtain the second sample point sequence set of the (i + 1)-th training round, and continue to perform model training on the third candidate point prediction model according to the second sample point sequence set until the training ends, to obtain a trained target point prediction model.

[0121] Optionally, obtain the second training sorting of each second sample point sequence in the second sample point sequence set, and input the second sample point sequence set into the third candidate point prediction model one by one according to the second training sorting to perform model training for the (i + 1)-th training round of the third candidate point prediction model until the (i + 1)-th training round ends, and obtain the fourth candidate point prediction model.

[0122] In the embodiments of the present disclosure, for the specific information on obtaining the fourth candidate point prediction model, refer to the relevant detailed content on obtaining the second candidate point prediction model proposed in the above embodiments, which will not be elaborated here.

[0123] Optionally, in response to the fourth candidate point prediction model satisfying the preset overall model training end condition, determine the fourth candidate point prediction model as the trained target point prediction model.

[0124] It can be understood that if the fourth candidate point prediction model satisfies the preset overall model training end condition, all model training can be ended, and the fourth candidate point prediction model can be used as the target point prediction model.

[0125] Optionally, in response to the fourth candidate point prediction model not satisfying the preset model training end condition, return to obtain the third sample point sequence set for the (i + 2)-th training round, and continue to train the fourth candidate point prediction model until the overall training ends, and obtain the trained target point prediction model.

[0126] It can be understood that if the fourth candidate point prediction model does not satisfy the overall model training end condition, it is necessary to clear the hidden state of the fourth candidate point prediction model, then return to obtain the third sample point sequence set used for the model training of the (i + 2)-th training round, and continue to train the cleared fourth candidate point prediction model according to the third sample point sequence set until the overall model training end condition is satisfied, and then the model training can be ended to obtain the trained target point prediction model.

[0127] The training method of the point prediction model proposed by the present disclosure inputs each first sample point sequence into the first candidate point prediction model one by one in sequence, realizing the training of multiple sub-rounds within the i-th training round of the first candidate point prediction model based on each first sample point sequence, optimizing the training effect of the first candidate point prediction model. For each first sample point in any first sample point sequence, the first candidate point prediction model sequentially obtains the first hidden state of each first sample point one by one, and trains the first candidate point prediction model according to the second hidden state sequence and the first hidden state cached in the model, reducing the possibility of repeated calculation of multiple historical touch points, reducing the consumption degree and cost of algorithm resources, improving the operation efficiency and operation accuracy of the trained target point prediction model, improving the prediction efficiency and prediction accuracy of touch points in the scenario of touch point prediction based on the target point prediction model, thereby reducing the follow-up delay degree between the user side and the human hand, improving the stability of system operation, and optimizing the user experience.

[0128] The present disclosure also proposes a touch point prediction method, which can be combined with Figure 5 for understanding. Figure 5 As shown in the flowchart of the touch point prediction method according to an embodiment of the present disclosure, as Figure 5 shown, the method includes:

[0129] S501, obtain a trained target point prediction model.

[0130] Among them, the target point prediction model is obtained based on Figures 1 to 4 the training method of the point prediction model proposed in the embodiment.

[0131] S502, obtain the candidate touch points currently received on the user side, input the candidate touch point information of the candidate touch points into the target point prediction model, and obtain the candidate hidden state of the candidate touch points through the target hidden layer in the target point prediction model.

[0132] In the embodiment of the present disclosure, the touch points currently generated on the user side can be marked as candidate touch points. Among them, metadata such as identification information, coordinate information, pressure information, and timestamp information of the candidate touch points can be obtained, so as to obtain the candidate touch point information of the candidate touch points.

[0133] Further, input the candidate touch point information into the target point prediction model, and calculate the hidden state of the candidate touch points through the target hidden layer configured in the target point prediction model, and determine the calculated result as the candidate hidden state of the candidate touch points.

[0134] S503, obtain the hidden state cache sequence in the target point prediction model, where the hidden state cache sequence includes the candidate hidden state.

[0135] In an embodiment of the present disclosure, the target point prediction model can cache candidate hidden states, where the sequence composed of all cached hidden states in the target point prediction model after caching can be marked as the hidden state cache sequence in the target point prediction model.

[0136] It should be noted that before the target point prediction model caches the hidden state, there may be no blank cache bits left in the cache area of the hidden state. In this scenario, based on the first-in, first-out principle, the earliest cached hidden state in the cache area can be cleared to obtain a new blank cache bit, and the candidate hidden state can be cached into the cache area of the hidden state based on this new blank cache bit, so as to obtain the hidden state cache sequence of the target point prediction model.

[0137] S504, in response to the hidden state cache sequence satisfying a preset model prediction condition, input the hidden state cache sequence into the downstream model layer of the target hidden layer in the target point prediction model to predict the touch point, and output the target predicted touch point sequence corresponding to the candidate touch point, where the target predicted touch point sequence is used to represent the touch sequence received by the user side based on the candidate touch point within the future time range.

[0138] In an embodiment of the present disclosure, the target point prediction model sets a corresponding model prediction condition for the hidden state cache sequence. It can be understood that when the hidden state cache sequence satisfies this condition, the hidden state cache sequence can be transmitted to the downstream model layer of the target hidden layer to realize the prediction of the touch point sequence that may be generated by the user side within the future time range.

[0139] Optionally, in response to the state cache quantity of the hidden state cache sequence being equal to the preset cache quantity upper limit value, it is determined that the hidden state cache sequence is recognized as satisfying the model prediction condition.

[0140] In an embodiment of the present disclosure, the upper limit of the number of hidden states that can be cached in the cache area of the hidden state of the target point prediction model can be marked as the cache quantity upper limit value.

[0141] Among them, the number of elements in the hidden state cache sequence can be compared with the cache quantity upper limit value. When the number of elements is the same as the cache quantity upper limit value, it can be determined that the hidden state cache sequence satisfies the model prediction condition preset by the target point prediction model.

[0142] Optionally, in response to the state cache quantity of the hidden state cache sequence being less than the preset cache quantity threshold, continue to cache the hidden state until the new hidden state cache sequence satisfies the model prediction condition.

[0143] In the embodiments of the present disclosure, a corresponding cache quantity threshold is set for the hidden state cache sequence. It can be understood that when the number of elements in the hidden state cache queue is greater than or equal to the threshold, the downstream model layer of the target hidden layer can accurately capture the respective feature information of the touch point data input to the model and the feature relationships between the touch point data from the hidden state cache sequence, and the captured relevant feature information can support accurate touch point prediction tasks.

[0144] In this scenario, the number of elements in the hidden state cache sequence can be compared with the cache quantity threshold. When the number of elements in the hidden state cache sequence is less than the cache quantity threshold, it can be determined that accurate prediction of the touch point sequence cannot be achieved based on the feature information carried in the current hidden state cache sequence.

[0145] In this scenario, the newly calculated hidden state of the model can continue to be cached to obtain a new hidden state cache sequence until the new hidden state cache sequence meets the model prediction conditions, and then the touch point sequence within the future time range can be predicted based on the new hidden state cache sequence.

[0146] Optionally, in response to the state cache quantity of the hidden state cache sequence being greater than or equal to the cache quantity threshold and less than the cache quantity upper limit value, the hidden state cache sequence is filled with blank positions, and the filled hidden state cache sequence is input into the downstream model layer of the target hidden layer in the target point prediction model to output the target prediction touch point sequence corresponding to the candidate touch points.

[0147] In the scenario where the number of elements in the hidden state cache sequence is less than the cache quantity upper limit value and greater than or equal to the cache quantity threshold, it can be determined that accurate prediction of the touch point sequence can be achieved based on the feature information carried in the current hidden state cache sequence.

[0148] Optionally, if the downstream model layer of the target hidden layer has format restrictions on the input data, then in this scenario, the current hidden state cache sequence can be filled with blank positions. Among them, a preset initial value can be obtained and filled into the blank positions, or 0 can be filled into the blank positions to obtain the hidden state cache sequence after filling the blank positions.

[0149] Furthermore, the filled hidden state cache sequence is input into the downstream model layer to achieve the prediction of the touch point sequence that may occur within the future time range, and the output point sequence obtained by the target point prediction model based on the candidate touch points is marked as the target prediction touch point sequence.

[0150] The touch point prediction method proposed in the present disclosure obtains a target predicted touch point sequence capable of hidden state caching according to the target point prediction model obtained by the training method proposed in the above embodiments. There is no need to repeatedly calculate the hidden state of historical touch points during touch point prediction in each round, reducing the consumption of computing resources. Only the hidden state of one touch point is calculated in each prediction round, and the target predicted touch point sequence is obtained based on the new hidden state and the historically cached hidden state sequence, improving the prediction efficiency and accuracy of the target predicted touch point sequence, thereby reducing the follow-up latency between the user side and the human hand, improving the stability of system operation, and optimizing the user experience.

[0151] Corresponding to the training methods of the point prediction models proposed in the above several embodiments, an embodiment of the present disclosure also proposes a training device for a point prediction model. Since the training device for the point prediction model proposed in the embodiments of the present disclosure corresponds to the training methods of the point prediction models proposed in the above several embodiments, the implementation manners of the above training methods of the point prediction models are also applicable to the training device for the point prediction model proposed in the embodiments of the present disclosure, and will not be described in detail in the following embodiments.

[0152] Figure 6 It is a structural schematic diagram of a training device for a point prediction model according to an embodiment of the present disclosure. As Figure 6 shown, the training device 600 for the point prediction model includes a first acquisition module 61, a first training module 62, a second training module 63, a clearing module 64, and a third training module 65, where:

[0153] The first acquisition module 61 is configured to acquire a first candidate point prediction model to be trained and a first sample point sequence set of the current i-th training round;

[0154] The first training module 62 is configured to obtain a first training order of each first sample point sequence in the first sample point sequence set, and input each first sample point sequence into the first candidate point prediction model one by one according to the first training order, so as to obtain a first hidden state of each first sample point in any sample point sequence one by one through the first candidate point prediction model;

[0155] The second training module 63 is configured to acquire a second hidden state sequence cached historically in the first candidate point prediction model, so as to perform model training on the first candidate point prediction model in the i-th training round according to the first hidden state and the second hidden state sequence, and obtain a second candidate point prediction model after the end of the i-th training round;

[0156] The clearing module 64 is configured to clear the hidden state of the second candidate point prediction model to obtain a third candidate point prediction model;

[0157] The third training module 65 is used to obtain the second set of sample point sequences in the (i + 1)-th training round, and continue to train the third candidate point prediction model according to the second set of sample point sequences until the training ends, obtaining a trained target point prediction model.

[0158] In the embodiment of the present disclosure, the first training module 62 is further used to: obtain the k-th first sample point sequence used in the j-th training sub-round in the i-th training round according to the first training sorting from the first set of sample point sequences; obtain the hidden layer in the first candidate point prediction model, where a recurrent neural network unit is set in the hidden layer; through the recurrent neural network unit in the hidden layer, based on the calculation sorting of each first sample point in the k-th first sample point sequence, obtain the first hidden state of each first sample point one by one.

[0159] In the embodiment of the present disclosure, the second training module 63 is further used to: for the j-th training sub-round, obtain the k-th first sample point sequence of the j-th training sub-round, and obtain the first target output point sequence of the k-th first sample point sequence output by the first candidate point prediction model according to the first hidden state and the second hidden state sequence of each first sample point in the k-th first sample point sequence, so as to obtain the first training loss of the first candidate point prediction model in the j-th training sub-round; adjust the model parameters of the first candidate point prediction model according to the first training loss, and return to obtain the (k + 1)-th first sample point sequence used in the (j + 1)-th training sub-round from the first set of sample point sequences to continue training the first candidate point prediction model with adjusted model parameters until the i-th training round ends, obtaining the second candidate point prediction model after the i-th training round ends.

[0160] In the embodiment of the present disclosure, the second training module 63 is further used to: obtain the output sequence length of the first candidate point prediction model; obtain the sample point sequence feature information of the k-th first sample point sequence according to the first hidden state and the second hidden state sequence, and obtain the first target output point sequence corresponding to the k-th first sample point sequence according to the output sequence length and the sample point sequence feature information; obtain the label point sequence of the k-th first sample point sequence, and obtain the first training loss of the first candidate point prediction model in the j-th training sub-round according to the first target output point sequence and the label point sequence.

[0161] In an embodiment of the present disclosure, the second training module 63 is further configured to: for any first sample point in the k-th first sample point sequence, obtain the number of output point timestamps corresponding to the first sample point according to the output sequence length; obtain the first point timestamp of the first sample point, the second point timestamp at the next moment of the first point timestamp, and the third point timestamp corresponding to the number of output point timestamps after the second point timestamp; obtain the first sample segment to which the first sample point belongs according to the sample point sequence feature information, and extract each candidate output point of the first sample point from the first sample segment according to the second point timestamp and the third point timestamp, so as to obtain a candidate output point sequence of the first sample point; and form a first target output point sequence of the k-th first sample point sequence according to the candidate output point sequences of the first sample points in the k-th first sample point sequence.

[0162] In an embodiment of the present disclosure, the clearing module 64 is further configured to: obtain a third hidden state sequence in the second candidate point prediction model, and adjust the third hidden state sequence to a cleared state sequence to obtain a third candidate point prediction model, where the cleared state sequence includes one of a zero vector sequence and a preset initial value vector sequence.

[0163] In an embodiment of the present disclosure, the third training module 65 is further configured to: obtain a second training ranking of each second sample point sequence in the second sample point sequence set; input the second sample point sequence set into the third candidate point prediction model one by one in order according to the second training ranking to perform model training for the (i + 1)-th training round on the third candidate point prediction model until the (i + 1)-th training round ends, to obtain a fourth candidate point prediction model; in response to the fourth candidate point prediction model satisfying a preset model training end condition, determine the fourth candidate point prediction model as the trained target point prediction model; in response to the fourth candidate point prediction model not satisfying a preset overall model training end condition, return to obtain a third sample point sequence set for the (i + 2)-th training round, and continue to train the fourth candidate point prediction model until the overall training ends, to obtain the trained target point prediction model.

[0164] In an embodiment of the present disclosure, the first acquisition module 61 is further configured to: acquire a second sample line segment set used in the i-th training round and a sample sampling timestamp set of each second sample line segment; perform point sampling on each second sample line segment based on the sample sampling timestamp set to obtain a sample candidate line segment point information set of each second sample line segment, where, for any sample candidate line segment point information, the sample candidate line segment point information includes at least a point identifier and corresponding metadata; for any sample sampling timestamp, obtain sample target line segment point information of each second sample line segment at the sample sampling timestamp from the sample candidate line segment point information set of each second sample line segment, and obtain a first sample point sequence corresponding to the sample sampling timestamp according to the sample target line segment point information; obtain a first sample point sequence set of the i-th training round according to the first sample point sequences of the sample sampling timestamp sets respectively.

[0165] The training device of the point prediction model proposed by the present disclosure acquires a first sample point sequence set of the i-th training round to train a first candidate point prediction model, obtains a second candidate point prediction model, clears the hidden state of the second candidate point prediction model to obtain a third candidate point prediction model, and returns to acquire a second sample point sequence set of the (i + 1)-th training round to continue training the third candidate point prediction model until the training ends to obtain a trained target point prediction model. In the present disclosure, each first sample point sequence is input into the first candidate point prediction model one by one in order, realizing multiple sub-rounds of training of the first candidate point prediction model within the i-th training round based on each first sample point sequence, optimizing the training effect of the first candidate point prediction model. For each first sample point in any first sample point sequence, the first hidden state of each first sample point is obtained one by one in order through the first candidate point prediction model, and the first candidate point prediction model is trained according to the second hidden state sequence and the first hidden state cached in the model, reducing the possibility of repeated calculation of multiple historical touch points, reducing the consumption degree and cost of algorithm resources, improving the operation efficiency and operation accuracy of the trained target point prediction model, improving the prediction efficiency and prediction accuracy of touch points in the scenario of predicting touch points based on the target point prediction model, thereby reducing the follow-up delay degree between the user side and the human hand, improving the stability of system operation, and optimizing the user experience.

[0166] Corresponding to the touch point prediction methods proposed in the above several embodiments, an embodiment of the present disclosure also proposes a touch point prediction device. Since the touch point prediction device proposed in the embodiment of the present disclosure corresponds to the touch point prediction methods proposed in the above several embodiments, the implementation manners of the above touch point prediction methods are also applicable to the touch point prediction device proposed in the embodiment of the present disclosure and will not be described in detail in the following embodiments.

[0167] Figure 7The structural schematic diagram of a touch point prediction device according to an embodiment of the present disclosure is as follows. Figure 7 As shown, the touch point prediction device 700 includes a second acquisition module 71, a third acquisition module 72, a fourth acquisition module 73, and a prediction module 74, where:

[0168] The second acquisition module 71 is configured to acquire a trained target point prediction model, where the target point prediction model is obtained based on the point prediction model training device of the above-mentioned claim 12;

[0169] The third acquisition module 72 is configured to acquire candidate touch points currently received on the user side, and input candidate touch point information of the candidate touch points into the target point prediction model to obtain candidate hidden states of the candidate touch points through a target hidden layer in the target point prediction model;

[0170] The fourth acquisition module 73 is configured to acquire a hidden state cache sequence in the target point prediction model, where the hidden state cache sequence includes candidate hidden states;

[0171] The prediction module 74 is configured to, in response to the hidden state cache sequence satisfying a preset model prediction condition, input the hidden state cache sequence into a downstream model layer of the target point prediction model located in the target hidden layer to perform prediction of touch points, and output a target prediction touch point sequence corresponding to the candidate touch points, where the target prediction touch point sequence is used to represent a received touch sequence on the user side based on the candidate touch points within a future time range.

[0172] In an embodiment of the present disclosure, the prediction module 74 is further configured to: determine that it is recognized that the hidden state cache sequence satisfies the model prediction condition in response to the state cache quantity of the hidden state cache sequence being equal to a preset cache quantity upper limit value; and continue to perform hidden state caching until a new hidden state cache sequence satisfies the model prediction condition in response to the state cache quantity of the hidden state cache sequence being less than a preset cache quantity threshold.

[0173] In an embodiment of the present disclosure, the prediction module 74 is further configured to: perform blank position filling on the hidden state cache sequence in response to the state cache quantity of the hidden state cache sequence being greater than or equal to the cache quantity threshold and less than the cache quantity upper limit value, and input the filled hidden state cache sequence into a downstream model layer of the target point prediction model located in the target hidden layer to output a target prediction touch point sequence corresponding to the candidate touch points.

[0174] The touch point prediction device proposed in the present disclosure obtains a target predicted touch point sequence capable of hidden state caching according to the target point prediction model obtained by the training method proposed in the above embodiment, without the need to repeatedly calculate the hidden state of historical touch points during the touch point prediction of each round, reducing the consumption degree of computing resources. Only the hidden state of one touch point is calculated in each prediction round, and according to the new hidden state and the historically cached hidden state sequence, the target predicted touch point sequence is obtained, improving the prediction efficiency and prediction accuracy of the target predicted touch point sequence, thereby reducing the follow-up latency between the user end and the human hand, improving the stability of system operation, and optimizing the user experience.

[0175] To achieve the above embodiment, the present disclosure also provides a chip, including one or more interface circuits and one or more processors; the interface circuit is used to receive a signal and send the signal to the processor, and the signal includes computer instructions stored in a memory. When the processor executes the computer instructions, the chip executes the steps of the training method of the point prediction model and / or the touch point prediction method provided in the above embodiment.

[0176] To achieve the above embodiment, the present disclosure also provides an electronic device, a computer-readable storage medium, and a computer program product.

[0177] Figure 8 It is a block diagram of an electronic device 800 according to an embodiment of the present disclosure. As Figure 8 shown, the electronic device 800 includes a memory 801, a processor 802, and a computer program stored on the memory 801 and executable on the processor 802. When the processor 802 executes the program instructions, the training method of the point prediction model and / or the touch point prediction method provided in the above embodiment is implemented.

[0178] The various embodiments of the systems and technologies described above in this article can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: implemented in one or more computer programs, the one or more computer programs can be executed and / or interpreted on a programmable system including at least one programmable processor, the programmable processor can be a dedicated or general programmable processor, and can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0179] The various embodiments of the systems and techniques described above in this document can be implemented in digital electronic circuitry, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be a special-purpose or general-purpose programmable processor that receives data and instructions from a storage system, at least one input device, and at least one output device, and transmits the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0180] The program code for implementing the methods of itself can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the program codes are executed by the processor or controller, the functions / operations specified in the flowchart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, executed partially on the machine as an independent software package and partially on a remote machine, or executed entirely on a remote machine or server.

[0181] In the context of this disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0182] To provide for interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can also be used to provide for interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0183] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local area network (LAN), wide area network (WAN), the Internet, and blockchain network.

[0184] A computer system can include a client and a server. The client and the server are generally remote from each other and typically interact through a communication network. The client-server relationship is created by computer programs that run on the respective computers and have a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system, solving the defects of difficult management and weak business scalability existing in traditional physical hosts and VPS services (“Virtual Private Server”, or simply “VPS”). The server can also be a server of a distributed system, or a server combined with blockchain.

[0185] In the description of this specification, the descriptions with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples", etc. mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, without contradiction, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0186] In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present disclosure, "a plurality of" means at least two, such as two, three, etc., unless otherwise specifically defined.

[0187] Any process or method description shown in the flowchart or described in other ways herein may be understood to represent a module, segment, or portion of code including one or more executable instructions for implementing a customized logical function or process, and the scope of the preferred embodiments of the present disclosure includes additional implementations, where the functions may be executed in a substantially simultaneous manner or in a reverse order according to the involved functions, rather than in the order shown or discussed, which should be understood by those skilled in the art to which the embodiments of the present disclosure pertain.

[0188] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a definite sequence list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other systems that can fetch and execute instructions from the instruction execution system, apparatus, or device), or used in combination with these instruction execution systems, apparatus, or devices. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. More specific examples (non-exhaustive list) of computer-readable media include the following: electrical connection parts with one or more wirings (electronic devices), portable computer disk cartridges (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), fiber optic devices, and portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium can even be paper or other suitable media on which a program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpretation, or other appropriate processing as necessary, and then stored in a computer memory.

[0189] It should be understood that various parts of the present disclosure can be implemented by hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or a combination of the following techniques well known in the art can be used: discrete logic circuits having logic gate circuits for implementing logical functions on data signals, application specific integrated circuits having appropriate combinational logic gate circuits, programmable gate arrays (PGA), field programmable gate arrays (FPGA), etc.

[0190] Those of ordinary skill in the art of this technology can understand that all or part of the steps carried by the methods of the above embodiments can be completed by instructing relevant hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiments.

[0191] In addition, in each of the embodiments of the present disclosure, each functional unit may be integrated into one processing module, or each unit may exist physically alone, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. When the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0192] The above-mentioned storage medium may be a read-only memory, a magnetic disk, an optical disc, etc. Although the embodiments of the present disclosure have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present disclosure. Those of ordinary skill in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present disclosure.

[0193] It should be understood that various forms of the processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in the present disclosure can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved. No limitation is imposed herein.

[0194] The above specific implementation manners do not constitute a limitation on the protection scope of the present disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principle of the present disclosure shall be included within the protection scope of the present disclosure.

Claims

1. A training method for a point prediction model, characterized in that The method includes: Obtaining a first candidate point prediction model to be trained and a set of first sample point sequences for the current i-th training round; Obtaining a first training order for each first sample point sequence in the set of first sample point sequences, and inputting each first sample point sequence into the first candidate point prediction model one by one according to the first training order, so as to obtain the first hidden state of each first sample point in any sample point sequence one by one in order through the first candidate point prediction model; Obtaining a second hidden state sequence cached in history in the first candidate point prediction model, and training the first candidate point prediction model for the i-th training round according to the first hidden state and the second hidden state sequence to obtain a second candidate point prediction model after the i-th training round ends; Clearing the hidden state of the second candidate point prediction model to obtain a third candidate point prediction model; Obtaining a set of second sample point sequences for the (i + 1)-th training round, and continuing to train the third candidate point prediction model according to the set of second sample point sequences until the training ends to obtain a trained target point prediction model.

2. The method according to claim 1, characterized in that The obtaining a first training order for each first sample point sequence in the set of first sample point sequences, and inputting each first sample point sequence into the first candidate point prediction model one by one according to the first training order, so as to obtain the first hidden state of each first sample point in any sample point sequence one by one in order through the first candidate point prediction model, includes: According to the first training order, obtaining the k-th first sample point sequence used in the j-th training sub-round in the i-th training round from the set of first sample point sequences; Obtaining a hidden layer in the first candidate point prediction model, wherein a recurrent neural network unit is arranged in the hidden layer; Through the recurrent neural network unit in the hidden layer, obtaining the first hidden state of each first sample point one by one based on the calculation order of each first sample point in the k-th first sample point sequence.

3. The method according to claim 2, wherein The obtaining a second hidden state sequence cached in history in the first candidate point prediction model, and training the first candidate point prediction model for the i-th training round according to the first hidden state and the second hidden state sequence to obtain a second candidate point prediction model after the i-th training round ends, includes: For the j-th training sub-round, obtaining the k-th first sample point sequence of the j-th training sub-round, and obtaining a first target output point sequence of the k-th first sample point sequence output by the first candidate point prediction model according to the first hidden state of each first sample point in the k-th first sample point sequence and the second hidden state sequence, so as to obtain a first training loss of the first candidate point prediction model in the j-th training sub-round; Adjust the model parameters of the first candidate point prediction model according to the first training loss, and return the first candidate point prediction model with adjusted model parameters that continues to be trained using the (k + 1)-th first sample point sequence obtained from the first sample point sequence set for the (j + 1)-th training sub-round until the end of the i-th training round, thereby obtaining the second candidate point prediction model after the end of the i-th training round.

4. The method according to claim 3, characterized in that, For the j-th training sub-round, obtain the k-th first sample point sequence of the j-th training sub-round. According to the first hidden states of the first sample points in the k-th first sample point sequence and the second hidden state sequence, obtain the first target output point sequence of the k-th first sample point sequence output by the first candidate point prediction model, so as to obtain the first training loss of the first candidate point prediction model in the j-th training sub-round, including: Obtain the output sequence length of the first candidate point prediction model; According to the first hidden state and the second hidden state sequence, obtain the sample point sequence feature information of the k-th first sample point sequence, and according to the output sequence length and the sample point sequence feature information, obtain the first target output point sequence corresponding to the k-th first sample point sequence; Obtain the label point sequence of the k-th first sample point sequence, and use the first target output point sequence and the label point sequence to obtain the first training loss of the first candidate point prediction model in the j-th training sub-round.

5. The method according to claim 4, wherein The step of obtaining the first target output point sequence corresponding to the k-th first sample point sequence according to the output sequence length and the sample point sequence feature information includes: For any first sample point in the k-th first sample point sequence, obtain the number of output point timestamps corresponding to the first sample point according to the output sequence length; Obtain the first point timestamp of the first sample point, the second point timestamp at the next moment of the first point timestamp, and the third point timestamp corresponding to the number of output point timestamps after the second point timestamp; According to the sample point sequence feature information, obtain the first sample segment to which the first sample point belongs, and according to the second point timestamp and the third point timestamp, extract the candidate output points of the first sample point from the first sample segment to obtain the candidate output point sequence of the first sample point; According to the candidate output point sequences of the first sample points in the k-th first sample point sequence, form the first target output point sequence of the k-th first sample point sequence.

6. The method according to claim 1, characterized in that The step of clearing the hidden state of the second candidate point prediction model to obtain a third candidate point prediction model includes: Obtain the third hidden state sequence in the second candidate point prediction model, and adjust the third hidden state sequence to a cleared state sequence to obtain the third candidate point prediction model, where the cleared state sequence includes either a zero vector sequence or a preset initial value vector sequence.

7. The method according to claim 1, wherein Obtaining a second sample point sequence set for the (i + 1)-th training round, and continuing to perform model training on the third candidate point prediction model according to the second sample point sequence set until the training ends to obtain a trained target point prediction model, including: Obtaining a second training sorting for each second sample point sequence in the second sample point sequence set; Sequentially inputting the second sample point sequence set into the third candidate point prediction model one by one according to the second training sorting to perform model training on the third candidate point prediction model for the (i + 1)-th training round until the (i + 1)-th training round ends to obtain a fourth candidate point prediction model; In response to the fourth candidate point prediction model satisfying a preset model training end condition, determining the fourth candidate point prediction model as the trained target point prediction model; In response to the fourth candidate point prediction model not satisfying a preset overall model training end condition, returning to obtain a third sample point sequence set for the (i + 2)-th training round, and continuing to train the fourth candidate point prediction model until the overall training ends to obtain a trained target point prediction model.

8. The method according to claim 1, wherein The obtaining of a first candidate point prediction model to be trained and a first sample point sequence set for the current i-th training round includes: Obtaining a second sample line segment set used in the i-th training round and a sample sampling timestamp set for each second sample line segment; Based on the sample sampling timestamp set, performing point sampling on each second sample line segment to obtain a sample candidate line segment point information set for each second sample line segment, where for any sample candidate line segment point information, the sample candidate line segment point information at least includes a point identifier and corresponding metadata; For any sample sampling timestamp, obtaining sample target line segment point information of each second sample line segment at the sample sampling timestamp from the sample candidate line segment point information set of each second sample line segment, and obtaining a first sample point sequence corresponding to the sample sampling timestamp according to the sample target line segment point information; According to the first sample point sequences of the respective sample sampling timestamp sets, obtaining the first sample point sequence set for the i-th training round.

9. A touch point prediction method, characterized in that The method includes: Obtaining a trained target point prediction model, where the target point prediction model is obtained based on the point prediction model training method described in any one of the above claims 1-8; Obtaining a candidate touch point currently received on the user side, and inputting candidate touch point information of the candidate touch point into the target point prediction model to obtain a candidate hidden state of the candidate touch point through a target hidden layer in the target point prediction model; Obtaining a hidden state cache sequence in the target point prediction model, where the hidden state cache sequence includes the candidate hidden state; In response to the hidden state cache sequence satisfying a preset model prediction condition, input the hidden state cache sequence into a downstream model layer of the target hidden layer in the target point prediction model to predict the touch point, and output a target predicted touch point sequence corresponding to the candidate touch point, where the target predicted touch point sequence is used to characterize the touch sequence received by the client based on the candidate touch point within a future time range.

10. The method according to claim 9, wherein The method further includes: In response to the number of state caches of the hidden state cache sequence being equal to a preset cache number upper limit value, determine that it is recognized that the hidden state cache sequence satisfies the model prediction condition; In response to the number of state caches of the hidden state cache sequence being less than a preset cache number threshold, continue to perform hidden state caching until a new hidden state cache sequence satisfies the model prediction condition.

11. The method according to claim 10, wherein The method further includes: In response to the number of state caches of the hidden state cache sequence being greater than or equal to the cache number threshold and less than the cache number upper limit value, perform blank position filling on the hidden state cache sequence, and input the filled hidden state cache sequence into a downstream model layer of the target hidden layer in the target point prediction model to output a target predicted touch point sequence corresponding to the candidate touch point.

12. A training device for a point prediction model, characterized in that, The apparatus includes: A first acquisition module, configured to acquire a first candidate point prediction model to be trained, and a first sample point sequence set of the current i-th training round; A first training module, configured to acquire a first training order of each first sample point sequence in the first sample point sequence set, and input each first sample point sequence into the first candidate point prediction model one by one according to the first training order, so as to obtain a first hidden state of each first sample point in any sample point sequence one by one in sequence through the first candidate point prediction model; A second training module, configured to acquire a second hidden state sequence cached in history in the first candidate point prediction model, and perform model training on the first candidate point prediction model for the i-th training round according to the first hidden state and the second hidden state sequence, to obtain a second candidate point prediction model after the end of the i-th training round of training; A clearing module, configured to clear the hidden state of the second candidate point prediction model to obtain a third candidate point prediction model; A third training module, configured to acquire a second sample point sequence set of the (i + 1)-th training round, and continue to perform model training on the third candidate point prediction model according to the second sample point sequence set until the training ends, to obtain a trained target point prediction model.

13. A touch point prediction device, characterized in that, The apparatus includes: A second acquisition module, configured to acquire a trained target point prediction model, where the target point prediction model is obtained based on the point prediction model training apparatus described in claim 12 above; A third acquisition module, configured to acquire candidate touch points currently received on a client, and input candidate touch point information of the candidate touch points into the target point prediction model, and obtain candidate hidden states of the candidate touch points through a target hidden layer in the target point prediction model; A fourth acquisition module, configured to acquire a hidden state cache sequence in the target point prediction model, where the hidden state cache sequence includes the candidate hidden states; A prediction module, configured to, in response to the hidden state cache sequence satisfying a preset model prediction condition, input the hidden state cache sequence into a downstream model layer of the target point prediction model located downstream of the target hidden layer to perform prediction of touch points, and output a target prediction touch point sequence corresponding to the candidate touch points, where the target prediction touch point sequence is used to represent a touch sequence received by the client based on the candidate touch points within a future time range.

14. An electronic device, characterized in that, Comprising: A processor; A memory for storing executable instructions of the processor; Wherein, the processor is configured to execute instructions to implement the method according to any one of claims 1-8 and / or 9-11.

15. A computer-readable storage medium, when instructions in the computer-readable storage medium are executed by a processor of an electronic device, enabling the electronic device to execute the method according to any one of claims 1-8 and / or 9-11.

16. A chip, characterized in that, Comprising one or more interface circuits and one or more processors; the interface circuit is configured to receive a signal and send the signal to the processor, the signal includes computer instructions stored in a memory, and when the processor executes the computer instructions, enabling the chip to execute the steps of the method according to any one of claims 1-8 and / or 9-11.

Images