Training method of touch track prediction model and prediction method of touch track

Abstract

The invention relates to a training method of a touch track prediction model and a prediction method of a touch track. Comprising the steps of obtaining a first track point sequence of a first time period and a second track point sequence of a second time period; training the feature extraction layer and the first feature mapping layer based on the first track point sequence and the second track point sequence to obtain a trained feature extraction layer and a trained first feature mapping layer; obtaining a first calibration period based on the first track point sequence, the second track point sequence, the trained feature extraction layer and the trained first feature mapping layer; training the second feature mapping layer based on the first track point sequence and the first calibration period to obtain a trained second feature mapping layer; and training the trained feature extraction layer, the trained first feature mapping layer and the trained second feature mapping layer based on the first track point sequence, the second track point sequence and the first calibration period to obtain a touch track prediction model. And efficient trajectory prediction can be carried out.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular, to a method for training a touch trajectory prediction model and a method for predicting a touch trajectory. Background Art

[0002] In the related art, when predicting a user's touch trajectory, two independent neural network models are usually used to predict a predicted trajectory and an optimal prediction period, and a part of the trajectory is selected from the predicted trajectory based on the prediction period as the final prediction result. Summary of the Invention

[0003] To overcome the problems existing in the related art, the present disclosure provides a method for training a touch trajectory prediction model, a method for predicting a touch trajectory, an apparatus, a device, and a storage medium.

[0004] According to a first aspect of an embodiment of the present disclosure, there is provided a method for training a touch trajectory prediction model. The touch trajectory prediction model includes an input layer, a feature extraction layer, a first feature mapping layer, a second feature mapping layer, a connection layer, and an output layer. The first feature mapping layer and the second feature mapping layer are respectively and independently connected to the feature extraction layer and the connection layer. The method includes: obtaining a first trajectory point sequence in a first time period and a second trajectory point sequence in a second time period; wherein the first time period and the second time period are consecutive time periods; training the feature extraction layer and the first feature mapping layer based on the first trajectory point sequence and the second trajectory point sequence to obtain a trained feature extraction layer and a trained first feature mapping layer; obtaining a first calibration period based on the first trajectory point sequence, the second trajectory point sequence, the trained feature extraction layer, and the trained first feature mapping layer; training the second feature mapping layer based on the first trajectory point sequence and the first calibration period to obtain a trained second feature mapping layer; and training the trained feature extraction layer, the trained first feature mapping layer, and the trained second feature mapping layer based on the first trajectory point sequence, the second trajectory point sequence, and the first calibration period to obtain the touch trajectory prediction model.

[0005] In one implementation, training the feature extraction layer and the first feature mapping layer based on the first trajectory point sequence and the second trajectory point sequence to obtain a trained feature extraction layer and a trained first feature mapping layer includes: inputting the first trajectory point sequence into the feature extraction layer for feature extraction to obtain first trajectory point features; inputting the first trajectory point features into the first feature mapping layer for feature mapping to obtain a first predicted trajectory point sequence; based on the first predicted trajectory point sequence and the second trajectory point sequence, adjusting the parameters of the feature extraction layer and the first feature mapping layer to obtain the trained feature extraction layer and the trained first feature mapping layer.

[0006] In one implementation, obtaining a first calibration period based on the first trajectory point sequence, the second trajectory point sequence, the trained feature extraction layer, and the trained first feature mapping layer includes: inputting the first trajectory point sequence into the trained feature extraction layer and the trained first feature mapping layer to obtain a second predicted trajectory point sequence corresponding to the second time period; obtaining, for each first trajectory point in the second trajectory sequence, the corresponding second trajectory point in the second predicted trajectory point sequence; obtaining the trajectory point deviation between each first trajectory point and the corresponding second trajectory point; based on the trajectory point deviation between each first trajectory point and the corresponding second trajectory point, obtaining target trajectory points from the first trajectory points; and obtaining the first calibration period based on the target trajectory points.

[0007] In one implementation, training the second feature mapping layer based on the first trajectory point sequence and the first calibration period to obtain a trained second feature mapping layer includes: inputting the first trajectory point sequence into the trained feature extraction layer for feature extraction to obtain second trajectory point features; based on the second trajectory point features and the second feature mapping layer, performing feature mapping to obtain a first predicted period; based on the first predicted period and the first calibration period, adjusting the parameters of the second feature mapping layer to obtain the trained second feature mapping layer.

[0008] In an optional implementation, performing feature mapping based on the second trajectory point features and the second feature mapping layer to obtain a first predicted period includes: inputting the second trajectory point features into the trained first feature mapping layer to obtain hidden layer features of the trained first feature mapping layer; inputting the second trajectory point features and the hidden layer features into the second feature mapping layer for feature mapping to obtain the first predicted period.

[0009] In one implementation, training the trained feature extraction layer, the trained first feature mapping layer, and the trained second feature mapping layer based on the first trajectory point sequence, the second trajectory point sequence, and the first calibration period to obtain the touch trajectory prediction model includes: inputting the first trajectory point sequence into the trained feature extraction layer for feature extraction to obtain third trajectory point features; inputting the third trajectory point features into the trained first feature mapping layer for feature mapping to obtain a third predicted trajectory point sequence; inputting the third trajectory point features into the trained second feature mapping layer for feature mapping to obtain a second prediction period; determining whether a convergence condition is satisfied based on the third predicted trajectory point sequence, the second trajectory point sequence, the second prediction period, and the first calibration period; and in response to the convergence condition being satisfied, obtaining the touch trajectory prediction model based on the trained feature extraction layer, the trained first feature mapping layer, and the trained second feature mapping layer.

[0010] In an alternative implementation, determining whether the convergence condition is satisfied based on the third predicted trajectory point sequence, the second trajectory point sequence, the second prediction period, and the first calibration period includes: obtaining a first deviation value between the third predicted trajectory point sequence and the second trajectory point sequence; obtaining a second deviation value between the first calibration period and the second prediction period; and determining that the convergence condition is satisfied in response to the first deviation value being less than or equal to a preset first threshold and / or the second deviation value being less than or equal to a preset second threshold.

[0011] In an alternative implementation, the method further includes: in response to the convergence condition not being satisfied, adjusting the parameters of the trained feature extraction layer, the parameters of the trained first feature mapping layer, and the parameters of the trained second feature mapping layer based on the third predicted trajectory point sequence, the second trajectory point sequence, the second prediction period, and the first calibration period to obtain a re-trained feature extraction layer, a re-trained first feature mapping layer, and a re-trained second feature mapping layer; determining a second calibration period based on the third predicted trajectory point sequence and the second trajectory point sequence; using the second calibration period as the new first calibration period, the re-trained feature extraction layer as the new trained feature extraction layer, the re-trained first feature mapping layer as the new trained first feature mapping layer, and the re-trained second feature mapping layer as the new trained second feature mapping layer, and returning to execute the step of inputting the first trajectory point sequence into the trained feature extraction layer for feature extraction to obtain third trajectory point features.

[0012] According to a second aspect of the embodiments of the present disclosure, a method for predicting a touch trajectory is provided, including: obtaining a sequence of trajectory points within a preset time period; inputting the sequence of trajectory points into a pre-trained touch trajectory prediction model to obtain output data of the touch trajectory prediction model; wherein, the touch trajectory prediction model is pre-trained based on the training method of the touch trajectory prediction model according to any one of claims 1 to 8; and obtaining a target sequence of trajectory points based on the output data.

[0013] In an optional implementation manner, the touch trajectory prediction model includes an input layer, a feature extraction layer, a first feature mapping layer, a second feature mapping layer, a connection layer, and an output layer. The first feature mapping layer and the second feature mapping layer are respectively and independently connected to the feature extraction layer and the connection layer. The step of inputting the sequence of trajectory points into a pre-trained touch trajectory prediction model to obtain output data of the touch trajectory prediction model includes: inputting the sequence of trajectory points through the input layer into the feature extraction layer for feature extraction to obtain trajectory point features; inputting the trajectory point features into the first feature mapping layer for feature mapping to obtain a predicted sequence of trajectory points; inputting the trajectory point features into the second feature mapping layer for feature mapping to obtain a prediction period; and inputting the predicted sequence of trajectory points and the prediction period into the output layer for splicing to obtain the output data.

[0014] Optionally, the step of obtaining a predicted sequence of trajectory points based on the output data includes: separating the predicted sequence of trajectory points and the prediction period from the output data; and intercepting the target sequence of trajectory points from the predicted sequence of trajectory points based on the prediction period.

[0015] According to a third aspect of the embodiments of the present disclosure, there is provided a training device for a touch trajectory prediction model. The touch trajectory prediction model includes an input layer, a feature extraction layer, a first feature mapping layer, a second feature mapping layer, a connection layer, and an output layer. The first feature mapping layer and the second feature mapping layer are respectively and independently connected to the feature extraction layer and the connection layer. The device includes: an acquisition module, configured to acquire a first trajectory point sequence in a first time period and a second trajectory point sequence in a second time period; wherein the first time period and the second time period are consecutive time periods; a first training module, configured to train the feature extraction layer and the first feature mapping layer based on the first trajectory point sequence and the second trajectory point sequence to obtain a trained feature extraction layer and a trained first feature mapping layer; a processing module, configured to obtain a first calibration period based on the first trajectory point sequence, the second trajectory point sequence, the trained feature extraction layer, and the trained first feature mapping layer; a second training module, configured to train the second feature mapping layer based on the first trajectory point sequence and the first calibration period to obtain a trained second feature mapping layer; and a third training module, configured to train the trained feature extraction layer, the trained first feature mapping layer, and the trained second feature mapping layer based on the first trajectory point sequence, the second trajectory point sequence, and the first calibration period to obtain the touch trajectory prediction model.

[0016] In one implementation, the first training module is specifically configured to: input the first trajectory point sequence into the feature extraction layer for feature extraction to obtain first trajectory point features; input the first trajectory point features into the first feature mapping layer for feature mapping to obtain a first predicted trajectory point sequence; and adjust the parameters of the feature extraction layer and the first feature mapping layer based on the first predicted trajectory point sequence and the second trajectory point sequence to obtain the trained feature extraction layer and the trained first feature mapping layer.

[0017] In one implementation, the processing module is specifically configured to: input the first trajectory point sequence into the trained feature extraction layer and the trained first feature mapping layer to obtain a second predicted trajectory point sequence corresponding to the second time period; acquire a second trajectory point corresponding to each first trajectory point in the second trajectory sequence in the second predicted trajectory point sequence; acquire a trajectory point deviation between each first trajectory point and the corresponding second trajectory point; acquire target trajectory points from the first trajectory points based on the trajectory point deviation between each first trajectory point and the corresponding second trajectory point; and obtain the first calibration period based on the target trajectory points.

[0018] In one implementation, the second training module is specifically configured to: input the first trajectory point sequence into the trained feature extraction layer for feature extraction to obtain second trajectory point features; perform feature mapping based on the second trajectory point features and the second feature mapping layer to obtain a first prediction period; adjust the parameters of the second feature mapping layer based on the first prediction period and the first calibration period to obtain the trained second feature mapping layer.

[0019] In an alternative implementation, the second training module is specifically configured to: input the second trajectory point features into the trained first feature mapping layer to obtain hidden layer features of the trained first feature mapping layer; input the second trajectory point features and the hidden layer features into the second feature mapping layer for feature mapping to obtain the first prediction period.

[0020] In one implementation, the third training module is specifically configured to: input the first trajectory point sequence into the trained feature extraction layer for feature extraction to obtain third trajectory point features; input the third trajectory point features into the trained first feature mapping layer for feature mapping to obtain a third predicted trajectory point sequence; input the third trajectory point features into the trained second feature mapping layer for feature mapping to obtain a second prediction period; determine whether the convergence condition is satisfied based on the third predicted trajectory point sequence, the second trajectory point sequence, the second prediction period, and the first calibration period; in response to satisfying the convergence condition, obtain the touch trajectory prediction model based on the trained feature extraction layer, the trained first feature mapping layer, and the trained second feature mapping layer.

[0021] In an alternative implementation, the third training module is specifically configured to: obtain a first deviation value between the third predicted trajectory point sequence and the second trajectory point sequence; obtain a second deviation value between the first calibration period and the second prediction period; determine that the convergence condition is satisfied in response to the first deviation value being less than or equal to a preset first threshold and / or the second deviation value being less than or equal to a preset second threshold.

[0022] In an alternative implementation, the third training module is further configured to: in response to the convergence condition not being met, based on the third predicted trajectory point sequence, the second trajectory point sequence, the second prediction period, and the first calibration period, adjust the parameters of the trained feature extraction layer, the parameters of the trained first feature mapping layer, and the parameters of the trained second feature mapping layer to obtain a re-trained feature extraction layer, a re-trained first feature mapping layer, and a re-trained second feature mapping layer; determine a second calibration period based on the third predicted trajectory point sequence and the second trajectory point sequence; use the second calibration period as the new first calibration period, the re-trained feature extraction layer as the new trained feature extraction layer, the re-trained first feature mapping layer as the new trained first feature mapping layer, and the re-trained second feature mapping layer as the new trained second feature mapping layer, and return to execute the step of inputting the first trajectory point sequence into the trained feature extraction layer to perform feature extraction to obtain the third trajectory point feature.

[0023] According to a fourth aspect of the embodiments of the present disclosure, there is provided a touch trajectory prediction device, including: an acquisition module configured to acquire a trajectory point sequence within a preset time period; a first processing module configured to input the trajectory point sequence into a pre-trained touch trajectory prediction model to obtain output data of the touch trajectory prediction model; wherein, the touch trajectory prediction model is pre-trained based on the touch trajectory prediction model training device as described in the first aspect; a second processing module configured to obtain a target trajectory point sequence based on the output data.

[0024] In an alternative implementation, the touch trajectory prediction model includes an input layer, a feature extraction layer, a first feature mapping layer, a second feature mapping layer, a connection layer, and an output layer. The first feature mapping layer and the second feature mapping layer are respectively and independently connected to the feature extraction layer and the connection layer. The first processing module is specifically configured to: input the trajectory point sequence into the feature extraction layer through the input layer to perform feature extraction to obtain trajectory point features; input the trajectory point features into the first feature mapping layer to perform feature mapping to obtain a predicted trajectory point sequence; input the trajectory point features into the second feature mapping layer to perform feature mapping to obtain a prediction period; and input the predicted trajectory point sequence and the prediction period into the output layer to perform splicing to obtain the output data.

[0025] Optionally, the second processing module is specifically configured to: separate the predicted trajectory point sequence and the prediction period from the output data; and intercept the target trajectory point sequence from the predicted trajectory point sequence based on the prediction period.

[0026] According to a fifth aspect of the embodiments of the present disclosure, there is provided an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and when the instructions are executed by the at least one processor, the at least one processor is enabled to execute the method as described in the foregoing first aspect.

[0027] According to a sixth aspect of the embodiments of the present disclosure, there is provided an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and when the instructions are executed by the at least one processor, the at least one processor is enabled to execute the method as described in the foregoing second aspect.

[0028] According to a seventh aspect of the embodiments of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the method as described in the foregoing first aspect.

[0029] According to an eighth aspect of the embodiments of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the method as described in the foregoing second aspect.

[0030] According to a ninth aspect of the embodiments of the present disclosure, the present disclosure provides a computer program product, including a computer program, and when the computer program is executed by a processor, the steps of the method as described in the first aspect are implemented.

[0031] According to a tenth aspect of the embodiments of the present disclosure, the present disclosure provides a computer program product, including a computer program, and when the computer program is executed by a processor, the steps of the method as described in the second aspect are implemented.

[0032] The technical solutions provided by the embodiments of the present disclosure may include the following beneficial effects: The feature extraction layer and the first feature mapping layer can be trained based on the first trajectory point sequence and the second trajectory point sequence of two consecutive time periods, and based on the first trajectory point sequence, the second trajectory point sequence, the trained feature extraction layer, and the trained first feature mapping layer, a first calibration period is obtained, so as to train the second feature mapping layer based on the first calibration period and the first trajectory point sequence, and perform cyclic training on the trained feature extraction layer, the trained first feature mapping layer, and the trained second feature mapping layer based on the first trajectory point sequence, the second trajectory point sequence, and the first calibration period, and finally obtain a touch trajectory prediction model. Thereby improving the training effect and obtaining a more effective touch trajectory prediction model, so as to perform more efficient and accurate touch trajectory prediction based on the trained touch trajectory prediction model.

[0033] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and do not limit the present disclosure. Brief Description of the Drawings

[0034] The accompanying drawings herein are incorporated into and constitute a part of this specification, showing embodiments consistent with the present invention, and are used together with the specification to explain the principles of the present invention.

[0035] Figure 1 is a schematic structural diagram of a touch trajectory prediction model shown according to an exemplary embodiment.

[0036] Figure 2 is a flowchart of a method for training a touch trajectory prediction model shown according to an exemplary embodiment.

[0037] Figure 3 is a flowchart of another method for training a touch trajectory prediction model shown according to an exemplary embodiment.

[0038] Figure 4 is a flowchart of still another method for training a touch trajectory prediction model shown according to an exemplary embodiment.

[0039] Figure 5 is a flowchart of still another method for training a touch trajectory prediction model shown according to an exemplary embodiment.

[0040] Figure 6 is a flowchart of still another method for training a touch trajectory prediction model shown according to an exemplary embodiment.

[0041] Figure 7 is a flowchart of a method for predicting a touch trajectory shown according to an exemplary embodiment.

[0042] Figure 8 is a flowchart of another method for predicting a touch trajectory shown according to an exemplary embodiment.

[0043] Figure 9 is a block diagram of an apparatus for training a touch trajectory prediction model shown according to an exemplary embodiment.

[0044] Figure 10 is a block diagram of an apparatus for predicting a touch trajectory shown according to an exemplary embodiment.

[0045] Figure 11 is a block diagram of an electronic device shown according to an exemplary embodiment.

[0046] Figure 12 is a block diagram of an apparatus shown according to an exemplary embodiment. Detailed Embodiments

[0047] Exemplary embodiments will be described in detail herein, and examples thereof are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present invention as detailed in the appended claims.

[0048] The various numerical numbers such as first and second involved in this disclosure are only for the convenience of description and do not limit the scope of the embodiments of this disclosure, nor do they represent the order of precedence.

[0049] Please refer to Figure 1 , Figure 1 which is a schematic structural diagram of a touch trajectory prediction model shown according to an exemplary embodiment. As Figure 1 shown, the trajectory prediction model includes an input layer, a feature extraction layer, a first feature mapping layer, a second feature mapping layer, a connection layer, and an output layer. The first feature mapping layer and the second feature mapping layer are respectively independently connected to the feature extraction layer and the connection layer. The connection layer can fuse the data output by the first feature mapping layer and the second feature mapping layer and output through the output layer.

[0050] Figure 2 which is a flowchart of a training method of a touch trajectory prediction model shown according to an exemplary embodiment. As Figure 2 shown, the method may include but is not limited to the following steps.

[0051] Step S201: Obtain a first trajectory point sequence in a first time period and a second trajectory point sequence in a second time period.

[0052] Wherein, in the embodiments of the present disclosure, the first time period and the second time period are consecutive time periods.

[0053] For example, obtain a first trajectory point sequence of touch trajectory points input by the user in the first time period, and obtain a second trajectory point sequence of touch trajectory points input by the user in a second time period after the first time period and consecutive with the first time period.

[0054] Wherein, in the embodiments of the present disclosure, the trajectory point sequence includes trajectory data of at least one trajectory point arranged in time within a certain time sequence length, and the trajectory data includes at least one of the following: coordinate data of the trajectory point, pressure data corresponding to the trajectory point, and time data of the trajectory point.

[0055] It should be noted that, in the embodiments of the present disclosure, the above-mentioned first time period and second time period are at least one. Each first time period corresponds to a first trajectory point sequence, and each second time period corresponds to a second trajectory point sequence.

[0056] Step S202: Train the feature extraction layer and the first feature mapping layer based on the first trajectory point sequence and the second trajectory point sequence to obtain the trained feature extraction layer and the trained first feature mapping layer.

[0057] Wherein, in the embodiments of the present disclosure, the above-mentioned feature extraction layer may be a temporal feature extraction layer. For example, the feature extraction layer may be any one of a GRU (Gated Recurrent Unit), an LSTM (Long Short Term Memory) model, and a Transformer model.

[0058] Wherein, in the embodiments of the present disclosure, the first feature mapping layer is used to convert the temporal features extracted by the feature extraction layer into a trajectory point sequence within a predicted time series.

[0059] For example, input the first trajectory point sequence into the feature extraction layer and the first feature mapping layer in sequence to obtain the output data of the first feature mapping layer, and adjust the parameters of the feature extraction layer based on the difference between the output data of the first feature mapping layer and the second trajectory point sequence to obtain the trained feature extraction layer, and adjust the parameters of the first feature mapping layer based on the difference between the output data of the first feature mapping layer and the second trajectory point sequence to obtain the trained first feature mapping layer.

[0060] Step S203: Obtain the first calibration period based on the first trajectory point sequence, the second trajectory point sequence, the trained feature extraction layer, and the trained first feature mapping layer.

[0061] For example, based on the first trajectory point sequence, the trained feature extraction layer, and the trained first feature mapping layer, obtain the predicted trajectory points, compare the predicted trajectory point sequence with the second trajectory point sequence, determine the difference between the corresponding trajectory points in the predicted trajectory point sequence and the second trajectory point sequence, and then determine the target trajectory point in the second trajectory point sequence corresponding to the difference closest to the preset difference threshold, so as to determine the first calibration period based on the time corresponding to the time series data of the target trajectory point.

[0062] Wherein, the above-mentioned first calibration period may be a duration, or the above-mentioned first calibration period may be a scalar indicating the sequence position in the trajectory point sequence.

[0063] Step S204: Train the second feature mapping layer based on the first trajectory point sequence and the first calibration period to obtain the trained second feature mapping layer.

[0064] Among them, in the embodiments of the present disclosure, the second feature mapping layer is used to convert the temporal features extracted by the feature extraction layer into an optimal prediction period. The prediction period can be a period duration, or the prediction period can be a scalar indicating the sequence position in the trajectory point sequence.

[0065] For example, input the feature data corresponding to the first trajectory point sequence into the second feature mapping layer to obtain the output data of the second feature mapping layer, and adjust the parameters of the second feature mapping layer based on the deviation value between the output data of the second feature mapping layer and the first calibration period to obtain the trained second feature mapping layer.

[0066] Among them, the above first calibration period can be a period duration, or the above first calibration period can be a scalar indicating the sequence position in the trajectory point sequence.

[0067] Step S205: Train the trained feature extraction layer, the trained first feature mapping layer, and the trained second feature mapping layer based on the first trajectory point sequence, the second trajectory point sequence, and the first calibration period to obtain a touch trajectory prediction model.

[0068] For example, input the first trajectory point sequence into the trained feature extraction layer for feature extraction to obtain feature data, input the feature data into the trained first feature mapping layer to obtain the first output data, and input the feature data into the trained second feature mapping layer to obtain the second output data. Then, adjust the parameters of the trained feature extraction layer, the trained first feature mapping layer, and the trained second feature mapping layer based on the difference between the first output data and the second trajectory point sequence and the difference between the second output data and the first calibration period, respectively, to obtain the trained feature extraction layer, the trained first feature mapping layer, and the trained second feature mapping layer. Thus, obtain the trained touch trajectory prediction model based on the trained feature extraction layer, the trained first feature mapping layer, and the trained second feature mapping layer.

[0069] By implementing the disclosed embodiments, the feature extraction layer and the first feature mapping layer can be trained based on the first trajectory point sequence and the second trajectory point sequence in two consecutive time periods, and the first calibration period can be obtained based on the first trajectory point sequence, the second trajectory point sequence, the trained feature extraction layer, and the trained first feature mapping layer. Then, the second feature mapping layer can be trained based on the first calibration period and the first trajectory point sequence, and the trained feature extraction layer, the trained first feature mapping layer, and the trained second feature mapping layer can be trained based on the first trajectory point sequence, the second trajectory point sequence, and the first calibration period. Finally, a touch trajectory prediction model is obtained. Thereby, the training effect is improved, and a more effective touch trajectory prediction model is obtained to perform more efficient and accurate touch trajectory prediction based on the trained touch trajectory prediction model.

[0070] In one implementation, the trajectory point sequence after the first period can be predicted based on the feature extraction layer and the first feature mapping layer, and the feature extraction layer and the first feature mapping layer can be trained based on the prediction result and the second trajectory point sequence. As an example, please refer to Figure 3 , Figure 3 which is a flowchart of another training method of a touch trajectory prediction model shown according to an exemplary embodiment, as Figure 3 shown, the method may include but is not limited to the following steps.

[0071] Step S301: Obtain a first trajectory point sequence in a first period and a second trajectory point sequence in a second period.

[0072] In the embodiments of the present disclosure, step S301 can be implemented in any one of the various embodiments of the present disclosure. The embodiments of the present disclosure do not make any limitations in this regard and will not be elaborated further.

[0073] Step S302: Input the first trajectory point sequence into the feature extraction layer for feature extraction to obtain first trajectory point features.

[0074] For example, the first trajectory point sequence is input into the input layer of the touch trajectory prediction model through the input layer, and then input into the feature extraction layer for feature extraction to obtain the first trajectory point features corresponding to the first trajectory point sequence.

[0075] In some embodiments, the above first trajectory point sequence can be represented as a dimensional vector of batchsize*inputLength*featureSize. Where batchsize is the number of input batches (i.e., the number of first trajectory point sequences), inputLength is the number of points in the input sequence (i.e., the number of touch trajectory points in each first trajectory point sequence), and featureSize is the number of features (i.e., the feature dimension of each touch trajectory point).

[0076] Step S303: Input the first trajectory point features into the first feature mapping layer for feature mapping to obtain a first predicted trajectory point sequence.

[0077] For example, input the first trajectory point features into the first feature mapping layer to perform feature mapping in the time and feature dimensions respectively, and obtain the first predicted trajectory point sequence output by the first feature mapping layer.

[0078] It can be understood that the first predicted trajectory point sequence is the predicted trajectory point sequence for a period of time after the first period predicted by the first mapping layer. There is at least a partial overlap period between this period of time after the first period and the second period, so that at least some of the trajectory points in the predicted trajectory point sequence have a corresponding relationship with at least some of the trajectory points in the second trajectory point sequence. For example, the trajectory point corresponding to the first moment in the predicted trajectory point sequence has a corresponding relationship with the trajectory point corresponding to the first moment in the second trajectory point sequence. Wherein, the first moment is any moment in the period where there is at least a partial overlap period between the period of time after the first period and the second period.

[0079] In some embodiments, the first feature mapping layer can perform feature mapping in the time and feature dimensions respectively through the included Transpose (transpose convolution) and / or Dense (fully connected) layers. Alternatively, the first feature mapping layer can use other decoders for feature mapping. For example, GRU.

[0080] Among them, the first predicted trajectory point sequence can be represented as a vector of batchsize*outputLength*outputDim. Wherein, batchsize is the input batch number, that is, the number of the first trajectory point sequences; outputLength is the time series length corresponding to the first predicted trajectory point sequence; outputDim is the dimension of a single element in the first predicted trajectory point sequence. For example, the X-axis coordinate value and Y-axis coordinate value of a single trajectory point.

[0081] Step S304: Based on the first predicted trajectory point sequence and the second trajectory point sequence, adjust the parameters of the feature extraction layer and the first feature mapping layer to obtain the trained feature extraction layer and the trained first feature mapping layer.

[0082] For example, based on the deviation value between the second predicted trajectory point sequence and the corresponding two trajectory points in the second trajectory point sequence, adjust the parameters of the feature extraction layer and the first feature mapping layer to reduce the deviation between the predicted trajectory point sequence output by the first feature mapping layer and the actual trajectory point sequence, and obtain the trained feature extraction layer and the trained first feature mapping layer.

[0083] Step S305: Based on the first trajectory point sequence, the second trajectory point sequence, the trained feature extraction layer and the trained first feature mapping layer, obtain the first calibration period.

[0084] In the embodiments of the present disclosure, step S305 can be implemented in any of the ways in the various embodiments of the present disclosure. The embodiments of the present disclosure do not limit this and will not be elaborated further.

[0085] Step S306: Train the second feature mapping layer based on the first trajectory point sequence and the first calibration period to obtain the trained second feature mapping layer.

[0086] In the embodiments of the present disclosure, step S306 can be implemented in any of the ways in the various embodiments of the present disclosure. The embodiments of the present disclosure do not limit this and will not elaborate further.

[0087] Step S307: Train the trained feature extraction layer, the trained first feature mapping layer, and the trained second feature mapping layer based on the first trajectory point sequence, the second trajectory point sequence, and the first calibration period to obtain a touch trajectory prediction model.

[0088] In the embodiments of the present disclosure, step S307 can be implemented in any of the ways in the various embodiments of the present disclosure. The embodiments of the present disclosure do not limit this and will not elaborate further.

[0089] By implementing the embodiments of the present disclosure, based on the first trajectory point sequence, the feature extraction layer, and the first feature mapping layer in the first time period, the deviation between the prediction data domain of the first feature mapping layer and the second trajectory point sequence can be obtained, so as to train the first feature mapping layer based on this deviation. The training effect can be improved, and a more effective touch trajectory prediction model can be obtained, so as to perform more efficient and accurate touch trajectory prediction based on the trained touch trajectory prediction model.

[0090] In some embodiments, the prediction data can be obtained based on the trained feature extraction layer and feature mapping layer, and the first calibration period can be obtained based on the prediction data and the second trajectory point sequence. As an example, please refer to Figure 4 , Figure 4 is a flowchart of another method for training a touch trajectory prediction model shown according to an exemplary embodiment. As shown in Figure 4 shown, the method can include but is not limited to the following steps.

[0091] Step S401: Obtain the first trajectory point sequence in the first time period and the second trajectory point sequence in the second time period.

[0092] In the embodiments of the present disclosure, step S401 can be implemented in any of the ways in the various embodiments of the present disclosure. The embodiments of the present disclosure do not limit this and will not elaborate further.

[0093] Step S402: Train the feature extraction layer and the first feature mapping layer based on the first trajectory point sequence and the second trajectory point sequence to obtain the trained feature extraction layer and the trained first feature mapping layer.

[0094] In an embodiment of the present disclosure, step S402 may be implemented in any one of the embodiments of the present disclosure. The embodiments of the present disclosure do not limit this and will not be elaborated further.

[0095] Step S403: Input the first trajectory point sequence into the trained feature extraction layer and the trained first feature mapping layer to obtain the second predicted trajectory point sequence corresponding to the second time period.

[0096] For example, input the first trajectory point sequence into the trained feature extraction layer for feature extraction to obtain feature data, and input the feature data into the trained first feature mapping layer for feature mapping to obtain the second predicted trajectory point sequence corresponding to the second time period predicted by the trained first feature mapping layer.

[0097] Step S404: Obtain the second trajectory point corresponding to each first trajectory point in the second trajectory sequence in the second predicted trajectory point sequence.

[0098] As an example, based on the sequence position of each first trajectory point in the first trajectory sequence and the sequence position of each second trajectory point in the second predicted trajectory point sequence, the first trajectory point and the second trajectory point with the same sequence position are used as a pair of corresponding trajectory points in the second trajectory sequence and the second predicted trajectory sequence.

[0099] As an example, based on the time data corresponding to each first trajectory point in the first trajectory sequence and the time data corresponding to each second trajectory point in the second predicted trajectory point sequence, the first trajectory point and the second trajectory point with the same time data are used as a pair of corresponding trajectory points in the second trajectory sequence and the second predicted trajectory sequence. Wherein, the time data may be a time tag.

[0100] Step S405: Obtain the trajectory point deviation between each first trajectory point and the corresponding second trajectory point.

[0101] For example, based on the trajectory point data corresponding to the first trajectory point and the trajectory point data corresponding to the second trajectory point, obtain the trajectory point deviation between each first trajectory point and the corresponding second trajectory point.

[0102] As an example, taking the trajectory point data including trajectory point coordinates as an example. The above trajectory point deviation may be a trajectory point coordinate deviation.

[0103] Step S406: Based on the trajectory point deviation between each first trajectory point and the corresponding second trajectory point, obtain the target trajectory point from the first trajectory points.

[0104] As an example, obtain the first trajectory point with the smallest trajectory point deviation as the target trajectory point.

[0105] As another example, obtain a first trajectory point whose trajectory point deviation is less than a preset deviation threshold as the target trajectory point.

[0106] Step S407: Obtain a first calibration period based on the target trajectory point.

[0107] In some embodiments, use the sequence position of the target trajectory point in the second trajectory sequence as the first calibration period.

[0108] As an example, take the target trajectory point as 1. If the sequence position of the target trajectory point in the second trajectory sequence is 6, then the first calibration period is 6.

[0109] As another example, take the target trajectory points as multiple. The evaluation value of each target trajectory point can be obtained based on the softmax (normalization) function, and the first calibration period can be determined based on the sequence position of the target trajectory point with the highest evaluation value.

[0110] Step S408: Train the second feature mapping layer based on the first trajectory point sequence and the first calibration period to obtain the trained second feature mapping layer.

[0111] In the embodiments of the present disclosure, step S408 can be implemented in any of the ways in the various embodiments of the present disclosure. The embodiments of the present disclosure do not make any limitations in this regard and will not be elaborated further.

[0112] Step S409: Train the trained feature extraction layer, the trained first feature mapping layer, and the trained second feature mapping layer based on the first trajectory point sequence, the second trajectory point sequence, and the first calibration period to obtain a touch trajectory prediction model.

[0113] In the embodiments of the present disclosure, step S409 can be implemented in any of the ways in the various embodiments of the present disclosure. The embodiments of the present disclosure do not make any limitations in this regard and will not be elaborated further.

[0114] By implementing the embodiments of the present disclosure, the first trajectory sequence can be input into the trained feature extraction layer and feature mapping layer to obtain a second predicted trajectory sequence, and calibration can be performed based on the second predicted trajectory sequence and the second trajectory point sequence to obtain a first calibration period, so as to train the second feature mapping layer based on the first calibration period. A more effective touch trajectory prediction model can be obtained, so as to perform more efficient and accurate touch trajectory prediction based on the trained touch trajectory prediction model.

[0115] In some embodiments, the feature extraction layer after training can be used to extract features from the first trajectory point sequence, so as to train the second feature mapping layer based on the extracted features. As an example, please refer to Figure 5 , Figure 5It is a flowchart of another method for training a touch trajectory prediction model shown according to an exemplary embodiment. As Figure 5 shown, the method may include but is not limited to the following steps.

[0116] Step S501: Obtain a first trajectory point sequence in a first time period and a second trajectory point sequence in a second time period.

[0117] In the embodiments of the present disclosure, step S501 can be implemented in any one of the embodiments of the present disclosure. The embodiments of the present disclosure do not make any limitations on this and will not be elaborated further.

[0118] Step S502: Train the feature extraction layer and the first feature mapping layer based on the first trajectory point sequence and the second trajectory point sequence to obtain a trained feature extraction layer and a trained first feature mapping layer.

[0119] In the embodiments of the present disclosure, step S502 can be implemented in any one of the embodiments of the present disclosure. The embodiments of the present disclosure do not make any limitations on this and will not be elaborated further.

[0120] Step S503: Obtain a first calibration period based on the first trajectory point sequence, the second trajectory point sequence, the trained feature extraction layer, and the trained first feature mapping layer.

[0121] In the embodiments of the present disclosure, step S503 can be implemented in any one of the embodiments of the present disclosure. The embodiments of the present disclosure do not make any limitations on this and will not be elaborated further.

[0122] Step S504: Input the first trajectory point sequence into the trained feature extraction layer for feature extraction to obtain second trajectory point features.

[0123] For example, input the first trajectory point sequence into the trained feature extraction layer for feature extraction to obtain the second trajectory point features output by the trained feature extraction layer.

[0124] Step S505: Perform feature mapping based on the second trajectory point features and the second feature mapping layer to obtain a first prediction period.

[0125] As an example, input the second trajectory point features into the second feature mapping layer for feature mapping to obtain the first prediction period output by the second feature mapping layer.

[0126] As an example, input the second trajectory point features into the second feature mapping layer for feature mapping to obtain multiple prediction periods output by the second feature mapping layer and the evaluation value corresponding to each prediction period, and select the one with the highest evaluation value from the multiple prediction periods as the first prediction period.

[0127] In some embodiments, the second feature mapping layer may perform feature mapping in the time and feature dimensions respectively through the included Transpose and / or Dense layers. Alternatively, the second feature mapping layer may use other decoders for feature mapping.

[0128] Step S506: Adjust the parameters of the second feature mapping layer based on the first prediction period and the first calibration period to obtain the trained second feature mapping layer.

[0129] For example, based on the deviation between the first prediction period and the first calibration period, adjust the parameters of the second mapping layer to minimize the deviation between the first prediction period and the first calibration period, and obtain the trained second feature mapping layer.

[0130] In an alternative implementation, performing feature mapping based on the second trajectory point feature and the second feature mapping layer to obtain the first prediction period includes: inputting the second trajectory point feature into the trained first feature mapping layer to obtain the hidden layer feature of the trained first feature mapping layer; inputting the second trajectory point feature and the hidden layer feature into the second feature mapping layer for feature mapping to obtain the first prediction period.

[0131] For example, input the second trajectory point feature into the trained first feature mapping layer to obtain the hidden layer feature of the trained first feature mapping layer, and input the hidden layer feature and the second trajectory point feature into the second feature mapping layer for feature mapping to obtain the first prediction period output by the second feature mapping layer.

[0132] In some embodiments of the present disclosure, when inputting the hidden layer feature into the second feature mapping layer for feature mapping, methods such as dropout may be used to avoid overfitting of the second feature mapping layer.

[0133] Step S507: Train the trained feature extraction layer, the trained first feature mapping layer, and the trained second feature mapping layer based on the first trajectory point sequence, the second trajectory point sequence, and the first calibration period to obtain the touch trajectory prediction model.

[0134] In the embodiments of the present disclosure, step S507 may be implemented in any one of the embodiments of the present disclosure. The embodiments of the present disclosure do not limit this and will not be elaborated further.

[0135] By implementing the embodiments of the present disclosure, a first calibration period can be obtained based on the first trajectory point sequence, the second trajectory point sequence, the trained feature extraction layer, and the trained first feature mapping layer, and the deviation between the prediction data of the second feature mapping layer and the first calibration period can be obtained based on the first trajectory point sequence, so as to train the second feature mapping layer based on the prediction deviation. Thereby, the training effect can be improved, and a more effective touch trajectory prediction model can be obtained, so as to perform more efficient and accurate touch trajectory prediction based on the trained touch trajectory prediction model.

[0136] In one implementation manner, the trained feature extraction layer, the trained first feature mapping layer, and the trained second feature mapping layer can be cyclically trained according to the first trajectory point data and the second trajectory point data. As an example, please refer to Figure 6 , Figure 6 which is a flowchart of another method for training a touch trajectory prediction model shown according to an exemplary embodiment, as Figure 6 shown, the method may include but is not limited to the following steps.

[0137] Step S601: Obtain a first trajectory point sequence in a first time period and a second trajectory point sequence in a second time period.

[0138] In the embodiments of the present disclosure, step S601 can be implemented in any one of the embodiments of the present disclosure, and the embodiments of the present disclosure do not make any limitations thereto and will not be elaborated herein.

[0139] Step S602: Train the feature extraction layer and the first feature mapping layer based on the first trajectory point sequence and the second trajectory point sequence to obtain the trained feature extraction layer and the trained first feature mapping layer.

[0140] In the embodiments of the present disclosure, step S602 can be implemented in any one of the embodiments of the present disclosure, and the embodiments of the present disclosure do not make any limitations thereto and will not be elaborated herein.

[0141] Step S603: Obtain a first calibration period based on the first trajectory point sequence, the second trajectory point sequence, the trained feature extraction layer, and the trained first feature mapping layer.

[0142] In the embodiments of the present disclosure, step S603 can be implemented in any one of the embodiments of the present disclosure, and the embodiments of the present disclosure do not make any limitations thereto and will not be elaborated herein.

[0143] Step S604: Train the second feature mapping layer based on the first trajectory point sequence and the first calibration period to obtain the trained second feature mapping layer.

[0144] In an embodiment of the present disclosure, step S604 can be implemented in any one of the embodiments of the present disclosure. The embodiments of the present disclosure do not limit this and will not be elaborated further.

[0145] Step S605: Input the first trajectory point sequence into the trained feature extraction layer for feature extraction to obtain the second trajectory point feature.

[0146] For example, input the first trajectory point sequence into the trained feature extraction layer for feature extraction to obtain the third trajectory point feature output by the trained feature extraction layer.

[0147] Step S606: Input the third trajectory point feature into the trained first feature mapping layer for feature mapping to obtain the third predicted trajectory point sequence.

[0148] For example, input the third trajectory point feature into the trained first feature mapping layer for feature mapping to obtain the third predicted trajectory point sequence output by the trained first feature mapping layer.

[0149] Step S607: Input the third trajectory point feature into the trained second feature mapping layer for feature mapping to obtain the second predicted period.

[0150] For example, input the third trajectory point feature into the trained second feature mapping layer for feature mapping to obtain the second predicted period output by the trained second feature mapping layer.

[0151] Step S608: Determine whether the convergence condition is satisfied based on the third predicted trajectory point sequence, the second trajectory point sequence, the second predicted period, and the first calibration period.

[0152] For example, determine whether the deviation value between the third predicted trajectory point sequence and the second trajectory point sequence is less than a preset first deviation value, and determine whether the deviation between the second predicted period and the first calibration period is less than a preset second deviation value to determine whether the convergence condition is satisfied.

[0153] In an optional implementation manner, determining whether the convergence condition is satisfied based on the third predicted trajectory point sequence, the second trajectory point sequence, the second predicted period, and the first calibration period includes: obtaining a first deviation value between the third predicted trajectory point sequence and the second trajectory point sequence; obtaining a second deviation value between the first calibration period and the second predicted period; and determining that the convergence condition is satisfied in response to the first deviation value being less than or equal to a preset first threshold and / or the second deviation value being less than or equal to a preset second threshold.

[0154] As an example, in response to that the first deviation value between the third predicted trajectory point sequence and the second trajectory point sequence is less than or equal to a preset first threshold, and the second deviation value between the first calibration period and the second prediction period is less than or equal to a preset second threshold, it is determined that the convergence condition is satisfied.

[0155] As an example, in response to that the first deviation value between the third predicted trajectory point sequence and the second trajectory point sequence is less than or equal to a preset first threshold, and the second deviation value between the first calibration period and the second prediction period is greater than the preset second threshold, it is determined that the convergence condition is satisfied.

[0156] As an example, in response to that the first deviation value between the third predicted trajectory point sequence and the second trajectory point sequence is greater than the preset first threshold, and the second deviation value between the first calibration period and the second prediction period is less than or equal to a preset second threshold, it is determined that the convergence condition is satisfied.

[0157] Step S609: In response to the satisfaction of the convergence condition, based on the trained feature extraction layer, the trained first feature mapping layer, and the trained second feature mapping layer, obtain a touch trajectory prediction model.

[0158] For example, in response to that the first deviation value between the third predicted trajectory point sequence and the second trajectory point sequence is less than or equal to a preset first threshold, and the second deviation value between the first calibration period and the second prediction period is greater than the preset second threshold, based on the trained feature extraction layer, the trained first feature mapping layer, and the trained second feature mapping layer, obtain a touch trajectory prediction model.

[0159] In an optional implementation manner, the above method further includes: in response to the non-satisfaction of the convergence condition, based on the third predicted trajectory point sequence, the second trajectory point sequence, the second prediction period, and the first calibration period, adjust the parameters of the trained feature extraction layer, the parameters of the trained first feature mapping layer, and the parameters of the trained second feature mapping layer to obtain the feature extraction layer after re-training, the first feature mapping layer after re-training, and the second feature mapping layer after re-training; based on the third predicted trajectory point sequence and the second trajectory point sequence, determine a second calibration period; use the second calibration period as the new first calibration period, the feature extraction layer after re-training as the new trained feature extraction layer, the first feature mapping layer after re-training as the new trained first feature mapping layer, and the second feature mapping layer after re-training as the new trained second feature mapping layer.

[0160] For example, in response to the non - satisfaction of the convergence condition, based on the deviation values between the third predicted trajectory point sequence, the second trajectory point sequence, and the deviation value between the second prediction period and the first calibration period, adjust the parameters of the trained feature extraction layer, the parameters of the trained first feature mapping layer, and the parameters of the trained second feature mapping layer to obtain the feature extraction layer after retraining, the first feature mapping layer after retraining, and the second feature mapping layer after retraining; and use the second calibration period as the first calibration period in steps S605 - S607, use the feature extraction layer after retraining as the trained feature extraction layer in steps S605 - S607, use the first feature mapping layer after retraining as the trained first feature mapping layer in steps S605 - S607, use the second feature mapping layer after retraining as the trained second feature mapping layer in steps S605 - S607, and return to re - execute step S605.

[0161] Among them, in the embodiments of the present disclosure, the specific implementation manner of determining the second calibration period based on the third predicted trajectory point sequence and the second trajectory point sequence may be the same as the specific implementation manner of determining the first calibration period based on the first predicted trajectory point sequence and the second trajectory point sequence, and the present disclosure will not elaborate herein.

[0162] Among them, in the embodiments of the present disclosure, the second calibration period may be a cycle duration, or the second calibration period may be a scalar indicating the sequence position in the trajectory point sequence.

[0163] By implementing the embodiments of the present disclosure, the trained feature extraction layer, the trained first feature mapping layer, and the trained second feature mapping layer can be trained based on the first trajectory point sequence, the second trajectory point sequence, and the first calibration period until the preset conditions are met, and finally a touch trajectory prediction model can be obtained. Thereby improving the training effect and obtaining a more effective touch trajectory prediction model to perform more efficient and accurate touch trajectory prediction based on the trained touch trajectory prediction model.

[0164] Please refer to Figure 7 , Figure 7 which is a flowchart of a method for predicting a touch trajectory shown according to an exemplary embodiment. As Figure 7 shown, the method may include but is not limited to the following steps.

[0165] Step S701: Obtain a trajectory point sequence within a preset time period.

[0166] For example, obtain a trajectory point sequence composed of touch trajectory points input by a user within a preset period of time.

[0167] Among them, in the embodiments of the present disclosure, the above-mentioned trajectory point sequence includes trajectory point data of at least one touch trajectory point. The trajectory point data includes at least one of the following: trajectory point coordinates, and the pressing force corresponding to the trajectory point.

[0168] Step S702: Input the trajectory point sequence into a pre-trained touch trajectory prediction model to obtain the output data of the touch trajectory prediction model.

[0169] Among them, the above-mentioned touch trajectory prediction model is pre-trained based on any one of the embodiments of the present disclosure. The embodiments of the present disclosure do not limit this and will not elaborate further.

[0170] Step S703: Obtain a target trajectory point sequence based on the output data.

[0171] For example, separate the predicted trajectory points within a future period of time from the output data, and select a part of the trajectory points with higher accuracy from them to form a target trajectory point sequence.

[0172] By implementing the embodiments of the present disclosure, touch trajectory prediction can be performed based on the trained touch trajectory prediction model, and the touch trajectory can be predicted efficiently and accurately.

[0173] Please refer to Figure 8 , Figure 8 which is a flowchart of another touch trajectory prediction method shown according to an exemplary embodiment. As Figure 8 shown, the method may include but is not limited to the following steps.

[0174] Step S801: Obtain a trajectory point sequence within a preset time period.

[0175] In the embodiments of the present disclosure, step S801 can be implemented in any one of the embodiments of the present disclosure. The embodiments of the present disclosure do not limit this and will not elaborate further.

[0176] Step S802: Input the trajectory point sequence into the feature extraction layer through the input layer for feature extraction to obtain trajectory point features.

[0177] In the embodiments of the present disclosure, the touch trajectory prediction model includes an input layer, a feature extraction layer, a first feature mapping layer, a second feature mapping layer, a connection layer, and an output layer. The first feature mapping layer and the second feature mapping layer are respectively and independently connected to the feature extraction layer and the connection layer.

[0178] For example, input the trajectory point sequence into the input layer of the touch trajectory prediction model, so as to input the trajectory point sequence into the feature extraction layer through the input layer for feature extraction to obtain the trajectory point features output by the feature extraction layer.

[0179] Step S803: Input the trajectory point features into the first feature mapping layer for feature mapping to obtain a predicted trajectory point sequence.

[0180] For example, input the trajectory point features into the first feature mapping layer for feature mapping to obtain a predicted trajectory point sequence output by the first feature mapping layer.

[0181] Step S804: Input the trajectory point features into the second feature mapping layer for feature mapping to obtain a predicted period.

[0182] For example, input the trajectory point features into the second feature mapping layer for feature mapping to obtain a predicted period output by the second feature mapping layer.

[0183] Step S805: Concatenate the predicted trajectory point sequence and the predicted period and input them into the output layer to obtain output data.

[0184] Step S806: Obtain a target trajectory point sequence based on the output data.

[0185] In one implementation, obtaining a predicted trajectory point sequence based on the output data includes: separating the predicted trajectory point sequence and the predicted period from the output data; intercepting the target trajectory point sequence from the predicted trajectory point sequence based on the predicted period.

[0186] As an example, taking the predicted trajectory point sequence as sequence A of 10*2 and the predicted period as the scalar 6, then select [0:6] from sequence A as the target trajectory point sequence.

[0187] By implementing the embodiments of the present disclosure, the feature extraction and feature mapping of the trajectory point sequence can be performed through a pre-trained touch trajectory prediction model, so as to obtain the output data of the touch trajectory prediction model, and the touch trajectory prediction is performed based on the output data of the touch trajectory prediction model. It can efficiently and accurately predict the touch trajectory.

[0188] Please refer to Figure 9 , Figure 9 is a block diagram of a training device for a touch trajectory prediction model shown according to an exemplary embodiment. The touch trajectory prediction model includes an input layer, a feature extraction layer, a first feature mapping layer, a second feature mapping layer, a connection layer, and an output layer. The first feature mapping layer and the second feature mapping layer are respectively and independently connected to the feature extraction layer and the connection layer. As Figure 9, the device 900 includes: an acquisition module 901, configured to acquire a first trajectory point sequence in a first time period and a second trajectory point sequence in a second time period; wherein, the first time period and the second time period are consecutive time periods; a first training module 902, configured to train a feature extraction layer and a first feature mapping layer based on the first trajectory point sequence and the second trajectory point sequence, to obtain a trained feature extraction layer and a trained first feature mapping layer; a processing module 903, configured to obtain a first calibration period based on the first trajectory point sequence, the second trajectory point sequence, the trained feature extraction layer, and the trained first feature mapping layer; a second training module 904, configured to train a second feature mapping layer based on the first trajectory point sequence and the first calibration period, to obtain a trained second feature mapping layer; a third training module 905, configured to train the trained feature extraction layer, the trained first feature mapping layer, and the trained second feature mapping layer based on the first trajectory point sequence, the second trajectory point sequence, and the first calibration period, to obtain a touch trajectory prediction model.

[0189] In one implementation, the first training module 902 is specifically configured to: input the first trajectory point sequence into the feature extraction layer for feature extraction, to obtain first trajectory point features; input the first trajectory point features into the first feature mapping layer for feature mapping, to obtain a first predicted trajectory point sequence; based on the first predicted trajectory point sequence and the second trajectory point sequence, adjust the parameters of the feature extraction layer and the first feature mapping layer, to obtain a trained feature extraction layer and a trained first feature mapping layer.

[0190] In one implementation, the processing module 903 is specifically configured to: input the first trajectory point sequence into the trained feature extraction layer and the trained first feature mapping layer, to obtain a second predicted trajectory point sequence corresponding to the second time period; obtain a second trajectory point corresponding to each first trajectory point in the second trajectory sequence in the second predicted trajectory point sequence; obtain a trajectory point deviation between each first trajectory point and the corresponding second trajectory point; based on the trajectory point deviation between each first trajectory point and the corresponding second trajectory point, obtain target trajectory points from the first trajectory points; obtain the first calibration period based on the target trajectory points.

[0191] In one implementation, the second training module 904 is specifically configured to: input the first trajectory point sequence into the trained feature extraction layer for feature extraction, to obtain second trajectory point features; perform feature mapping based on the second trajectory point features and the second feature mapping layer, to obtain a first predicted period; based on the first predicted period and the first calibration period, adjust the parameters of the second feature mapping layer, to obtain a trained second feature mapping layer.

[0192] In an optional implementation, the second training module 904 is specifically configured to: input the second trajectory point feature into the trained first feature mapping layer to obtain the hidden layer feature of the trained first feature mapping layer; input the second trajectory point feature and the hidden layer feature into the second feature mapping layer for feature mapping to obtain the first prediction period.

[0193] In an implementation, the third training module 905 is specifically configured to: input the first trajectory point sequence into the trained feature extraction layer for feature extraction to obtain the third trajectory point feature; input the third trajectory point feature into the trained first feature mapping layer for feature mapping to obtain the third predicted trajectory point sequence; input the third trajectory point feature into the trained second feature mapping layer for feature mapping to obtain the second prediction period; determine whether the convergence condition is satisfied based on the third predicted trajectory point sequence, the second trajectory point sequence, the second prediction period, and the first calibration period; in response to satisfying the convergence condition, obtain the touch trajectory prediction model based on the trained feature extraction layer, the trained first feature mapping layer, and the trained second feature mapping layer.

[0194] In an optional implementation, the third training module 905 is specifically configured to: obtain a first deviation value between the third predicted trajectory point sequence and the second trajectory point sequence; obtain a second deviation value between the first calibration period and the second prediction period; determine that the convergence condition is satisfied in response to the first deviation value being less than or equal to a preset first threshold and / or the second deviation value being less than or equal to a preset second threshold.

[0195] In an optional implementation, the third training module 905 is further configured to: in response to not satisfying the convergence condition, adjust the parameters of the trained feature extraction layer, the parameters of the trained first feature mapping layer, and the parameters of the trained second feature mapping layer based on the third predicted trajectory point sequence, the second trajectory point sequence, the second prediction period, and the first calibration period to obtain the re-trained feature extraction layer, the re-trained first feature mapping layer, and the re-trained second feature mapping layer; determine a second calibration period based on the third predicted trajectory point sequence and the second trajectory point sequence; use the second calibration period as the new first calibration period, the re-trained feature extraction layer as the new trained feature extraction layer, the re-trained first feature mapping layer as the new trained first feature mapping layer, and the re-trained second feature mapping layer as the new trained second feature mapping layer, and return to execute the step of inputting the first trajectory point sequence into the trained feature extraction layer for feature extraction to obtain the third trajectory point feature.

[0196] Please refer to Figure 10 , Figure 10 is a block diagram of a touch trajectory prediction device shown according to an exemplary embodiment. As Figure 10As shown, the device 1000 includes: an acquisition module 1001, configured to acquire a sequence of trajectory points within a preset time period; a first processing module 1002, configured to input the sequence of trajectory points into a pre-trained touch trajectory prediction model to obtain output data of the touch trajectory prediction model; wherein, the touch trajectory prediction model is pre-trained based on the training device of the touch trajectory prediction model according to the first aspect; and a second processing module 1003, configured to obtain a target sequence of trajectory points based on the output data.

[0197] In an optional implementation, the touch trajectory prediction model includes an input layer, a feature extraction layer, a first feature mapping layer, a second feature mapping layer, a connection layer, and an output layer. The first feature mapping layer and the second feature mapping layer are respectively and independently connected to the feature extraction layer and the connection layer. The first processing module 1002 is specifically configured to: input the sequence of trajectory points through the input layer into the feature extraction layer for feature extraction to obtain trajectory point features; input the trajectory point features into the first feature mapping layer for feature mapping to obtain a predicted sequence of trajectory points; input the trajectory point features into the second feature mapping layer for feature mapping to obtain a predicted period; and input the predicted sequence of trajectory points and the predicted period into the output layer for splicing to obtain output data.

[0198] Optionally, the second processing module 1003 is specifically configured to: separate the predicted sequence of trajectory points and the predicted period from the output data; and intercept the target sequence of trajectory points from the predicted sequence of trajectory points based on the predicted period.

[0199] Regarding the device in the above embodiments, the specific manners in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated herein.

[0200] Figure 11 It is a block diagram of an electronic device shown according to an exemplary embodiment. The electronic device can be used in a touch trajectory prediction model training method and a touch trajectory prediction method.

[0201] Referring to Figure 11 , the electronic device 1100 may include one or more of the following components: a processing component 1102, a memory 1104, a power component 1106, a multimedia component 1108, an input / output (I / O) interface 1110, and a communication component 1112.

[0202] The processing component 1102 generally controls the overall operation of the electronic device 1100, such as operations associated with display, data communication, and recording operations. The processing component 1102 may include one or more processors 1120 to execute instructions to complete all or part of the steps of the above-described methods. Additionally, the processing component 1102 may include one or more modules to facilitate the interaction between the processing component 1102 and other components. For example, the processing component 1102 may include a multimedia module to facilitate the interaction between the multimedia component 1108 and the processing component 1102.

[0203] The memory 1104 is configured to store various types of data to support the operation of the device 1100. Examples of such data include instructions for any application or method operating on the electronic device 1100, contact data, phone book data, messages, pictures, videos, and the like. The memory 1104 may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, a magnetic disk, or an optical disk.

[0204] The power component 1106 provides power to the various components of the electronic device 1100. The power component 1106 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for the electronic device 1100.

[0205] The multimedia component 1108 includes a screen that provides an output interface between the electronic device 1100 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe actions but also detect the duration and pressure associated with the touch or swipe operation.

[0206] The input / output interface 1110 provides an interface between the processing component 1102 and a peripheral interface module, which may be a keyboard, click wheel, buttons, and the like.

[0207] The communication component 1112 is configured to facilitate communication between the electronic device 1100 and other devices in a wired or wireless manner. The electronic device 1100 can access a communication standard-based wireless network, such as WiFi, 2G, or 3G, or a combination thereof. In an exemplary embodiment, the communication component 1112 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 1112 further includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0208] In an exemplary embodiment, the electronic device 1100 can be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components for performing the above method.

[0209] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 1104 including instructions, and the above instructions can be executed by a processor 1120 of the electronic device 1100 to complete the above method. For example, the non-transitory computer-readable storage medium can be a ROM, Random Access Memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0210] Figure 12 is a block diagram of a device shown according to an exemplary embodiment. For example, the device 1200 can be provided as a server. Referring to Figure 12 , the device 1200 includes a processing component 1222, which further includes one or more processors, and memory resources represented by a memory 1232 for storing instructions executable by the processing component 1222, such as application programs. The application programs stored in the memory 1232 can include one or more modules each corresponding to a set of instructions. In addition, the processing component 1222 is configured to execute instructions to perform the steps in the above method. The device 1200 may further include a power component 1226, a network interface 1250, and an input / output interface 1258.

[0211] The present disclosure also provides a computer program product, which implements the functions of any of the above method embodiments when executed by a computer.

[0212] Other embodiments of the present invention will be readily apparent to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of the invention following the general principles of the invention and including known common general knowledge or conventional technical means in the technical field not disclosed herein. The specification and examples are only to be considered as exemplary, and the true scope and spirit of the invention are pointed out by the following claims.

[0213] It should be understood that the present invention is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present invention is only limited by the appended claims.

Claims

1. A training method for a touch trajectory prediction model, characterized in that, The touch trajectory prediction model includes an input layer, a feature extraction layer, a first feature mapping layer, a second feature mapping layer, a connection layer, and an output layer. The first feature mapping layer and the second feature mapping layer are respectively and independently connected to the feature extraction layer and the connection layer. The method includes: Obtain a first trajectory point sequence in a first time period and a second trajectory point sequence in a second time period; wherein, the first time period and the second time period are consecutive time periods; Train the feature extraction layer and the first feature mapping layer based on the first trajectory point sequence and the second trajectory point sequence to obtain a trained feature extraction layer and a trained first feature mapping layer; Obtain a first calibration period based on the first trajectory point sequence, the second trajectory point sequence, the trained feature extraction layer, and the trained first feature mapping layer; Train the second feature mapping layer based on the first trajectory point sequence and the first calibration period to obtain a trained second feature mapping layer; Train the trained feature extraction layer, the trained first feature mapping layer, and the trained second feature mapping layer based on the first trajectory point sequence, the second trajectory point sequence, and the first calibration period to obtain the touch trajectory prediction model.

2. The method according to claim 1, characterized in that, The training the feature extraction layer and the first feature mapping layer based on the first trajectory point sequence and the second trajectory point sequence to obtain a trained feature extraction layer and a trained first feature mapping layer includes: Input the first trajectory point sequence into the feature extraction layer for feature extraction to obtain first trajectory point features; Input the first trajectory point features into the first feature mapping layer for feature mapping to obtain a first predicted trajectory point sequence; Adjust the parameters of the feature extraction layer and the first feature mapping layer based on the first predicted trajectory point sequence and the second trajectory point sequence to obtain the trained feature extraction layer and the trained first feature mapping layer.

3. The method according to claim 1, characterized in that The obtaining a first calibration period based on the first trajectory point sequence, the second trajectory point sequence, the trained feature extraction layer, and the trained first feature mapping layer includes: Input the first trajectory point sequence into the trained feature extraction layer and the trained first feature mapping layer to obtain a second predicted trajectory point sequence corresponding to the second time period; Obtain the second trajectory point corresponding to each first trajectory point in the second trajectory sequence in the second predicted trajectory point sequence; Obtain the trajectory point deviation between each first trajectory point and the corresponding second trajectory point; Obtain target trajectory points from the first trajectory points based on the trajectory point deviation between each first trajectory point and the corresponding second trajectory point; Obtain the first calibration period based on the target trajectory points.

4. The method according to claim 1, wherein The training the second feature mapping layer based on the first trajectory point sequence and the first calibration period to obtain a trained second feature mapping layer includes: Input the first trajectory point sequence into the trained feature extraction layer for feature extraction to obtain second trajectory point features; Perform feature mapping based on the second trajectory point feature and the second feature mapping layer to obtain the first prediction period; Based on the first prediction period and the first calibration period, adjust the parameters of the second feature mapping layer to obtain the trained second feature mapping layer.

5. The method according to claim 4, wherein The performing feature mapping based on the second trajectory point feature and the second feature mapping layer to obtain the first prediction period includes: Input the second trajectory point feature into the trained first feature mapping layer to obtain the hidden layer feature of the trained first feature mapping layer; Input the second trajectory point feature and the hidden layer feature into the second feature mapping layer for feature mapping to obtain the first prediction period.

6. The method according to claim 1, characterized in that The training the trained feature extraction layer, the trained first feature mapping layer, and the trained second feature mapping layer based on the first trajectory point sequence, the second trajectory point sequence, and the first calibration period to obtain the touch trajectory prediction model includes: Input the first trajectory point sequence into the trained feature extraction layer for feature extraction to obtain the third trajectory point feature; Input the third trajectory point feature into the trained first feature mapping layer for feature mapping to obtain the third predicted trajectory point sequence; Input the third trajectory point feature into the trained second feature mapping layer for feature mapping to obtain the second prediction period; Based on the third predicted trajectory point sequence, the second trajectory point sequence, the second prediction period, and the first calibration period, determine whether the convergence condition is satisfied; In response to the convergence condition being satisfied, obtain the touch trajectory prediction model based on the trained feature extraction layer, the trained first feature mapping layer, and the trained second feature mapping layer.

7. The method according to claim 6, characterized in that, The determining whether the convergence condition is satisfied based on the third predicted trajectory point sequence, the second trajectory point sequence, the second prediction period, and the first calibration period includes: Obtain the first deviation value between the third predicted trajectory point sequence and the second trajectory point sequence; Obtain the second deviation value between the first calibration period and the second prediction period; In response to the first deviation value being less than or equal to a preset first threshold, and / or the second deviation value being less than or equal to a preset second threshold, determine that the convergence condition is satisfied.

8. The method according to claim 6, wherein The method further includes: In response to the convergence condition not being satisfied, based on the third predicted trajectory point sequence, the second trajectory point sequence, the second prediction period, and the first calibration period, adjust the parameters of the trained feature extraction layer, the parameters of the trained first feature mapping layer, and the parameters of the trained second feature mapping layer to obtain the re-trained feature extraction layer, the re-trained first feature mapping layer, and the re-trained second feature mapping layer; Based on the third predicted trajectory point sequence and the second trajectory point sequence, determine the second calibration period; Take the second calibration period as the new first calibration period, the re-trained feature extraction layer as the new trained feature extraction layer, the re-trained first feature mapping layer as the new trained first feature mapping layer, and the re-trained second feature mapping layer as the new trained second feature mapping layer, and return to execute the step of inputting the first trajectory point sequence into the trained feature extraction layer for feature extraction to obtain the third trajectory point feature.

9. A method for predicting a touch trajectory, characterized in that, It includes: Obtain the trajectory point sequence within a preset time period; Input the trajectory point sequence into a pre-trained touch trajectory prediction model to obtain the output data of the touch trajectory prediction model; wherein, the touch trajectory prediction model is pre-trained based on the training method of the touch trajectory prediction model according to any one of claims 1 to 8; Obtain the target trajectory point sequence based on the output data.

10. The method according to claim 9, wherein The touch trajectory prediction model includes an input layer, a feature extraction layer, a first feature mapping layer, a second feature mapping layer, a connection layer and an output layer. The first feature mapping layer and the second feature mapping layer are respectively and independently connected to the feature extraction layer and the connection layer. The step of inputting the trajectory point sequence into the pre-trained touch trajectory prediction model to obtain the output data of the touch trajectory prediction model includes: Input the trajectory point sequence into the feature extraction layer through the input layer for feature extraction to obtain trajectory point features; Input the trajectory point features into the first feature mapping layer for feature mapping to obtain a predicted trajectory point sequence; Input the trajectory point features into the second feature mapping layer for feature mapping to obtain a prediction period; Input the predicted trajectory point sequence and the prediction period into the output layer for splicing to obtain the output data.

11. The method according to claim 10, wherein The step of obtaining the predicted trajectory point sequence based on the output data includes: Separate the predicted trajectory point sequence and the prediction period from the output data; Intercept the target trajectory point sequence from the predicted trajectory point sequence based on the prediction period.

12. A training device for a touch trajectory prediction model, characterized in that, The touch trajectory prediction model includes an input layer, a feature extraction layer, a first feature mapping layer, a second feature mapping layer, a connection layer and an output layer. The first feature mapping layer and the second feature mapping layer are respectively and independently connected to the feature extraction layer and the connection layer. The device includes: An acquisition module, configured to acquire a first trajectory point sequence in a first time period and a second trajectory point sequence in a second time period; wherein, the first time period and the second time period are consecutive time periods; A first training module, configured to train the feature extraction layer and the first feature mapping layer based on the first trajectory point sequence and the second trajectory point sequence to obtain a trained feature extraction layer and a trained first feature mapping layer; A processing module, configured to obtain a first calibration period based on the first trajectory point sequence, the second trajectory point sequence, the trained feature extraction layer and the trained first feature mapping layer; A second training module, configured to train the second feature mapping layer based on the first trajectory point sequence and the first calibration period, so as to obtain a trained second feature mapping layer; A third training module, configured to train the trained feature extraction layer, the trained first feature mapping layer, and the trained second feature mapping layer based on the first trajectory point sequence, the second trajectory point sequence, and the first calibration period, so as to obtain the touch trajectory prediction model.

13. A prediction device for a touch trajectory, characterized in that, Comprising: An acquisition module, configured to acquire a trajectory point sequence within a preset time period; A first processing module, configured to input the trajectory point sequence into a pre-trained touch trajectory prediction model to obtain output data of the touch trajectory prediction model; wherein, the touch trajectory prediction model is pre-trained based on the touch trajectory prediction model training method according to any one of claims 1 to 8; A second processing module, configured to obtain a target trajectory point sequence based on the output data.

14. An electronic device, characterized in that, The device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor, so that the at least one processor can execute the method according to any one of claims 1 to 8, or so that the at least one processor can execute the method according to any one of claims 9 to 11.

15. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 8, or the computer instructions are used to cause the computer to execute the method according to any one of claims 9 to 11.

Images