Input mode recognition method, related device and readable storage medium

Abstract

The present application discloses an input method identification method, related equipment and readable storage medium. In this scheme, after obtaining the input track generated by the user on the input method keyboard, the input box features, the historical input behavior features of the user, the operation features of the N input tracks and the handwriting probability features of the input track are first determined, and then the input method corresponding to the input track is determined based on the above features. Based on this scheme, after the user turns on the keyboard handwriting function, it is possible to identify whether the input track of the user on the keyboard in the input modes of pinyin, strokes, double pinyin, etc. is generated by key input or handwriting input.

Description

Technical Field

[0001] The present application relates to the technical field of input methods, and more specifically, to an input mode recognition method, related equipment and a readable storage medium. Background Art

[0002] With the development of input methods, various input methods have been able to integrate multiple input modes such as voice, handwriting, pinyin, strokes, and double pinyin. After turning on the keyboard handwriting function, when the user inputs on the keyboard of pinyin, strokes, double pinyin and other input modes, it is necessary to identify whether the user's input is generated by key input or handwriting input.

[0003] Currently, there is no input mode recognition method that can meet the above requirements.

[0004] Therefore, how to provide an input mode recognition method to achieve the above requirements has become a technical problem that needs to be solved urgently by those skilled in the art. Summary of the invention

[0005] In view of the above problems, the present application proposes an input mode recognition method, related devices and readable storage medium. The specific scheme is as follows:

[0006] An input mode recognition method, the method comprising:

[0007] Get the input track generated by the user on the input method keyboard;

[0008] Determine input box features, historical input behavior features of the user, operation features of N input trajectories, and handwriting probability features of the input trajectories, wherein the N input trajectories include the input trajectories and nearly N-1 historical input trajectories generated before the input trajectories; N is an integer greater than or equal to 1;

[0009] An input mode corresponding to the input trajectory is determined based on the input box feature, the historical input behavior feature of the user, the operation features of the N input trajectories, and the handwriting probability feature of the input trajectory.

[0010] Optionally, determining the input box feature includes:

[0011] Determine the attributes of the input box corresponding to the input method keyboard;

[0012] The input box feature is determined based on the attribute of the input box corresponding to the input method keyboard.

[0013] Optionally, the user's historical input behavior characteristics include:

[0014] One or more of the user's input mode history usage frequency feature, the user's historical input stroke feature, and the user's handwriting font feature.

[0015] Optionally, determining the historical usage frequency characteristics of the user's input mode includes:

[0016] Obtain the historical usage count of each input mode by the user;

[0017] Calculate the historical usage frequency of each input mode by the user based on the historical usage times of each input mode by the user;

[0018] The historical usage frequencies of various input modes by the user are combined to obtain the historical usage frequency characteristics of the input mode of the user.

[0019] Optionally, determining the historical input stroke features of the user includes:

[0020] Determining the first handwriting stroke sample data of the user from the historical input data of the user;

[0021] For each handwritten first stroke sample data, based on the handwritten first stroke sample data, determine the basic stroke features, area features, pressure features, and speed features of the handwritten first stroke sample data;

[0022] Based on the basic stroke features, area features, pressure features, and speed features of each handwritten first stroke sample data, the historical input stroke features of the user are obtained by aggregating and averaging the basic strokes.

[0023] Optionally, determining the handwriting font features of the user includes:

[0024] Determining historical keystroke features of the user based on the historical input data of the user;

[0025] Determining the handwriting font type of the user based on the historical keystroke characteristics of the user;

[0026] Based on the handwriting font type of the user, a handwriting font feature of the user is determined.

[0027] Optionally, determining the operational characteristics of the N input trajectories includes:

[0028] determining trajectory characteristics of the input trajectory;

[0029] generating an operation feature of the input trajectory based on the trajectory feature of the input trajectory;

[0030] Obtaining operation features of nearly N-1 historical input trajectories generated before the input trajectory;

[0031] The operation features of the input trajectory and the operation features of nearly N-1 input trajectories generated before the input trajectory are combined to obtain the operation features of the N input trajectories.

[0032] Optionally, determining a handwriting probability feature of the input trajectory includes:

[0033] Determining the historical handwriting input trajectory features of the user and the historical keyboard input trajectory features of the user from the historical input data of the user;

[0034] Based on the trajectory features of the user's historical handwriting input trajectory, the trajectory features of the user's historical keyboard input trajectory, and the trajectory features of the input trajectory, a handwriting probability feature of the input trajectory is determined.

[0035] Optionally, the trajectory features include:

[0036] Any one or more of velocity features, pressure features, overlap domain features, center point offset features, and time interval features from the last input.

[0037] Optionally, the operation features of the N input trajectories include an input validity dimension, a key input mode recognition result dimension, a handwriting input mode recognition result dimension, and other input mode recognition dimensions; then the generating of the operation features of the input trajectory based on the trajectory features of the input trajectory includes:

[0038] determining an input validity characteristic of the input trajectory;

[0039] The input validity feature of the input trajectory is filled into the input validity dimension, and the trajectory feature of the input trajectory is filled into the other input method recognition result dimension; the key input method recognition result dimension and the handwriting input method recognition result dimension are filled with 0 to obtain the operation feature of the input trajectory.

[0040] Optionally, after determining the input mode corresponding to the input trajectory, the method further includes:

[0041] Obtaining the on-screen result of the input trajectory;

[0042] Based on the on-screen result of the input trajectory and the input mode corresponding to the input trajectory, the operation feature of the input trajectory is updated and saved as the operation feature of the historical input trajectory when the input mode is recognized for the next input trajectory.

[0043] Optionally, the determining the input mode corresponding to the input track based on the input box feature, the user's historical input behavior feature, and the operation features of the N input tracks includes:

[0044] The input box features, the user's historical input behavior features, the operation features of the N input trajectories, and the handwriting probability features of the input trajectories are input into an input method recognition model, and the input method recognition model outputs the input method corresponding to the input trajectory. The input method recognition model is trained using the input box features corresponding to the training input trajectory, the user's historical input behavior features, the operation features of the N input trajectories, and the handwriting probability features of the input trajectory as training samples, and the input methods annotated with the training input trajectory as sample labels.

[0045] An input mode recognition device, the device comprising:

[0046] An acquisition unit, used for acquiring an input track generated by a user on an input method keyboard;

[0047] a feature determination unit, configured to determine an input box feature, a historical input behavior feature of the user, an operation feature of N input trajectories, and a handwriting probability feature of the input trajectories, wherein the N input trajectories include the input trajectories and nearly N-1 historical input trajectories generated before the input trajectories; N is an integer greater than or equal to 1;

[0048] The input mode determination unit is used to determine the input mode corresponding to the input trajectory based on the input box feature, the user's historical input behavior feature, the operation features of the N input trajectories, and the handwriting probability feature of the input trajectory.

[0049] Optionally, the feature determination unit includes: an input box feature determination unit, the input box feature determination unit being configured to:

[0050] Determine the attributes of the input box corresponding to the input method keyboard;

[0051] The input box feature is determined based on the attribute of the input box corresponding to the input method keyboard.

[0052] Optionally, the user's historical input behavior characteristics include:

[0053] One or more of the user's input mode history usage frequency feature, the user's historical input stroke feature, and the user's handwriting font feature.

[0054] Optionally, the feature determination unit includes: a user input mode history usage frequency feature determination unit, the user input mode history usage frequency feature determination unit is used to:

[0055] Obtain the historical usage count of each input mode by the user;

[0056] Calculate the historical usage frequency of each input mode by the user based on the historical usage times of each input mode by the user;

[0057] The historical usage frequencies of various input modes by the user are combined to obtain the historical usage frequency characteristics of the input mode of the user.

[0058] Optionally, the feature determination unit includes: a user's historical input stroke feature determination unit, the user's historical input stroke feature determination unit is used to:

[0059] Determining the first handwriting stroke sample data of the user from the historical input data of the user;

[0060] For each handwritten first stroke sample data, based on the handwritten first stroke sample data, determine the basic stroke features, area features, pressure features, and speed features of the handwritten first stroke sample data;

[0061] Based on the basic stroke features, area features, pressure features, and speed features of each handwritten first stroke sample data, the historical input stroke features of the user are obtained by aggregating and averaging the basic strokes.

[0062] Optionally, the feature determination unit includes: a user's handwriting font feature determination unit, the user's handwriting font feature determination unit is used to:

[0063] Determining historical keystroke features of the user based on the historical input data of the user;

[0064] Determining the handwriting font type of the user based on the historical keystroke characteristics of the user;

[0065] Based on the handwriting font type of the user, a handwriting font feature of the user is determined.

[0066] Optionally, the feature determination unit comprises: an operation feature determination unit for N input trajectories, the operation feature determination unit for N input trajectories being configured to:

[0067] determining trajectory characteristics of the input trajectory;

[0068] generating an operation feature of the input trajectory based on the trajectory feature of the input trajectory;

[0069] Obtaining operation features of nearly N-1 historical input trajectories generated before the input trajectory;

[0070] The operation features of the input trajectory and the operation features of nearly N-1 input trajectories generated before the input trajectory are combined to obtain the operation features of the N input trajectories.

[0071] Optionally, the feature determination unit includes: a handwriting probability feature determination unit for an input trajectory, the handwriting probability feature determination unit for the input trajectory being configured to:

[0072] Determining the historical handwriting input trajectory features of the user and the historical keyboard input trajectory features of the user from the historical input data of the user;

[0073] Based on the trajectory features of the user's historical handwriting input trajectory, the trajectory features of the user's historical keyboard input trajectory, and the trajectory features of the input trajectory, a handwriting probability feature of the input trajectory is determined.

[0074] Optionally, the trajectory features include:

[0075] Any one or more of velocity features, pressure features, overlap domain features, center point offset features, and time interval features from the last input.

[0076] Optionally, the operation characteristics of the N input traces include an input validity dimension, a key input mode recognition result dimension, a handwriting input mode recognition result dimension, and other input mode recognition dimensions; then the operation characteristic determination unit of the N input traces is specifically used to:

[0077] determining an input validity characteristic of the input trajectory;

[0078] The input validity feature of the input trajectory is filled into the input validity dimension, and the trajectory feature of the input trajectory is filled into the other input method recognition result dimension; the key input method recognition result dimension and the handwriting input method recognition result dimension are filled with 0 to obtain the operation feature of the input trajectory.

[0079] Optionally, the device further comprises: an operation feature updating and saving unit for the input trajectory,

[0080] The operation feature updating and saving unit of the input trajectory is used to obtain the on-screen result of the input trajectory after determining the input mode corresponding to the input trajectory; based on the on-screen result of the input trajectory and the input mode corresponding to the input trajectory, update and save the operation feature of the input trajectory as the operation feature of the historical input trajectory when the input mode is recognized for the next input trajectory.

[0081] Optionally, the input mode determination unit includes:

[0082] The input method recognition model application unit is used to input the input box features, the user's historical input behavior features, the operation features of the N input trajectories, and the handwriting probability features of the input trajectories into the input method recognition model, and the input method recognition model outputs the input method corresponding to the input trajectory. The input method recognition model is trained using the input box features corresponding to the training input trajectory, the user's historical input behavior features, the operation features of the N input trajectories, and the handwriting probability features of the input trajectory as training samples, and the input methods annotated with the training input trajectory as sample labels.

[0083] An input mode recognition device includes a memory and a processor;

[0084] The memory is used to store programs;

[0085] The processor is used to execute the program to implement the various steps of the input mode recognition method as described above.

[0086] A readable storage medium stores a computer program, and when the computer program is executed by a processor, each step of the input mode recognition method described above is implemented.

[0087] By means of the above technical solution, the present application discloses an input method identification method, related equipment and readable storage medium. In this solution, after obtaining the input track generated by the user on the input method keyboard, the input box features, the historical input behavior features of the user, the operation features of the N input tracks and the handwriting probability features of the input tracks are first determined, and then the input method corresponding to the input track is determined based on the above features. Based on this solution, after the user turns on the handwriting function of the keyboard, it is possible to identify whether the input track of the user on the keyboard in the input modes of pinyin, strokes, double pinyin, etc. is generated by key input or handwriting input. BRIEF DESCRIPTION OF THE DRAWINGS

[0088] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0089] Figure 1 A schematic diagram of the flow of the input mode recognition method disclosed in the embodiment of the present application;

[0090] Figure 2 An input mode recognition model structure disclosed in an embodiment of the present application;

[0091] Figure 3A schematic diagram of the structure of an input mode recognition device disclosed in an embodiment of the present application;

[0092] Figure 4 This is a hardware structure block diagram of an input mode recognition device disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0093] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0094] Next, the input mode recognition method provided by the present application is introduced through the following embodiments.

[0095] Reference Figure 1 , Figure 1 This is a flow chart of the input mode recognition method disclosed in the embodiment of the present application. The method may include:

[0096] Step S101: Acquire the input trajectory generated by the user on the input method keyboard.

[0097] Step S102: Determine input box features, historical input behavior features of the user, operation features of N input trajectories, and handwriting probability features of the input trajectories.

[0098] The N input trajectories include the input trajectory and nearly N-1 historical input trajectories generated before the input trajectory; N is an integer greater than or equal to 1.

[0099] In this application, the input box feature is used to characterize the attributes of the input box corresponding to the input method keyboard, the user's historical input behavior feature is used to characterize the user's historical input behavior, the operation feature of N input trajectories is used to characterize the operation behavior of the recent N input trajectories, and the handwriting probability feature of the input trajectory is used to characterize the handwriting probability of the input trajectory. The specific determination method of each feature will be described in detail through subsequent embodiments and will not be described in detail here.

[0100] Step S103: determining the input mode corresponding to the input trajectory based on the input box feature, the historical input behavior feature of the user, the operation features of the N input trajectories, and the handwriting probability feature of the input trajectory.

[0101] In the present application, after determining the input box features, the historical input behavior features of the user, the operation features of the N input trajectories, and the handwriting probability features of the input trajectories, the input mode corresponding to the input trajectories can be determined based on the input box features, the historical input behavior features of the user, the operation features of the N input trajectories, and the handwriting probability features of the input trajectories. The specific implementation method will be described in detail through subsequent embodiments and will not be described in detail here.

[0102] This embodiment discloses an input method recognition method, in which, after obtaining the input track generated by the user on the input method keyboard, the input box features, the historical input behavior features of the user, the operation features of N input tracks, and the handwriting probability features of the input track are first determined, and then the input method corresponding to the input track is determined based on the above features. Based on this method, after the user turns on the handwriting function of the keyboard, it is possible to identify whether the input track of the user on the keyboard in the input modes of pinyin, strokes, double pinyin, etc. is generated by key input or handwriting input.

[0103] In one embodiment of the present application, an implementation method for determining the input box feature is introduced, and the method may include the following steps:

[0104] Step S201: Determine the attributes of the input box corresponding to the input method keyboard.

[0105] The attributes of the input boxes of different applications are different. The input boxes of some applications (such as instant messaging apps) are chat boxes, the input boxes of some applications (such as browsers) are search boxes, and the input boxes of some applications (such as notepads) are ordinary text boxes, etc. In different applications, different attributes of the input boxes are represented by different identifiers. When a user uses an application, if there is an input requirement, he will click on the input box of the application. After the user clicks the input box, the input method keyboard pops up, and the user can perform input operations on the input method keyboard to generate an input trajectory. In this application, the input box corresponding to the input method keyboard is the attribute of the input box that triggers the input method keyboard to pop up.

[0106] Step S202: Determine the input box feature based on the attribute of the input box corresponding to the input method keyboard.

[0107] In this application, a one-hot feature can be generated as the input box feature according to the attributes of the input box corresponding to the input method keyboard and the attributes of all the preset input boxes. For ease of understanding, assuming that there are n attributes of all the preset input boxes, such as a chat box, a search box, a normal text box, etc., the input box feature is i represents the codes corresponding to different attributes of the input box. The code corresponding to the attribute of the input box corresponding to the input method keyboard is 1, and the codes corresponding to other attributes are 0. For example, the input box attribute corresponding to i1 is a chat box, and the attribute of the input box corresponding to the input method keyboard is a chat box, then the input box feature is (1, 0, ..., 0).

[0108] In one embodiment of the present application, an implementation method for determining the historical input behavior characteristics of the user is introduced, and the method may include: determining one or more of the historical usage frequency characteristics of the user's input mode, the historical input stroke characteristics of the user, and the handwriting font characteristics of the user as the historical input behavior characteristics of the user.

[0109] The handwriting font feature of the user is used to characterize the degree of connected strokes of the handwriting font of the user. As an implementable method, the degree of connected strokes can be divided into three types: no connected strokes, average connected strokes, and high connected strokes.

[0110] In one embodiment of the present application, a method for determining the historical usage frequency characteristics of the user's input mode is introduced, and the method may include the following steps:

[0111] Step S301: Obtain the historical usage times of each input mode by the user.

[0112] In this application, input mode refers to: nine-key keyboard input, twenty-six-key keyboard input, handwriting panel input, nine-key keyboard handwriting, twenty-six-key keyboard handwriting, etc.

[0113] Step S302: Calculate the historical usage frequency of each input mode by the user based on the historical usage times of each input mode by the user.

[0114] Assume that there are m input modes in total, and the number of times the user has used input mode m is num m , the number of times the user has used all input modes is num all , the number of times the user has used all input modes in history is the sum of the number of times the user has used each input mode in history, and the historical frequency of use of input mode m by the user can be calculated using the following formula:

[0115]

[0116] Step S303: The historical usage frequencies of various input modes by the user are combined to obtain the input mode historical usage frequency characteristics of the user.

[0117] Taking the above steps as an example, assuming that there are m input modes in total, the historical frequency characteristics of the user's input mode are:

[0118] In one embodiment of the present application, a method for determining the historical input stroke features of the user is introduced, and the method may include the following steps:

[0119] Step S401: Determine the first handwriting stroke sample data of the user from the historical input data of the user.

[0120] In the present application, the data selected for the screen after the user's handwriting can be screened from the user's historical input data, and for each data, an information set of the first stroke sampling point is extracted as a handwritten first stroke sample data of the user. It should be noted that if the user is a new user, the user's historical input data can be the historical input data of all current users.

[0121] It should be noted that the data selected on the upper screen after handwriting includes information of multiple sampling points. The information of each sampling point includes x-coordinate, y-coordinate, sampling point pressure, and sampling point timestamp. The data sample is as follows:

[0122] MotionEvent.x = 263.30048;

[0123] MotionEvent.y = 419.67273;

[0124] MotionEvent.pressure = 0.2;

[0125] t=1630553084000.

[0126] In the present application, the relative displacement of the x and y coordinates between each sampling point can be calculated based on the information of each sampling point in the data selected on the upper screen after the user's handwriting, and the first stroke sampling point can be determined based on the relative displacement of the x and y coordinates between each sampling point.

[0127] As an implementation method, if the relative displacements of the x and y coordinates of two adjacent sampling points in a plurality of consecutive sampling points starting from the first sampling point are both less than a set value, and the relative displacements of the x and y coordinates of a subsequent sampling point and the last sampling point in the plurality of consecutive sampling points exceed the set value, then the plurality of consecutive sampling points starting from the first sampling point are determined to be the first stroke sampling points. The information set of this part of sampling points is a handwritten first stroke sample of the user.

[0128] Step S402: for each handwritten first stroke sample data, based on the handwritten first stroke sample data, determine the basic stroke feature, area feature, pressure feature, and speed feature of the handwritten first stroke sample data.

[0129] As an implementable method, determining the basic stroke features of the handwritten first stroke sample data includes:

[0130] The coordinates of the handwritten first stroke sample data are standardized to obtain the standardized handwritten first stroke sample data; the standardized handwritten first stroke sample data is input into a Gaussian mixture model to obtain the basic stroke features of the handwritten first stroke sample data.

[0131] As an implementable method for standardizing the coordinates of the handwritten first stroke sample data, the edge of the handwritten character corresponding to the handwritten first stroke sample data can be processed, the handwritten character can be defined by a rectangular frame, and the lower left corner of the rectangular frame is used as the starting point coordinate of the handwritten first stroke, and each point coordinate uses the relative coordinate of the starting point coordinate.

[0132] As an implementable method, determining the regional features of the handwritten first stroke sample data includes:

[0133] Determine the minimum and maximum values ​​of the x-coordinates and the minimum and maximum values ​​of the y-coordinates of all sampling points in the handwritten first stroke sample data; and determine the regional features of the handwritten first stroke sample data based on the minimum and maximum values ​​of the x-coordinates and the minimum and maximum values ​​of the y-coordinates of all sampling points in the handwritten first stroke sample data.

[0134] For ease of understanding, assume that x left Indicates the minimum x-coordinate value of all sampling points in the sample data of the first stroke of a handwriting. right Indicates the maximum x-coordinate value of all sampling points in the sample data of the first stroke of a handwriting, y up Indicates the maximum value of the y coordinates of all sampling points in the sample data of the first stroke of a handwriting, y down represents the minimum value of the y coordinates of all sampling points in a handwritten first stroke sample data, then the regional feature of the handwritten first stroke sample data is X left , x right ,y up ,y down .

[0135] As an implementable embodiment, determining the pressure characteristics of the handwritten first stroke sample data includes: based on the pressure of all sampling points in the handwritten first stroke sample data, calculating the average pressure of all sampling points in the handwritten first stroke sample data as the pressure characteristics of the handwritten first stroke sample data.

[0136] As an implementation method, determining the speed feature of the handwritten first stroke sample data includes: calculating the speed between adjacent sampling points in the handwritten first stroke sample data; and calculating the average speed of all sampling points in the handwritten first stroke sample data based on the speed between each adjacent sampling point as the speed feature of the handwritten first stroke sample data. The speed between adjacent sampling points can be calculated based on the distance and time difference between adjacent sampling points.

[0137] Step S403: Based on the basic stroke features, area features, pressure features, and speed features of each handwritten first stroke sample data, the historical input stroke features of the user are obtained by aggregating and averaging the basic strokes.

[0138] In this step, the basic stroke features, area features, pressure features, and speed features of each handwritten first stroke sample data with the same basic stroke can be averaged to obtain the basic stroke features, area features, pressure features, and speed features corresponding to the basic stroke historically input by the user. The basic stroke features, area features, pressure features, and speed features corresponding to each basic stroke historically input by the user are combined to obtain the historical input stroke features of the user.

[0139] For ease of understanding, there are 48 basic strokes, and the historical input stroke features of the user can be shown in the following table:

[0140]

[0141] In one embodiment of the present application, a method for determining the handwriting font features of the user is introduced, and the method may include the following steps:

[0142] Step S501: Determine the historical keystroke features of the user based on the historical input data of the user.

[0143] In the present application, the key pressure and key speed of the user may be determined as the historical key features of the user based on the historical input data of the user.

[0144] Step S502: Determine the handwriting font type of the user based on the historical keystroke features of the user.

[0145] In the present application, the user's historical key features can be input into a trained classification model, and the classification model outputs the user's handwriting font type. The classification model is trained using the training key features as training samples and the handwriting font types annotated with the training key features as sample labels.

[0146] As an implementable embodiment, the handwriting font type may include three types: no connected strokes, normal connected strokes, and high connected strokes.

[0147] Step S503: Determine the handwriting font features of the user based on the handwriting font type of the user.

[0148] The handwriting font feature can be a three-dimensional feature, each dimension corresponds to a handwriting font type, for example, the first dimension corresponds to no connected strokes, the second dimension corresponds to normal connected strokes, and the third dimension corresponds to high connected strokes. If the user's handwriting font type is normal connected strokes, then the user's handwriting font feature is (0,1,0).

[0149] In another embodiment of the present application, an implementation method for determining the operation characteristics of N input trajectories is introduced, and the method may include the following steps:

[0150] Step S601: determining the trajectory features of the input trajectory;

[0151] In the present application, the trajectory features include any one or more of: speed features, pressure features, overlapping domain features, center point offset features, and time interval features from the last input.

[0152] As an implementable method, the input mode used by the user when generating the input trajectory on the input method keyboard can be determined, and the edge coordinates of each key on the input method keyboard under the input mode can be obtained; and the sampling point information of the input trajectory can be obtained, wherein the sampling point information includes the sampling point coordinates, the sampling pressure, and the sampling time. The sampling point information can be obtained from the client. Taking the Android client as an example, the x coordinate, y coordinate and pressure value can be obtained respectively at each sampling point through the getX(), getY(), and getPressure() methods in the MotionEvent class, and the current time is recorded at the same time. The input trajectory information is recorded as ((x1, y1, p1, t1), (x2, y2, p2, t2), ... (x n ,y n , p n , t n )).

[0153] The sampling points of the input trajectory are combined in pairs, the instantaneous speed of each group of sampling points is calculated, and then the speed characteristics of the input trajectory are calculated based on the instantaneous speed of each group of sampling points.

[0154] The coordinates of the sampling points of the input trajectory are analyzed, and the overlapping domain features are calculated in combination with the edge coordinates of each key on the input method keyboard in the input mode.

[0155] The coordinates of the sampling points of the input trajectory are analyzed, and the center point offset features are calculated in combination with the edge coordinates of each key on the input method keyboard in the input mode.

[0156] The time interval feature between the input trajectory and the last input can be determined based on the time interval between the sampling time of a specific sampling point of the current input trajectory and the sampling time of a specific sampling point of the input trajectory of the last input. The specific sampling point can be the first sampling point or the last sampling point, and this application does not impose any limitation on this.

[0157] As an implementation method, the formula S=(x max -x min )(y max -y min ) / S func Calculate the overlapping domain feature S, where x max is the maximum x-coordinate value of the input trajectory, and the meanings of other values ​​are similar. func is the key area calculated based on the edge coordinates of each key on the input method keyboard in the input mode. If the input trajectory coordinates cross multiple key areas at the same time, then S func is the sum of the areas of multiple key areas.

[0158] As an implementable method, the center point offset feature d may be calculated based on the following formula:

[0159]

[0160] d=max{Distance1, Distance2,...,Distance i}

[0161] Among them, (xcentre, ycentre) represents the coordinates of the center point of the key. If the input trajectory passes through only one key area, the straight-line distance between the center point of the input trajectory and the center point of the key is calculated as the center point offset feature. If it passes through multiple key areas, the maximum straight-line distance between the center point of the trajectory and the center points of each key is taken as the center point offset feature.

[0162] Step S602: Generate an operation feature of the input trajectory based on the trajectory feature of the input trajectory.

[0163] In the present application, the operation characteristics of the N input trajectories include an input validity dimension, a key input method recognition result dimension, a handwriting input method recognition result dimension, and other input method recognition dimensions; then the operation characteristics of the input trajectory based on the trajectory characteristics of the input trajectory are generated, including: determining the input validity characteristics of the input trajectory; filling the input validity characteristics of the input trajectory into the input validity dimension, and filling the trajectory characteristics of the input trajectory into the other input method recognition result dimension; filling the key input method recognition result dimension and the handwriting input method recognition result dimension with 0 to obtain the operation characteristics of the input trajectory.

[0164] It should be noted that, in the present application, the input validity feature of the input trajectory is a preset validity feature (eg, 0.5).

[0165] Step S603: Acquire the operation features of the nearly N-1 historical input trajectories generated before the input trajectory.

[0166] In the present application, the operation features of the nearly N-1 historical input trajectories generated before the input trajectory may be pre-stored. Among the operation features of the nearly N-1 historical input trajectories generated before the input trajectory, the input validity dimension of the operation features of each historical input trajectory is filled with input validity features. As an implementation method, if the historical input trajectory selects the upper screen after recognition, its input validity feature is determined to be 1; if the historical input trajectory does not select the upper screen after recognition, its input validity feature is determined to be 0. The input mode recognition result dimension corresponding to the input recognition result of each historical input trajectory is filled with the trajectory features of this historical input trajectory; the input mode recognition result dimension that does not correspond to this historical input recognition result is filled with 0.

[0167] Step S604: combining the operation features of the input trajectory and the operation features of the nearly N-1 input trajectories generated before the input trajectory to obtain the operation features of the N input trajectories.

[0168] For ease of understanding, assuming N = 3, the operational characteristics of N input trajectories can be shown in the following table:

[0169]

[0170]

[0171] Among them, [v, p, s, d, t] represents the trajectory features of the input trajectory or historical input trajectory.

[0172] In another embodiment of the present application, a specific method for determining the handwriting probability feature of the input trajectory is introduced, and the method may include the following steps:

[0173] Step S701: determining the historical handwriting input trajectory features of the user and the historical keyboard input trajectory features of the user from the historical input data of the user.

[0174] It should be noted that in the above embodiment, the method of generating trajectory features is explained by taking the input trajectory generated by the user on the input method keyboard as an example. The calculation method of the trajectory features of the user's historical handwriting input trajectory and the trajectory features of the user's historical keyboard input trajectory can refer to this method, which will not be repeated in this application.

[0175] Step S702: determining a handwriting probability feature of the input trajectory based on the trajectory features of the user's historical handwriting input trajectory, the trajectory features of the user's historical keyboard input trajectory, and the trajectory features of the input trajectory.

[0176] As an implementation method, the handwriting probability feature P of the input trajectory can be calculated based on the following formula: handwriting :

[0177] P handwriting =(sim(trace input , trace i ), sim(trace his , trace i ))

[0178] Among them, trace input The trace feature of the historical keyboard input trace of the user; trace his The trace feature of the user's historical handwriting input trace; i is the trajectory feature of the input trajectory.

[0179] In another embodiment of the present application, after determining the input method corresponding to the input trajectory, the method further includes: obtaining an on-screen result of the input trajectory; based on the on-screen result of the input trajectory and the input method corresponding to the input trajectory, updating and saving the operation characteristics of the input trajectory as the operation characteristics of the historical input trajectory when the input method is identified for the next input trajectory.

[0180] It should be noted that updating the operation characteristics of the input trajectory includes determining the input validity characteristics of the input trajectory based on the on-screen result of the input trajectory, and determining the filling dimension of its trajectory characteristics based on the input method corresponding to the input trajectory. The specific method can be found in the relevant introduction to the operation characteristics of the N-1 historical input trajectories, which will not be repeated here.

[0181] In another embodiment of the present application, a method for determining the input method corresponding to the input track based on the input box feature, the user's historical input behavior feature, and the operation feature of the N input tracks is introduced, and the method may include:

[0182] The input box features, the user's historical input behavior features, the operation features of the N input trajectories, and the handwriting probability features of the input trajectories are input into an input method recognition model, and the input method recognition model outputs the input method corresponding to the input trajectory. The input method recognition model is trained using the input box features corresponding to the training input trajectory, the user's historical input behavior features, the operation features of the N input trajectories, and the handwriting probability features of the input trajectory as training samples, and the input methods annotated with the training input trajectory as sample labels.

[0183] It should be noted that the input mode recognition model can adopt a variety of neural network model structures.

[0184] As an example of an implementation method, please refer to the attached Figure 2 , Figure 2 An input mode recognition model structure disclosed in the embodiment of the present application is based on Figure 2 In the input method recognition model shown, the user's historical input stroke features pass through the Embedding module, and then are combined with the operation features of N input trajectories passing through two hidden layers and input into the Attention module; the input box features, the user's input mode historical usage frequency features, the user's handwriting font features, and the handwriting probability features of the input trajectory are combined and input into the hidden layer, and finally the two-part vectors are concatenated and output as a two-dimensional prediction result Y through the softmax layer.

[0185] The input method recognition device disclosed in the embodiment of the present application is described below. The input method recognition device described below and the input method recognition method described above can be referenced to each other.

[0186] Reference Figure 3 , Figure 3 Schematic diagram of the structure of an input mode recognition device disclosed in an embodiment of the present application. Figure 3 As shown, the input mode recognition device may include:

[0187] An acquisition unit 31 is used to acquire an input track generated by a user on an input method keyboard;

[0188] A feature determination unit 32 is used to determine input box features, historical input behavior features of the user, operation features of N input trajectories, and handwriting probability features of the input trajectories, wherein the N input trajectories include the input trajectories and nearly N-1 historical input trajectories generated before the input trajectories; N is an integer greater than or equal to 1;

[0189] The input mode determination unit 33 is used to determine the input mode corresponding to the input trajectory based on the input box feature, the user's historical input behavior feature, the operation features of the N input trajectories, and the handwriting probability feature of the input trajectory.

[0190] As an implementable embodiment, the feature determination unit includes: an input box feature determination unit, the input box feature determination unit is used to:

[0191] Determine the attributes of the input box corresponding to the input method keyboard;

[0192] The input box feature is determined based on the attribute of the input box corresponding to the input method keyboard.

[0193] As an implementable method, the user's historical input behavior characteristics include:

[0194] One or more of the user's input mode history usage frequency feature, the user's historical input stroke feature, and the user's handwriting font feature.

[0195] As an implementable embodiment, the feature determination unit includes: a user input mode history usage frequency feature determination unit, the user input mode history usage frequency feature determination unit is used to:

[0196] Obtain the historical usage count of each input mode by the user;

[0197] Calculate the historical usage frequency of each input mode by the user based on the historical usage times of each input mode by the user;

[0198] The historical usage frequencies of various input modes by the user are combined to obtain the historical usage frequency characteristics of the input mode of the user.

[0199] As an implementable embodiment, the feature determination unit includes: a user's historical input stroke feature determination unit, the user's historical input stroke feature determination unit is used to:

[0200] Determining the first handwriting stroke sample data of the user from the historical input data of the user;

[0201] For each handwritten first stroke sample data, based on the handwritten first stroke sample data, determine the basic stroke features, area features, pressure features, and speed features of the handwritten first stroke sample data;

[0202] Based on the basic stroke features, area features, pressure features, and speed features of each handwritten first stroke sample data, the historical input stroke features of the user are obtained by aggregating and averaging the basic strokes.

[0203] As an implementable embodiment, the feature determination unit includes: a user's handwriting font feature determination unit, the user's handwriting font feature determination unit is used to:

[0204] Determining historical keystroke features of the user based on the historical input data of the user;

[0205] Determining the handwriting font type of the user based on the historical keystroke characteristics of the user;

[0206] Based on the handwriting font type of the user, a handwriting font feature of the user is determined.

[0207] As an implementable embodiment, the feature determination unit includes: an operation feature determination unit for N input trajectories, the operation feature determination unit for N input trajectories being used to:

[0208] determining trajectory characteristics of the input trajectory;

[0209] generating an operation feature of the input trajectory based on the trajectory feature of the input trajectory;

[0210] Obtaining operation features of nearly N-1 historical input trajectories generated before the input trajectory;

[0211] The operation features of the input trajectory and the operation features of nearly N-1 input trajectories generated before the input trajectory are combined to obtain the operation features of the N input trajectories.

[0212] As an implementation method, the feature determination unit includes: a handwriting probability feature determination unit for an input trajectory, the handwriting probability feature determination unit for an input trajectory being configured to:

[0213] Determining the historical handwriting input trajectory features of the user and the historical keyboard input trajectory features of the user from the historical input data of the user;

[0214] Based on the trajectory features of the user's historical handwriting input trajectory, the trajectory features of the user's historical keyboard input trajectory, and the trajectory features of the input trajectory, a handwriting probability feature of the input trajectory is determined.

[0215] As an implementable embodiment, the trajectory features include:

[0216] Any one or more of velocity features, pressure features, overlap domain features, center point offset features, and time interval features from the last input.

[0217] As an implementable embodiment, the operation characteristics of the N input traces include an input validity dimension, a key input mode recognition result dimension, a handwriting input mode recognition result dimension, and other input mode recognition dimensions; then the operation characteristic determination unit of the N input traces is specifically used to:

[0218] determining an input validity characteristic of the input trajectory;

[0219] The input validity feature of the input trajectory is filled into the input validity dimension, and the trajectory feature of the input trajectory is filled into the other input method recognition result dimension; the key input method recognition result dimension and the handwriting input method recognition result dimension are filled with 0 to obtain the operation feature of the input trajectory.

[0220] As an implementation method, the device further includes: an operation feature updating and saving unit for inputting the trajectory,

[0221] The operation feature updating and saving unit of the input trajectory is used to obtain the on-screen result of the input trajectory after determining the input mode corresponding to the input trajectory; based on the on-screen result of the input trajectory and the input mode corresponding to the input trajectory, update and save the operation feature of the input trajectory as the operation feature of the historical input trajectory when the input mode is recognized for the next input trajectory.

[0222] As an implementable embodiment, the input mode determination unit includes:

[0223] The input method recognition model application unit is used to input the input box features, the user's historical input behavior features, the operation features of the N input trajectories, and the handwriting probability features of the input trajectories into the input method recognition model, and the input method recognition model outputs the input method corresponding to the input trajectory. The input method recognition model is trained using the input box features corresponding to the training input trajectory, the user's historical input behavior features, the operation features of the N input trajectories, and the handwriting probability features of the input trajectory as training samples, and the input methods annotated with the training input trajectory as sample labels.

[0224] Reference Figure 4 , Figure 4 The hardware structure diagram of the input mode recognition device provided in the embodiment of the present application is shown in FIG. Figure 4 ,The hardware structure of the input mode recognition device may include: at least one processor 1, at least one communication interface 2, at least one memory 3, and at least one communication bus 4;

[0225] In the embodiment of the present application, the number of the processor 1, the communication interface 2, the memory 3, and the communication bus 4 is at least one, and the processor 1, the communication interface 2, and the memory 3 communicate with each other through the communication bus 4;

[0226] The processor 1 may be a central processing unit CPU, or an application-specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present invention, etc.;

[0227] The memory 3 may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk memory;

[0228] The memory stores a program, and the processor can call the program stored in the memory, wherein the program is used to:

[0229] Get the input track generated by the user on the input method keyboard;

[0230] Determine input box features, historical input behavior features of the user, operation features of N input trajectories, and handwriting probability features of the input trajectories, wherein the N input trajectories include the input trajectories and nearly N-1 historical input trajectories generated before the input trajectories; N is an integer greater than or equal to 1;

[0231] An input mode corresponding to the input trajectory is determined based on the input box feature, the historical input behavior feature of the user, the operation features of the N input trajectories, and the handwriting probability feature of the input trajectory.

[0232] Optionally, the detailed functions and extended functions of the program may refer to the above description.

[0233] The embodiment of the present application further provides a readable storage medium, which may store a program suitable for execution by a processor, wherein the program is used to:

[0234] Get the input track generated by the user on the input method keyboard;

[0235] Determine input box features, historical input behavior features of the user, operation features of N input trajectories, and handwriting probability features of the input trajectories, wherein the N input trajectories include the input trajectories and nearly N-1 historical input trajectories generated before the input trajectories; N is an integer greater than or equal to 1;

[0236] An input mode corresponding to the input trajectory is determined based on the input box feature, the historical input behavior feature of the user, the operation features of the N input trajectories, and the handwriting probability feature of the input trajectory.

[0237] Optionally, the detailed functions and extended functions of the program may refer to the above description.

[0238] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0239] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0240] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for identifying an input mode, characterized in that: The method comprises: Get the input track generated by the user on the input method keyboard; Determine input box features, historical input behavior features of the user, operational features of N input trajectories, and handwriting probability features of the input trajectories, wherein the N input trajectories include the input trajectories and nearly N-1 historical input trajectories generated before the input trajectories; N is an integer greater than or equal to 1; the input box features are used to characterize the attributes of the input box corresponding to the input method keyboard, and the attributes of input boxes of different applications are different; the operational features of the N input trajectories include input validity dimension, key input mode recognition result dimension, handwriting input mode recognition result dimension, and other input mode recognition result dimension; the handwriting probability features of the input trajectories are determined based on the trajectory features of the user's historical handwriting input trajectories, the trajectory features of the user's historical keyboard input trajectories, and the trajectory features of the input trajectories; Based on the input box features, the user's historical input behavior features, the operation features of the N input tracks, and the handwriting probability features of the input tracks, an input mode corresponding to the input track is determined, and the input mode is a key input mode or a handwriting input mode.

2. The method according to claim 1, characterized in that Determining the input box features includes: Determine the attributes of the input box corresponding to the input method keyboard; The input box feature is determined based on the attribute of the input box corresponding to the input method keyboard.

3. The method according to claim 1, characterized in that The user's historical input behavior characteristics include: One or more of the user's input mode history usage frequency feature, the user's historical input stroke feature, and the user's handwriting font feature.

4. The method according to claim 3, characterized in that Determining the historical frequency characteristics of the user's input mode includes: Obtain the historical usage count of each input mode by the user; Calculate the historical usage frequency of each input mode by the user based on the historical usage times of each input mode by the user; The historical usage frequencies of various input modes by the user are combined to obtain the historical usage frequency characteristics of the input mode of the user.

5. The method according to claim 3, characterized in that: Determining the historical input stroke features of the user includes: Determining the first handwriting stroke sample data of the user from the historical input data of the user; For each handwritten first stroke sample data, based on the handwritten first stroke sample data, determine the basic stroke features, area features, pressure features, and speed features of the handwritten first stroke sample data; Based on the basic stroke features, area features, pressure features, and speed features of each handwritten first stroke sample data, the historical input stroke features of the user are obtained by aggregating and averaging the basic strokes.

6. The method according to claim 5, characterized in that Determining the handwriting font characteristics of the user includes: Determining historical keystroke features of the user based on the historical input data of the user; Determining the handwriting font type of the user based on the historical keystroke characteristics of the user; Based on the handwriting font type of the user, a handwriting font feature of the user is determined.

7. The method according to claim 1, characterized in that Determine the operational characteristics of N input trajectories, including: determining trajectory characteristics of the input trajectory; generating an operation feature of the input trajectory based on the trajectory feature of the input trajectory; Obtaining operation features of nearly N-1 historical input trajectories generated before the input trajectory; The operation features of the input trajectory and the operation features of nearly N-1 input trajectories generated before the input trajectory are combined to obtain the operation features of the N input trajectories.

8. The method according to claim 7, characterized in that Determining the handwriting probability feature of the input trajectory includes: Determining the historical handwriting input trajectory features of the user and the historical keyboard input trajectory features of the user from the historical input data of the user; Based on the trajectory features of the user's historical handwriting input trajectory, the trajectory features of the user's historical keyboard input trajectory, and the trajectory features of the input trajectory, a handwriting probability feature of the input trajectory is determined.

9. The method according to claim 7 or 8, characterized in that: The trajectory features include: Any one or more of velocity features, pressure features, overlap domain features, center point offset features, and time interval features from the last input.

10. The method according to claim 7, characterized in that The step of generating the operation feature of the input trajectory based on the trajectory feature of the input trajectory includes: determining an input validity characteristic of the input trajectory; The input validity feature of the input trajectory is filled into the input validity dimension, and the trajectory feature of the input trajectory is filled into the other input method recognition result dimension; the key input method recognition result dimension and the handwriting input method recognition result dimension are filled with 0 to obtain the operation feature of the input trajectory.

11. The method according to claim 10, characterized in that After determining the input mode corresponding to the input trajectory, the method further includes: Obtaining the on-screen result of the input trajectory; Based on the on-screen result of the input trajectory and the input mode corresponding to the input trajectory, the operation feature of the input trajectory is updated and saved as the operation feature of the historical input trajectory when the input mode is recognized for the next input trajectory.

12. The method according to claim 1, characterized in that The determining the input mode corresponding to the input track based on the input box feature, the user's historical input behavior feature, and the operation features of the N input tracks includes: The input box features, the user's historical input behavior features, the operation features of the N input trajectories, and the handwriting probability features of the input trajectories are input into an input method recognition model, and the input method recognition model outputs the input method corresponding to the input trajectory. The input method recognition model is trained using the input box features corresponding to the training input trajectory, the user's historical input behavior features, the operation features of the N input trajectories, and the handwriting probability features of the input trajectory as training samples, and the input methods annotated with the training input trajectory as sample labels.

13. An input mode recognition device, characterized in that: The device comprises: An acquisition unit, used for acquiring an input track generated by a user on an input method keyboard; A feature determination unit is used to determine input box features, historical input behavior features of the user, operation features of N input trajectories, and handwriting probability features of the input trajectories, wherein the N input trajectories include the input trajectories and nearly N-1 historical input trajectories generated before the input trajectories; N is an integer greater than or equal to 1; the input box features are used to characterize the attributes of the input box corresponding to the input method keyboard, and the attributes of input boxes of different applications are different; the operation features of the N input trajectories include input validity dimension, key input mode recognition result dimension, handwriting input mode recognition result dimension, and other input mode recognition result dimension; the handwriting probability features of the input trajectories are determined based on the trajectory features of the user's historical handwriting input trajectories, the trajectory features of the user's historical keyboard input trajectories, and the trajectory features of the input trajectories; An input mode determination unit is used to determine the input mode corresponding to the input trajectory based on the input box characteristics, the user's historical input behavior characteristics, the operation characteristics of the N input trajectories, and the handwriting probability characteristics of the input trajectory, wherein the input mode is a key input mode or a handwriting input mode.

14. An input mode recognition device, characterized in that: including memory and processor; The memory is used to store programs; The processor is used to execute the program to implement each step of the input mode recognition method according to any one of claims 1 to 12.

15. A readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, each step of the input mode recognition method according to any one of claims 1 to 12 is implemented.

Images