Handwriting trajectory recognition method, device, electronic device and storage medium

Abstract

The present invention discloses a method for recognizing handwriting trajectories, which uses a feature extraction network to extract features from a trajectory coordinate vector set of handwriting trajectory data to obtain a feature vector of each trajectory coordinate; the feature vector of each trajectory coordinate is input into a gated cyclic unit to obtain a sequence encoding feature of each trajectory coordinate; the sequence encoding feature of each trajectory coordinate is decoded to obtain a trajectory formula of the handwriting trajectory data. In this way, the feature extraction network is used to extract features from the trajectory coordinate vector set, so that the global features of the handwriting trajectory data are extracted, and then the feature vector of each trajectory coordinate obtained is input into the gated cyclic unit to extract the context features of the handwriting trajectory, so that the sequence encoding features contain both global features and context features, thereby improving the matching degree between the sequence encoding features and the handwriting trajectory data, and the accuracy of the trajectory formula is also improved accordingly.

Description

Technical Field

[0001] The present invention relates to the technical field of handwriting recognition processing, and in particular to a handwriting trajectory recognition method, device and electronic equipment. Background Art

[0002] With the rapid development of Internet technology, in order to facilitate user input, multiple input methods are usually provided, such as voice input, handwriting input, and pinyin input. When the user uses handwriting input, the handwriting input data will be recognized by the handwriting recognition model to improve the recognition efficiency and provide a better user experience.

[0003] However, in the prior art, when handwritten data is recognized, it is usually based on an encoding and decoding framework. At the encoding node, a gated recurrent unit (GRU) is usually used to encode coordinate features, and a trajectory formula sequence is output at the decoding stage. Since GRU only extracts context information, the trajectory formula output by subsequent decoding is not accurate enough. Summary of the invention

[0004] The embodiments of the present invention provide a handwriting trajectory recognition method, device and electronic device, which can effectively improve the accuracy of the trajectory formula, thereby improving the accuracy of handwriting recognition.

[0005] A first aspect of an embodiment of the present invention provides a method for recognizing a handwriting trajectory, the method comprising:

[0006] Using a feature extraction network to perform feature extraction on a trajectory coordinate vector set of handwriting trajectory data, to obtain a feature vector of each trajectory coordinate in the trajectory coordinate vector set;

[0007] The feature vector of each trajectory coordinate is input into the gated recurrent unit to obtain the sequence encoding feature of each trajectory coordinate;

[0008] The sequence encoding feature of each trajectory coordinate is decoded to obtain the trajectory formula of the handwriting trajectory data.

[0009] Optionally, the step of performing feature extraction on the trajectory coordinate vector set using a feature extraction network to obtain a feature vector of each trajectory coordinate in the trajectory coordinate vector set includes:

[0010] A residual network is used to extract features from each trajectory coordinate vector in the trajectory coordinate vector set to obtain a feature vector for each trajectory coordinate.

[0011] Optionally, the residual network carries a SE module.

[0012] Optionally, the step of inputting the feature vector of each trajectory coordinate into a gated recurrent unit to obtain a sequence encoding feature of each trajectory coordinate includes:

[0013] The feature vector of each trajectory coordinate is reduced in dimension to obtain a reduced-dimensional feature vector of each trajectory coordinate;

[0014] The dimension-reduced feature vector of each trajectory coordinate is input into the gated recurrent unit to obtain a sequence encoding feature of each trajectory coordinate, wherein the sequence encoding feature includes a first direction sequence encoding feature and a second direction sequence encoding feature.

[0015] Optionally, decoding the sequence encoding feature of each trajectory coordinate to obtain the trajectory formula of the handwriting trajectory data includes:

[0016] The first direction sequence coding feature and the second direction sequence coding feature of each trajectory coordinate are fused to obtain a fusion feature of each trajectory coordinate;

[0017] Decoding is performed according to the fusion features of each trajectory coordinate to obtain the trajectory formula.

[0018] Optionally, before performing feature extraction on the trajectory coordinate vector set of the handwriting trajectory data using the feature extraction network, the method further includes:

[0019] The handwriting trajectory data is preprocessed to obtain the trajectory coordinate vector set.

[0020] Optionally, the performing data preprocessing on the handwriting trajectory data to obtain the trajectory coordinate vector set includes:

[0021] Normalizing the handwriting trajectory coordinate set of the handwriting trajectory data to obtain a target handwriting trajectory coordinate set;

[0022] Performing dimension conversion on each handwriting trajectory coordinate in the target handwriting trajectory coordinate set to obtain each handwriting trajectory coordinate after dimension conversion, and obtaining the trajectory coordinate vector set according to each handwriting trajectory coordinate after dimension conversion.

[0023] A second aspect of an embodiment of the present invention further provides a handwriting trajectory recognition device, characterized in that the device comprises:

[0024] A feature extraction unit, used to extract features from a trajectory coordinate vector set of handwriting trajectory data using a feature extraction network to obtain a feature vector of each trajectory coordinate in the trajectory coordinate vector set;

[0025] An encoding unit, used to input the feature vector of each trajectory coordinate into the gated recurrent unit to obtain a sequence encoding feature of each trajectory coordinate;

[0026] The decoding unit is used to decode the sequence encoding feature of each trajectory coordinate to obtain the trajectory formula of the handwriting trajectory data.

[0027] Optionally, the feature extraction unit is used to perform feature extraction on each trajectory coordinate vector in the trajectory coordinate vector set using a residual network to obtain a feature vector for each trajectory coordinate.

[0028] Optionally, the encoding unit is used to reduce the dimension of the feature vector of each trajectory coordinate to obtain a reduced-dimensional feature vector of each trajectory coordinate; the reduced-dimensional feature vector of each trajectory coordinate is input into the gated recurrent unit to obtain a sequence encoding feature of each trajectory coordinate, wherein the sequence encoding feature includes a first-direction sequence encoding feature and a second-direction sequence encoding feature.

[0029] A third aspect of an embodiment of the present invention provides an electronic device, comprising a memory and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors to execute operation instructions corresponding to the handwriting trajectory recognition method provided in the first aspect.

[0030] A fourth aspect of an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps corresponding to the handwriting trajectory recognition method provided in the first aspect.

[0031] The above one or at least one technical solution in the embodiments of the present application has at least the following technical effects:

[0032] Based on the above technical solution, a feature extraction network is used to perform feature extraction on a trajectory coordinate vector set of handwriting trajectory data to obtain a feature vector of each trajectory coordinate in the trajectory coordinate vector set; the feature vector of each trajectory coordinate is input into a gated recurrent unit to obtain a sequence encoding feature of each trajectory coordinate; the sequence encoding feature of each trajectory coordinate is decoded to obtain a trajectory formula of the handwriting trajectory data; in this way, a feature extraction network is first used to perform feature extraction on a trajectory coordinate vector set to extract global features of the handwriting trajectory data, and then the extracted feature vector of each trajectory coordinate is input into a gated recurrent unit to extract context features of the handwriting trajectory, thereby making the sequence encoding feature contain both global features and context features, thereby improving the matching degree between the sequence encoding feature and the handwriting trajectory data, and on the basis of improving the matching degree between the sequence encoding feature and the handwriting trajectory data, the accuracy of the trajectory formula obtained based on the decoding of the sequence encoding feature is also improved, which leads to an improvement in the accuracy of handwriting recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 A schematic diagram of a flow chart of a handwriting trajectory recognition method provided in an embodiment of the present application;

[0034] Figure 2 Schematic diagram of the structure of the residual network with SE module provided in the embodiment of the present application

[0035] Figure 3 A block diagram of a handwriting trajectory recognition device provided in an embodiment of the present application;

[0036] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0037] In the technical solution provided in the embodiment of the present application, a method for recognizing handwriting trajectories is provided. Based on the above technical solution, a feature extraction network is used to extract features from a trajectory coordinate vector set of handwriting trajectory data to obtain a feature vector of each trajectory coordinate in the trajectory coordinate vector set; the feature vector of each trajectory coordinate is input into a gated cyclic unit to obtain a sequence encoding feature of each trajectory coordinate; the sequence encoding feature of each trajectory coordinate is decoded to obtain a trajectory formula of the handwriting trajectory data. In this way, the feature extraction network is first used to extract features from the trajectory coordinate vector set so that the global features of the handwriting trajectory data are extracted, and then the feature vector of each trajectory coordinate obtained is input into a gated cyclic unit to extract the context features of the handwriting trajectory. As a result, the sequence encoding features contain both global features and context features, thereby improving the matching degree between the sequence encoding features and the handwriting trajectory data. On the basis of improving the matching degree between the sequence encoding features and the handwriting trajectory data, the accuracy of the trajectory formula obtained by decoding the sequence encoding features is also improved, which promotes the improvement of the accuracy of handwriting recognition.

[0038] The main implementation principles, specific implementation methods and corresponding beneficial effects of the technical solutions of the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0039] Example

[0040] Please refer to Figure 1 , the present application embodiment provides a method for recognizing a handwriting trajectory, the method comprising:

[0041] S101, using a feature extraction network to perform feature extraction on a trajectory coordinate vector set of handwriting trajectory data, to obtain a feature vector of each trajectory coordinate in the trajectory coordinate vector set;

[0042] S102, inputting the feature vector of each trajectory coordinate into a gated recurrent unit to obtain a sequence encoding feature of each trajectory coordinate;

[0043] S103, decoding the sequence encoding feature of each trajectory coordinate to obtain the trajectory formula of the handwriting trajectory data.

[0044] Before step S101, it is necessary to obtain the trajectory coordinate vector set of the handwriting trajectory data in advance, and after obtaining the trajectory coordinate vector set, step S101 is performed again to extract features of the trajectory coordinate vector set using a feature extraction network, thereby extracting a feature vector of each trajectory coordinate. The feature extraction network may be a residual network (resnet network), a convolutional neural network (CNN), a deep neural network (DNN), etc.

[0045] Thus, before executing step S101, when obtaining the trajectory coordinate vector set of the handwriting trajectory data, the handwriting trajectory data may be preprocessed to obtain the trajectory coordinate vector set. The data preprocessing includes at least one of normalization processing, cleaning processing and dimension conversion processing.

[0046] In one embodiment, when preprocessing handwriting trajectory data, the handwriting trajectory coordinate set of the handwriting trajectory data can be normalized first to obtain a target handwriting trajectory coordinate set; then, each handwriting trajectory coordinate in the target handwriting trajectory coordinate set can be dimensionally transformed to obtain each handwriting trajectory coordinate after dimension transformation, and a trajectory coordinate vector set can be obtained based on each handwriting trajectory coordinate after dimension transformation.

[0047] Specifically, after the handwriting trajectory coordinate set of the handwriting trajectory data is normalized, it can also be cleaned based on the distance between each two adjacent coordinates, and the remaining trajectory coordinates after the coordinate cleaning are used as the target handwriting trajectory coordinate set. For example, a set distance can be pre-set. If the distance between the two trajectory coordinates after the normalization process is less than the set distance, one of the points can be discarded until each trajectory coordinate after the normalization process is cleaned, and all the trajectory coordinates that are not lost (i.e., the remaining trajectory coordinates) are used as the target handwriting trajectory coordinate set. Among them, the set distance can be set manually or between devices, or it can be set according to actual needs, for example, it can be 0.1, 0.2, and 0.3, etc. In this way, by cleaning the trajectory coordinates after the normalization process, the number of trajectory coordinates calculated subsequently can be effectively reduced, and the accuracy of the trajectory coordinates is ensured. In this way, the amount of calculation can be reduced while ensuring the calculation accuracy to improve the calculation efficiency.

[0048] Specifically, when the dimension conversion is performed on each handwriting trajectory coordinate in the target handwriting trajectory coordinate set, the actual dimension of each handwriting trajectory coordinate after the dimension conversion can be determined according to the vector dimension supported by the feature extraction network. Of course, the actual dimension of each handwriting trajectory coordinate after the dimension conversion can also be set manually or by a device. If the feature extraction network is a resnet network, the actual dimension of each handwriting trajectory coordinate after the dimension conversion can be 8×2, 16×2, 32×2, etc.

[0049] For example, taking smartphone A as an example, the handwriting input trajectory of the user on A is obtained, and according to the handwriting input trajectory, the handwriting trajectory coordinate set corresponding to the handwriting input trajectory is obtained as M trajectory coordinate points [(x0, y0), (x1, y1), ... (xm, ym)], where m is an integer greater than 2; and then normalized according to the x and y coordinates, xmin represents the minimum value of x among the m trajectory coordinate points, ymin represents the minimum value of y among the m trajectory coordinate points, and similarly xmax and ymax represent the maximum values ​​of x and y among the m trajectory coordinate points respectively;

[0050] After obtaining m trajectory coordinate points, cleaning processing is performed. If the distance is set to 0.2, points with a distance between two trajectory coordinate points less than or equal to 0.2 need to be removed. The M trajectory coordinate points are reduced to N trajectory coordinate points, and the N trajectory coordinate points are the target handwriting trajectory coordinate set, where N is an integer greater than 1.

[0051] And, after obtaining N trajectory coordinate points, the dimension conversion is performed to convert each trajectory coordinate point into an 8×2 dimensional vector. Taking the i-th coordinate point among the N trajectory coordinate points as an example, the original coordinates of the i-th coordinate point are (x i ,y i ), the original coordinates of the i+1th coordinate point are (x i+1 ,y i+1 ), the original coordinates of the i+2th coordinate point are (x i+2 ,y i+2 ), and the coordinates of the i-th coordinate point after dimension conversion are (X i , Y i ) indicates that, where X i =[x i ,y i ,Δx i ,Δy i ,Δ 2 x i ,Δ 2 y i ,sflag1,sflag2],Y i =[y i ,x i,Δy i ,Δx i ,Δ 2 y i ,Δ 2 x i ,sflag1,sflag2], where Δx i =x i+1 -x i ,Δy i =y i+1 -y i ,Δ 2 x i =x i+2 -x i ,Δ 2 y i =y i+2 -y i , sflag1 and sflag2 are used to indicate the current handwriting state. If the current handwriting state indicates that handwriting is in progress, sflag1 is 1 and sflag2 is 0; if the current handwriting state indicates that the handwriting is stopped, sflag1 is 0 and sflag2 is 1. Of course, flag1 and sflag2 can also be set to other values, and this manual does not make specific restrictions.

[0052] In this way, after dimension conversion is performed on each of the N trajectory coordinate points, feature vectors X and Y of 8×1×N dimensions corresponding to the N trajectory coordinate points can be obtained.

[0053] And, after obtaining the trajectory coordinate vector set, step S101 is executed. In this step, a residual network can be used to extract features from each trajectory coordinate vector in the trajectory coordinate vector set to obtain a feature vector of each trajectory coordinate.

[0054] Specifically, in order to match the feature vector of each trajectory coordinate with the residual network, the convolution structure of the residual network can be modified. For example, N trajectory coordinate points correspond to 8×1×N-dimensional feature vectors X and Y. In order to make the residual network adapt to the 8×1×N-dimensional feature vector, the 3×3 convolution in the residual network can be modified to 3×1 convolution information.

[0055] In another embodiment, in order to improve the semantic information carried by the feature vector of each trajectory coordinate extracted by the feature extraction, the residual network can also carry an SE (Sequeeze-and-Excitation) module represented by SE-ResNet. The residual network with the SE module is specifically as follows: Figure 2 As shown. Among them, the input is represented by x and the output is represented by It indicates that SE-ResNet includes a Residual layer with a 3×1 convolution, a global pooling layer, two fully connected (FC) layers, and an activation function (Sigmoid).

[0056] And, after obtaining SE-ResNet, each trajectory coordinate vector in the trajectory coordinate vector set is input into SE-ResNet for feature extraction to obtain a feature vector of each trajectory coordinate. In this way, SE-ResNet performs feature extraction, so that the extracted feature vector of each trajectory coordinate also carries semantic information, so that the recognition accuracy can be effectively improved in subsequent recognition.

[0057] And, after obtaining the feature vector of each trajectory coordinate, step S102 is executed.

[0058] In step S102, the feature vector of each trajectory coordinate can be directly input into the gated recurrent unit to obtain the sequence encoding feature of each trajectory coordinate. Of course, in order to reduce the amount of calculation, the feature vector of each trajectory coordinate can also be reduced in dimension to obtain the reduced-dimensional feature vector of each trajectory coordinate; and then the reduced-dimensional feature vector of each trajectory coordinate is input into the gated recurrent unit (GRU) to obtain the sequence encoding feature of each trajectory coordinate, wherein the sequence encoding feature includes the first direction sequence encoding feature and the second direction sequence encoding feature.

[0059] Specifically, the feature vector of each trajectory coordinate is represented by X1 and Y1, where X1 contains the X-axis vector of each trajectory coordinate, and Y1 also contains the Y-axis vector of each trajectory coordinate. If the vector dimensions of X1 and Y1 can be 512×1×L, where L can be N / 4 or N / 2, etc., then X1 and Y1 are reduced in dimension so that the dimension of the reduced-dimensional feature vectors of X1 and Y1 is L×512; the reduced-dimensional feature vectors of X1 and Y1 are then input into the GRU, and the sequence encoding features of each trajectory coordinate can be represented by X2 and Y2, where X2 is the first-direction sequence encoding feature, which is used to represent the sequence encoding features of the trajectory coordinates from left to right in the X-axis direction; Y2 is the second-direction sequence encoding feature, which is used to represent the sequence encoding features of the trajectory coordinates from top to bottom in the Y-axis direction, where the encoding dimension of X2 and Y2 can be L×D, where D is the encoding dimension, which can be set manually or by the device, or can be set according to actual needs, for example, D can be 180, 200, and 220, etc.

[0060] After the sequence encoding feature of each trajectory coordinate is obtained, step S103 is executed.

[0061] In step S103, the first direction sequence coding feature and the second direction sequence coding feature of each trajectory coordinate may be fused to obtain the fused feature of each trajectory coordinate; and then decoding is performed according to the fused feature of each trajectory coordinate to obtain the trajectory formula.

[0062] Specifically, the first direction sequence coding feature and the second direction sequence coding feature can be directly combined to obtain the combined feature of each trajectory coordinate, and the combined feature of each trajectory coordinate can be directly used as the fusion feature of each trajectory coordinate. At this time, the first direction sequence coding feature and the second direction sequence coding feature can be combined by the concat function, that is, X2 and Y2 are combined to obtain the fusion feature represented by (X2, Y2), and the vector dimension of X2 and Y2 is L×2D.

[0063] Specifically, after obtaining the combined features of each trajectory coordinate, the combined features of each trajectory coordinate can also be input into the convolutional network to obtain the fusion features of each trajectory coordinate. Specifically, the combined features of each trajectory coordinate can first be input into the fully connected layer of the convolutional network to learn the weight ratio of the X and Y axis directions, and then activated by an activation function such as tanh to output the fusion features FXY in both directions, where the vector dimension of FXY is L×D. The specific formula of this process is described as:

[0064] In this way, after obtaining the fusion features of each trajectory coordinate, the fusion features of each trajectory coordinate are input into the decoding model to decode, wherein the decoding model can be an attention model, etc., and the trajectory formula corresponding to the handwriting trajectory data is decoded. In this way, the feature extraction network is first used to extract features from the trajectory coordinate vector set, so that the global features of the handwriting trajectory data are extracted, and then the feature vector of each trajectory coordinate extracted is input into the gated recurrent unit to extract the contextual features of the handwriting trajectory, so that the sequence encoding features can contain both global features and contextual features. Thereby, the matching degree between the sequence encoding features and the handwriting trajectory data is improved. On the basis of the improvement of the matching degree between the sequence encoding features and the handwriting trajectory data, the accuracy of the trajectory formula obtained by decoding based on the sequence encoding features is also improved.

[0065] In another embodiment, after the trajectory formula corresponding to the handwriting trajectory data is obtained, the target candidate words of the handwriting trajectory data may be determined according to the trajectory formula, and the target candidate words may be output.

[0066] Specifically, the trajectory formula is analyzed to obtain multiple candidate words corresponding to the trajectory formula, and according to the confidence of each candidate word in the multiple candidate words, one or more candidate words with a confidence not less than a set confidence are selected from the multiple candidate words as target candidate words, and the target candidate words are displayed on the user terminal. The confidence setting can be set manually or by the device, or can be set according to actual needs, and this manual does not make any specific restrictions.

[0067] In this way, when the trajectory formula is more accurate, by analyzing the trajectory formula, the multiple candidate words obtained will have a higher degree of match with the handwriting trajectory data, and then one or more candidate words with a higher confidence level will be selected from the multiple candidate words as target candidate words for output, further improving the match between the target candidate words and the handwriting trajectory data, which can further improve the accuracy of handwriting recognition and provide a better user experience.

[0068] The above one or at least one technical solution in the embodiments of the present application has at least the following technical effects:

[0069] Based on the above technical solution, a feature extraction network is used to perform feature extraction on a trajectory coordinate vector set of handwriting trajectory data to obtain a feature vector of each trajectory coordinate in the trajectory coordinate vector set; the feature vector of each trajectory coordinate is input into a gated recurrent unit to obtain a sequence encoding feature of each trajectory coordinate; the sequence encoding feature of each trajectory coordinate is decoded to obtain a trajectory formula of the handwriting trajectory data; in this way, a feature extraction network is first used to perform feature extraction on a trajectory coordinate vector set to extract global features of the handwriting trajectory data, and then the extracted feature vector of each trajectory coordinate is input into a gated recurrent unit to extract context features of the handwriting trajectory, thereby making the sequence encoding feature contain both global features and context features, thereby improving the matching degree between the sequence encoding feature and the handwriting trajectory data, and on the basis of improving the matching degree between the sequence encoding feature and the handwriting trajectory data, the accuracy of the trajectory formula obtained based on the decoding of the sequence encoding feature is also improved, which leads to an improvement in the accuracy of handwriting recognition.

[0070] In view of the above-mentioned embodiment, a method for recognizing a handwriting trajectory is provided. In the embodiment of the present application, a device for recognizing a handwriting trajectory is also provided. Figure 3 , the device comprises:

[0071] A feature extraction unit 301 is used to perform feature extraction on a trajectory coordinate vector set of handwriting trajectory data using a feature extraction network to obtain a feature vector of each trajectory coordinate in the trajectory coordinate vector set;

[0072] The encoding unit 302 is used to input the feature vector of each trajectory coordinate into the gated recurrent unit to obtain the sequence encoding feature of each trajectory coordinate;

[0073] The decoding unit 303 is used to decode the sequence encoding feature of each trajectory coordinate to obtain the trajectory formula of the handwriting trajectory data.

[0074] In an optional implementation, the feature extraction unit 301 is used to extract features from each trajectory coordinate vector in the trajectory coordinate vector set using a residual network to obtain a feature vector for each trajectory coordinate. The feature extraction unit 301 is used to extract features from each trajectory coordinate vector in the trajectory coordinate vector set using a residual network to obtain a feature vector for each trajectory coordinate.

[0075] In an optional implementation, the residual network carries a SE module.

[0076] In an optional embodiment, the encoding unit 302 is used to reduce the dimension of the feature vector of each trajectory coordinate to obtain a reduced-dimensional feature vector of each trajectory coordinate; the reduced-dimensional feature vector of each trajectory coordinate is input into the gated recurrent unit to obtain a sequence encoding feature of each trajectory coordinate, wherein the sequence encoding feature includes a first-direction sequence encoding feature and a second-direction sequence encoding feature.

[0077] In an optional implementation, the decoding unit 303 is used to fuse the first direction sequence coding features and the second direction sequence coding features of each trajectory coordinate to obtain the fused features of each trajectory coordinate; and decode according to the fused features of each trajectory coordinate to obtain the trajectory formula.

[0078] In an optional implementation, it also includes:

[0079] The trajectory coordinate vector acquisition unit is used to perform data preprocessing on the handwriting trajectory data before using a feature extraction network to extract features from the trajectory coordinate vector set of the handwriting trajectory data to obtain the trajectory coordinate vector set.

[0080] In an optional embodiment, the trajectory coordinate vector acquisition unit is used to normalize the handwriting trajectory coordinate set in the handwriting trajectory data to obtain a target handwriting trajectory coordinate set; perform dimension conversion on each handwriting trajectory coordinate in the target handwriting trajectory coordinate set to obtain each handwriting trajectory coordinate after dimension conversion; and obtain the trajectory coordinate vector set based on each handwriting trajectory coordinate after dimension conversion.

[0081] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0082] Figure 4 1 is a block diagram of an electronic device 800 for a handwriting trajectory recognition method according to an exemplary embodiment. For example, the electronic device 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0083] Reference Figure 4 , the electronic device 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / display (I / O) interface 812 , a sensor component 814 , and a communication component 816 .

[0084] The processing component 802 generally controls the overall operation of the electronic device 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0085] The memory 804 is configured to store various types of data to support operations on the device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0086] The power supply component 806 provides power to the various components of the electronic device 800. The power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 800.

[0087] The multimedia component 808 includes a screen that provides a display interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0088] The audio component 810 is configured to present and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), and when the electronic device 800 is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 804 or sent via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for presenting audio signals.

[0089] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.

[0090] The sensor assembly 814 includes one or more sensors for providing various aspects of status assessment for the electronic device 800. For example, the sensor assembly 814 can detect the open / closed state of the device 800, the relative positioning of components, such as the display and keypad of the electronic device 800, and the sensor assembly 814 can also detect the position change of the electronic device 800 or a component of the electronic device 800, the presence or absence of contact between the user and the electronic device 800, the orientation or acceleration / deceleration of the electronic device 800, and the temperature change of the electronic device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 814 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0091] The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0092] In an exemplary embodiment, the electronic device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.

[0093] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, and the instructions can be executed by a processor 820 of an electronic device 800 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0094] A non-transitory computer-readable storage medium, when the instructions in the storage medium are executed by a processor of a mobile terminal, enables the mobile terminal to perform a handwriting trajectory recognition method, the method comprising:

[0095] Performing data preprocessing on the handwriting trajectory data to obtain a trajectory coordinate vector set of the handwriting trajectory data;

[0096] Using a feature extraction network to perform feature extraction on a trajectory coordinate vector set of handwriting trajectory data, to obtain a feature vector of each trajectory coordinate in the trajectory coordinate vector set;

[0097] The feature vector of each trajectory coordinate is input into the gated recurrent unit to obtain the sequence encoding feature of each trajectory coordinate;

[0098] The sequence encoding feature of each trajectory coordinate is decoded to obtain the trajectory formula of the handwriting trajectory data.

[0099] Those skilled in the art will readily appreciate other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary techniques in the art that are not disclosed in this disclosure. The specification and examples are to be considered exemplary only, and the true scope and spirit of the present invention are indicated by the following claims.

[0100] It should be understood that the present invention is not limited to the precise structure described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

[0101] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for recognizing handwriting trajectories, characterized in that: The method comprises: A feature extraction network is used to extract features from the trajectory coordinate vector set of the handwriting trajectory data to obtain a feature vector for each trajectory coordinate; The feature vector of each trajectory coordinate is reduced in dimension to obtain a reduced-dimensional feature vector of each trajectory coordinate; Input the dimension reduction feature vector of each trajectory coordinate into the gated recurrent unit to obtain the first direction sequence coding feature and the second direction sequence coding feature of each trajectory coordinate, wherein the first direction sequence coding feature is used to represent the sequence coding feature of the trajectory coordinate from left to right in the X-axis direction, and the second direction sequence coding feature is used to represent the sequence coding feature of the trajectory coordinate from top to bottom in the Y-axis direction; The fusion feature of each trajectory coordinate is decoded to obtain the trajectory formula of the handwriting trajectory data, wherein the fusion feature is obtained by combining the first direction sequence coding feature and the second direction sequence coding feature.

2. The method according to claim 1, characterized in that The feature extraction network is used to extract features from the trajectory coordinate vector set to obtain a feature vector for each trajectory coordinate, including: A residual network is used to extract features of each trajectory coordinate in the trajectory coordinate vector set to obtain a feature vector of each trajectory coordinate.

3. The method according to claim 2, characterized in that The residual network carries a SE module.

4. The method according to any one of claims 1 to 3, characterized in that: Before extracting features from the trajectory coordinate vector set of the handwriting trajectory data using the feature extraction network, the method further includes: The handwriting trajectory data is preprocessed to obtain the trajectory coordinate vector set.

5. The method according to claim 4, characterized in that The step of performing data preprocessing on the handwriting trajectory data to obtain the trajectory coordinate vector set includes: Normalizing the handwriting trajectory coordinate set of the handwriting trajectory data to obtain a target handwriting trajectory coordinate set; Perform dimension conversion on each trajectory coordinate in the target handwriting trajectory coordinate set to obtain each trajectory coordinate after dimension conversion, and obtain the trajectory coordinate vector set according to each trajectory coordinate after dimension conversion.

6. A handwriting trajectory recognition device, characterized in that: The device comprises: A feature extraction unit, used for extracting features from a trajectory coordinate vector set of handwriting trajectory data using a feature extraction network to obtain a feature vector of each trajectory coordinate; The encoding unit is used to reduce the dimension of the feature vector of each trajectory coordinate to obtain the reduced dimension feature vector of each trajectory coordinate; and input the reduced dimension feature vector of each trajectory coordinate into the gated recurrent unit to obtain the first direction sequence encoding feature and the second direction sequence encoding feature of each trajectory coordinate, wherein the first direction sequence encoding feature is used to represent the sequence encoding feature of the trajectory coordinate from left to right in the X-axis direction, and the second direction sequence encoding feature is used to represent the sequence encoding feature of the trajectory coordinate from top to bottom in the Y-axis direction; A decoding unit is used to decode the fusion feature of each trajectory coordinate to obtain the trajectory formula of the handwriting trajectory data, wherein the fusion feature is obtained by combining the first direction sequence coding feature and the second direction sequence coding feature.

7. The device according to claim 6, characterized in that The feature extraction unit is used to extract features from each trajectory coordinate in the trajectory coordinate vector set using a residual network to obtain a feature vector for each trajectory coordinate.

8. The device according to claim 7, characterized in that The residual network carries a SE module.

9. The device according to any one of claims 6 to 8, characterized in that: The device also includes a pre-processing unit; The preprocessing unit is used to perform data preprocessing on the handwriting trajectory data to obtain the trajectory coordinate vector set.

10. The device according to claim 9, characterized in that The step of performing data preprocessing on the handwriting trajectory data to obtain the trajectory coordinate vector set includes: Normalizing the handwriting trajectory coordinate set of the handwriting trajectory data to obtain a target handwriting trajectory coordinate set; Perform dimension conversion on each trajectory coordinate in the target handwriting trajectory coordinate set to obtain each trajectory coordinate after dimension conversion, and obtain the trajectory coordinate vector set according to each trajectory coordinate after dimension conversion.

11. An electronic device, characterized in that: The invention comprises a memory and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors to execute the operation instructions corresponding to the method according to any one of claims 1 to 5 contained in the one or more programs.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps corresponding to the method according to any one of claims 1 to 5 are implemented.

Images