Handwriting trajectory recognition method, device, mobile terminal and storage medium

Abstract

The present invention relates to the field of image processing technology, and provides a handwriting trajectory recognition method, device, mobile terminal and storage medium. Through the collected video screen, multiple key points of the hand in the video screen are obtained; when the video screen includes a preset writing gesture, the target key point is determined from the multiple key points, and the target key point represents the fingertip used for writing; then according to the preset cycle, the position of the target key point in the video screen, that is, the coordinate point, is obtained, and then the coordinate points obtained in two adjacent cycles are connected to obtain the handwriting trajectory. The handwriting trajectory in the air can be identified without relying on special equipment, which improves the convenience of handwriting input.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a handwriting trajectory recognition method, device, mobile terminal and storage medium. Background Art

[0002] In the prior art, special devices are generally required to implement handwriting input, such as touch screens or handwriting tablets. Moreover, when using these devices to handwrite, the user's hand needs to be in contact with the device. This contact-based handwriting method not only relies on special devices, but also causes inconvenience during the handwriting process, resulting in a poor user experience. Summary of the invention

[0003] In view of this, an object of the present invention is to provide a handwriting trajectory recognition method, device, mobile terminal and storage medium.

[0004] In order to achieve the above purpose, the technical solution adopted by the embodiment of the present invention is as follows:

[0005] In a first aspect, the present invention provides a handwriting trajectory recognition method, the method comprising:

[0006] According to the collected video picture, a plurality of key points of the hand in the video picture are obtained;

[0007] When the video screen includes a preset writing gesture, determining a target key point from the plurality of key points; the target key point represents a fingertip used for writing;

[0008] Obtaining coordinate points according to a preset period; the coordinate points represent the positions of the target key points in the video picture;

[0009] The coordinate points obtained in two adjacent cycles are connected to obtain a handwriting trajectory.

[0010] In an optional embodiment, after the step of obtaining the coordinate points according to the preset period, the method further includes:

[0011] According to the obtained coordinate point of the current cycle, confirm whether the first coordinate point is an inflection point; the first coordinate point is a coordinate point obtained in a cycle before the current cycle;

[0012] If so, the first coordinate point is connected to the last determined inflection point straight line to obtain a handwriting trajectory.

[0013] In an optional implementation, the step of confirming whether the first coordinate point is an inflection point based on the obtained coordinate point of the current cycle includes:

[0014] According to the obtained coordinate point of the current cycle, confirm whether the angle between the first vector and the second vector is within a preset range; the first vector is a vector starting from the second coordinate point and ending at the first coordinate point; the second vector is a vector starting from the first coordinate point and ending at the coordinate point of the current cycle; the second coordinate point is a coordinate point obtained at a period interval from the current cycle;

[0015] If so, the first coordinate point is an inflection point;

[0016] If not, the first coordinate point is not an inflection point.

[0017] In an optional embodiment, the method further comprises:

[0018] When a preset number of coordinate points are obtained within a preset radius range centered at the starting point, the acquisition of the coordinate points stops; the starting point is the coordinate point obtained in the first cycle.

[0019] In an optional embodiment, the method further comprises:

[0020] When the video screen includes a preset end gesture, the coordinate point is stopped from being obtained.

[0021] In an optional implementation manner, the step of obtaining coordinate points according to a preset period includes:

[0022] Get the pending video images of the current cycle;

[0023] Fitting the plurality of key points in the pending video picture with a preset hand model;

[0024] According to the fitting result, the position of the target key point in the to-be-determined video picture is determined, and the coordinate point corresponding to the current period is obtained.

[0025] In a second aspect, the present invention provides a handwriting trajectory recognition device, the device comprising:

[0026] An acquisition module, used for acquiring a plurality of key points of the hand in the video picture according to the collected video picture;

[0027] A processing module, configured to determine a target key point from the plurality of key points when the video screen includes a preset writing gesture; the target key point represents a fingertip used for writing;

[0028] Obtaining coordinate points according to a preset period; the coordinate points represent the positions of the target key points in the video picture;

[0029] The connection module is used to connect the coordinate points obtained in two adjacent cycles to obtain a handwriting trajectory.

[0030] In an optional embodiment, the connection module is further used for:

[0031] According to the obtained coordinate point of the current cycle, confirm whether the first coordinate point is an inflection point; the first coordinate point is a coordinate point obtained in a cycle before the current cycle;

[0032] If so, the first coordinate point is connected to the last determined inflection point straight line to obtain a handwriting trajectory.

[0033] In a third aspect, the present invention provides a mobile terminal, including a processor and a memory, wherein the memory stores a computer program, and when the processor executes the computer program, the method described in any one of the aforementioned embodiments is implemented.

[0034] In a fourth aspect, the present invention provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any one of the aforementioned embodiments.

[0035] The handwriting trajectory recognition method, device, mobile terminal and storage medium provided by the embodiments of the present invention. First, based on the collected video picture, multiple key points of the hand in the video picture are obtained; when the video picture includes a preset writing gesture, the target key point is determined from the multiple key points, and the target key point represents the fingertip used for writing; then according to the preset period, the position of the target key point in the video picture, that is, the coordinate point, is obtained, and then the coordinate points obtained in two adjacent periods are connected to obtain the handwriting trajectory. The handwriting trajectory in the air can be identified without relying on special equipment, thereby improving the convenience of handwriting input.

[0036] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0038] Figure 1 A schematic block diagram of a mobile terminal provided by an embodiment of the present invention is shown;

[0039] Figure 2 A schematic diagram showing a flow chart of a handwriting trajectory recognition method provided by an embodiment of the present invention;

[0040] Figure 3 An example diagram of a handwriting trajectory recognition method provided by an embodiment of the present invention is shown;

[0041] Figure 4 Another schematic diagram of a flow chart of a handwriting trajectory recognition method provided by an embodiment of the present invention is shown;

[0042] Figure 5 Another schematic diagram of a flow chart of a handwriting trajectory recognition method provided by an embodiment of the present invention is shown;

[0043] Figure 6 Another schematic diagram of a flow chart of a handwriting trajectory recognition method provided by an embodiment of the present invention is shown;

[0044] Figure 7 Another schematic diagram of a flow chart of a handwriting trajectory recognition method provided by an embodiment of the present invention is shown;

[0045] Figure 8 Another example diagram of the handwriting trajectory recognition method provided by an embodiment of the present invention is shown;

[0046] Fig. 9 A functional module diagram of a handwriting trajectory recognition device provided by an embodiment of the present invention is shown.

[0047] Icon: 110 - bus; 120 - processor; 130 - memory; 150 - I / O module; 170 - communication interface; 300 - handwriting trajectory recognition device; 310 - acquisition module; 330 - processing module; 350 - connection module; 370 - end module. DETAILED DESCRIPTION

[0048] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0049] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present invention.

[0050] It should be noted that relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0051] In the process of live video broadcasting, in order to enhance the interactivity between the anchor and the audience, the live broadcast room usually sets up some small games such as "You Draw, I Guess", where the anchor draws an image and the audience guesses the content of the image. The anchor can draw an image by handwriting input, but handwriting input in the prior art usually requires the use of special equipment, which brings many inconveniences. Furthermore, an embodiment of the present invention provides a handwriting trajectory recognition method, which does not rely on special equipment. By collecting video images during the live broadcast process, the trajectory of the anchor's handwriting in the air can be identified. The handwriting trajectory recognition method provided by an embodiment of the present invention will be introduced in detail below.

[0052] Please refer to Figure 1 , is a block diagram of a mobile terminal provided by an embodiment of the present invention. The mobile terminal includes a bus 110 , a processor 120 , a memory 130 , an I / O module 150 , and a communication interface 170 .

[0053] The bus 110 may be a circuit that connects the above elements to each other and transfers communications (eg, control messages) between the above elements.

[0054] The processor 120 may receive commands from the other elements described above (eg, the memory 130 , the I / O module 150 , the communication interface 170 , etc.) through the bus 110 , may interpret the received commands, and may perform calculations or data processing according to the interpreted commands.

[0055] The processor 120 may be an integrated circuit chip with signal processing capabilities. The processor 120 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.

[0056] The memory 130 may store commands or data received from the processor 120 or other elements (eg, the I / O module 150 , the communication interface 170 , etc.) or commands or data generated by the processor 120 or other elements.

[0057] The memory 130 can be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), etc.

[0058] The I / O module 150 can receive commands or data input from the user via input-output means (e.g., sensors, keyboards, touch screens, etc.), and can transmit the received commands or data to the processor 120 or the memory 130 via the bus 110. It is also used to display various information (e.g., multimedia data, text data) received, stored, and processed from the above elements, and can display videos, images, data, etc. to the user. Optionally, the I / O module 150 may also include a device for collecting images, such as a camera, etc.

[0059] The communication interface 170 may be used to communicate signaling or data with other node devices.

[0060] Understandably, Figure 1 The structure shown is only a schematic diagram of the structure of the mobile terminal. The mobile terminal may also include Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown. Figure 1 Each component shown in the figure can be implemented by hardware, software or a combination thereof.

[0061] The mobile terminal provided in the embodiment of the present invention may be a smart phone, a personal computer, a tablet computer, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), etc. The embodiment of the present invention does not impose any limitation on this.

[0062] It is understandable that an application is installed in the mobile terminal, such as an application for live broadcasting is installed in the anchor's mobile terminal. During the operation of this application, the handwriting trajectory recognition method provided by the embodiment of the present invention can be implemented.

[0063] The above-mentioned mobile terminal will be used as the execution subject to execute each step in each method provided in the embodiment of the present invention and achieve corresponding technical effects.

[0064] See also Figure 2 , Figure 2 It is a flowchart of a handwriting trajectory recognition method provided by an embodiment of the present invention.

[0065] Step S202, obtaining a plurality of key points of the hand in the video picture according to the collected video picture;

[0066] The video screen is the video screen of the anchor during the live broadcast.

[0067] Key points represent key parts of the hand, such as the palm, finger joints, finger tips, etc.

[0068] Optionally, the camera of the mobile terminal can be used to capture the host's video footage during the live broadcast in real time, and then based on the gesture recognition algorithm in the application, multiple key points of the hand in the video footage, i.e., the key parts of the host's hand, can be obtained.

[0069] like Figure 3 (a) in the figure is an example diagram of multiple key points of a hand provided by an embodiment of the present invention, which includes 21 key points, namely K0 to K20. Each key point represents a key part of the hand, and gestures can be recognized based on the relative positions of these key points. It can be understood that this example diagram is only an exemplary display of the key points of the hand, which can be set according to actual needs and is not limited by the embodiment of the present invention.

[0070] Step S204, when the video screen includes a preset writing gesture, determining a target key point from a plurality of key points;

[0071] Among them, the target key point represents the fingertip used for writing.

[0072] Optionally, the application has the function of displaying writing prompts, and the host can make a preset start gesture based on the writing prompt. When the preset start gesture is included in the video screen, handwriting trajectory recognition begins; then the host can make a preset writing gesture based on the writing prompt. When the preset writing gesture is included in the video screen, a target key point is obtained from multiple key points, and the target key point is the fingertip of the finger used for writing.

[0073] Optionally, the preset start gesture may be a five-finger spread gesture, such as Figure 3 The gesture in (b) above, the preset start gesture may also be set according to actual needs, which is not limited in the embodiment of the present invention.

[0074] The preset writing gestures can be as follows Figure 3 In the gesture of (c), according to the preset writing gesture, the distance values ​​between multiple key points and the palm key point K0 can be obtained. The maximum distance value among all the distance values ​​can be determined to determine the target key point, namely key point K8, which represents the fingertip of the index finger used for writing.

[0075] Step S206, obtaining coordinate points according to a preset period;

[0076] The coordinate point represents the position of the target key point in the video image.

[0077] Optionally, the video screen is acquired once according to a preset cycle, and the position of the target key point in the video screen is obtained according to the video screen acquired in each cycle, and the position can be represented by a coordinate point. Optionally, the coordinate points obtained in each cycle can be stored in a trajectory array.

[0078] Step S208, connecting the coordinate points obtained in two adjacent periods to obtain a handwriting trajectory.

[0079] Optionally, the coordinate points obtained in two adjacent periods are connected with a straight line to obtain the handwriting trajectory of the host in the air.

[0080] It can be seen that based on the above design, first, multiple key points of the hand in the video screen are obtained according to the collected video screen; when the video screen includes a preset writing gesture, the target key point is determined from the multiple key points, and the target key point represents the fingertip used for writing; then according to the preset cycle, the position of the target key point in the video screen, that is, the coordinate point, is obtained, and then the coordinate points obtained in two adjacent cycles are connected to obtain the handwriting trajectory. Compared with the existing technology, it can identify the handwriting trajectory in the air without relying on special equipment.

[0081] It is understandable that the coordinate points may be offset, resulting in inaccurate coordinate points. In order to obtain more accurate coordinate points, an embodiment of the present invention provides a possible implementation method. Figure 4 , is a flow chart of a handwriting trajectory recognition method provided by an embodiment of the present invention. Step S206 includes the following steps:

[0082] Step S206-1, obtaining the pending video images of the current cycle;

[0083] Optionally, according to a preset cycle, a video picture corresponding to the current cycle is obtained in each cycle, and the video picture is the to-be-determined video picture.

[0084] Step S206-3, fitting multiple key points in the to-be-determined video image with a preset hand model;

[0085] The application includes a preset hand model, which is a physical hand model obtained by using the forward dynamics of the hand; it can be understood that the preset hand model is a standard hand model.

[0086] Optionally, the pending video picture obtained in each cycle is passed through a gesture recognition algorithm to obtain multiple key points of the hand in the pending video picture, and the multiple key points are fitted with a preset hand model.

[0087] Step S206-5, determining the position of the target key point in the to-be-determined video screen according to the fitting result, and obtaining the coordinate point corresponding to the current period;

[0088] Optionally, by fitting multiple key points of the hand in the pending video screen with a preset hand model, an optimal fitting result can be obtained. Based on this optimal fitting result, the position of the target key point in the pending video screen can be obtained, and the coordinate point corresponding to the current period can be obtained.

[0089] It can be seen that the pending video picture corresponding to each period is obtained, and multiple key points in the pending video picture are fitted with the preset hand model. According to the optimal fitting result, the position of the target key point in the pending video picture can be obtained, thereby improving the accuracy of the coordinate points obtained in each period and making the obtained handwriting trajectory more stable.

[0090] Optionally, if the obtained coordinate points are directly connected by straight lines, the handwriting trajectory may not be smooth enough, and thus the present invention provides a possible implementation method. Figure 5 , is a flow chart of a handwriting trajectory recognition method provided by an embodiment of the present invention. Among them, step S208 includes the following steps:

[0091] Step S208-1, confirming whether the first coordinate point is an inflection point according to the obtained coordinate point of the current cycle;

[0092] The first coordinate point is a coordinate point obtained in a cycle before the current cycle.

[0093] It can be understood that if the coordinate points obtained in two adjacent periods are connected by straight lines, each coordinate point has two corresponding straight lines.

[0094] The inflection point means that when the angle formed by the two corresponding straight lines with the coordinate point as the center is greater than a certain preset angle value, it can be considered that the turning point between the two straight lines is large, and the coordinate point is identified as the inflection point.

[0095] Optionally, the coordinate points obtained in each cycle are stored in a trajectory array. After a coordinate point is obtained in each cycle, it can be determined whether the coordinate point obtained in the cycle before the current cycle is an inflection point.

[0096] If the first coordinate point is not an inflection point, the first coordinate point is deleted from the trajectory array;

[0097] If the first coordinate point is an inflection point, execute step S208-3.

[0098] Step S208-3, connecting the first coordinate point with the last determined inflection point straight line to obtain a handwriting trajectory;

[0099] Optionally, the inflection point determined this time, namely the first coordinate point, is connected with the inflection point determined last time by a straight line, so that the handwriting trajectory can be obtained.

[0100] Regarding the above step S208-1, an embodiment of the present invention provides a possible implementation method. Step S208-1 includes: confirming whether the angle between the first vector and the second vector is within a preset range according to the obtained coordinate point of the current cycle;

[0101] The first vector is a vector starting from the second coordinate point and ending at the first coordinate point; the second vector is a vector starting from the first coordinate point and ending at the coordinate point of the current cycle.

[0102] For example, T N is the coordinate point obtained in the current cycle, and the first coordinate point is represented by T N-1 ; then the second coordinate point is represented by T N-2 , then the first vector is expressed as The second vector is represented as

[0103] Optionally, when the number of coordinate points in the trajectory array is greater than 3, after obtaining the coordinate points in each cycle, the angle between the first vector and the second vector is calculated; and it is confirmed whether the angle is within a preset range.

[0104] If the angle is within the preset range, the first coordinate point is an inflection point; if the angle is not within the preset range, the first coordinate point is not an inflection point.

[0105] Optionally, the preset range may be 16 to 180 degrees, or may be set according to actual needs, which is not limited in the embodiment of the present invention.

[0106] It can be understood that when the angle between the first vector and the second vector is between 16 and 180 degrees, it means that the coordinate points obtained in three adjacent cycles are not on the same straight line. Then the coordinate point obtained in the cycle before the current cycle, i.e., T N-1 It's a turning point.

[0107] When the angle between the first vector and the second vector is not between 16 and 180 degrees, that is, between 0 and 15 degrees, it means that the coordinates obtained in the three adjacent cycles are on the same straight line, and the coordinate point obtained in the previous cycle of the current cycle is T N-1 Not a turning point.

[0108] It can be seen that after obtaining the coordinate points corresponding to each period, it can be determined whether the coordinate points obtained in the previous period of the current period are inflection points. If they are inflection points, they can be connected to the inflection points determined last time. By screening out inflection points and connecting them, the impact of fluctuations in key points on the handwriting trajectory can be reduced, and the obtained handwriting trajectory can be made smoother, which can better show the trend of the trajectory.

[0109] Optionally, regarding the method of ending the recognition of the handwriting trajectory, the embodiment of the present invention provides two possible implementation methods, which will be introduced below respectively.

[0110] See also Figure 6 , is a flow chart of a handwriting trajectory recognition method provided by an embodiment of the present invention. After step S208, the following steps may also be included:

[0111] Step S210A, when a preset number of coordinate points are obtained within a preset radius range centered at the starting point, stop obtaining coordinate points;

[0112] The starting point is the coordinate point obtained in the first cycle.

[0113] Optionally, the coordinate points obtained in each cycle are stored in a trajectory array. When a preset number of coordinate points are obtained in continuous cycles within a preset radius range centered on the starting point, it means that there are multiple coordinate points near the starting point. It can be considered that the current handwriting trajectory is constantly approaching the starting point, which indicates that the handwriting trajectory is a closed trajectory. The coordinate points can be stopped from being obtained, and the recognition of the handwriting trajectory is ended.

[0114] Optionally, the coordinate points within the preset radius may be coordinate points caused by hand shaking. In order to make the obtained handwriting trajectory smoother, the coordinate points within the preset radius can be deleted from the trajectory array and the coordinate points within the preset radius are not connected.

[0115] See also Figure 7, is a flow chart of another handwriting trajectory recognition method provided by an embodiment of the present invention. After step S208, the following steps may also be included:

[0116] Step S208B, when the video screen includes a preset end gesture, stop obtaining coordinate points;

[0117] Optionally, when the host completes the handwriting input, he or she may make a preset end gesture according to the writing prompt. When the preset end gesture is included in the video screen, the coordinate points are stopped to end the recognition of the handwriting trajectory.

[0118] Optionally, the preset ending gesture may be a fist gesture, such as Figure 8 When the key point of each fingertip is detected to be close to the key point of the palm in the video screen, it means that the video screen includes a preset end gesture, and the handwriting trajectory recognition ends. It can be understood that the preset end gesture can also be set according to actual needs, and the embodiment of the present invention does not limit it.

[0119] In order to execute the corresponding steps in the above embodiments and various possible methods, a method for implementing a handwriting trajectory recognition device is given below. Fig. 9 , Fig. 9 This is a functional module diagram of a handwriting trajectory recognition device 300 provided in an embodiment of the present invention. It should be noted that the basic principle and technical effects of the handwriting trajectory recognition device 300 provided in this embodiment are the same as those of the above embodiments. For the sake of brief description, for parts not mentioned in this embodiment, reference can be made to the corresponding contents in the above embodiments. The handwriting trajectory recognition device 300 includes: an acquisition module 310, a processing module 330, and a connection module 350.

[0120] An acquisition module 310 is used to obtain a plurality of key points of the hand in the video picture according to the collected video picture;

[0121] The processing module 330 is used to determine a target key point from a plurality of key points when the video screen includes a preset writing gesture; the target key point represents a fingertip used for writing;

[0122] According to a preset cycle, coordinate points are obtained; the coordinate points represent the positions of the target key points in the video screen;

[0123] The connection module 350 is used to connect the coordinate points obtained in two adjacent periods to obtain a handwriting trajectory.

[0124] Optionally, the connection module 350 is further used to confirm whether the first coordinate point is an inflection point according to the coordinate point of the current cycle obtained; the first coordinate point is a coordinate point obtained in a cycle before the current cycle;

[0125] If so, the first coordinate point is connected to the last determined inflection point straight line to obtain the handwriting trajectory.

[0126] Optionally, the connection module 350 is further used to confirm whether the angle between the first vector and the second vector is within a preset range according to the coordinate point of the current cycle obtained; the first vector is a vector starting from the second coordinate point and ending at the first coordinate point; the second vector is a vector starting from the first coordinate point and ending at the coordinate point of the current cycle; the second coordinate point is a coordinate point obtained at an interval of one cycle from the current cycle;

[0127] If so, the first coordinate point is the inflection point;

[0128] If not, the first coordinate point is not an inflection point.

[0129] Optionally, the processing module 330 is further used to obtain a pending video picture of the current cycle;

[0130] Fitting multiple key points in the pending video image with a preset hand model;

[0131] According to the fitting results, the position of the target key point in the undetermined video screen is determined, and the coordinate point corresponding to the current cycle is obtained.

[0132] Optionally, the handwriting trajectory recognition device 300 further includes a module end module 370, which is used to stop obtaining coordinate points when a preset number of coordinate points are obtained within a preset radius range centered on the starting point; the starting point is the coordinate point obtained in the first cycle.

[0133] Optionally, the ending module 370 is further configured to stop obtaining coordinate points when a preset ending gesture is included in the video picture.

[0134] The embodiment of the present invention further provides a mobile terminal, including a processor 120 and a memory 130. The memory 130 stores a computer program. When the processor executes the computer program, the method for handwriting trajectory recognition disclosed in the above embodiment is implemented.

[0135] The embodiment of the present invention further provides a storage medium on which a computer program is stored. When the computer program is executed by the processor 120, the handwriting trajectory recognition method disclosed in the embodiment of the present invention is implemented.

[0136] In summary, the handwriting trajectory recognition method, device, mobile terminal and storage medium provided by the embodiments of the present invention. Based on the collected video picture, multiple key points of the hand in the video picture can be obtained; when the video picture includes a preset writing gesture, the target key point is determined from the multiple key points, and the target key point represents the fingertip used for writing; then according to the preset period, the position of the target key point in the video picture, that is, the coordinate point, is obtained, and then the coordinate points obtained in two adjacent periods are connected to obtain the handwriting trajectory. The handwriting trajectory in the air can be identified without relying on special equipment, thereby achieving the effect of improving the convenience of handwriting input. In addition, a method for achieving smooth handwriting trajectory can be proposed to ensure the smoothness and stability of the handwriting trajectory.

[0137] In several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0138] In addition, the functional modules in the various embodiments of the present invention may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

[0139] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0140] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A handwriting trajectory recognition method, characterized in that: The method comprises: According to the collected video picture, a plurality of key points of the hand in the video picture are obtained; When the video screen includes a preset writing gesture, determining a target key point from the plurality of key points; the target key point represents a fingertip used for writing; Obtaining coordinate points according to a preset period; the coordinate points represent the positions of the target key points in the video picture; Connect the coordinate points obtained in two adjacent cycles to obtain a handwriting trajectory; The step of obtaining the coordinate points according to the preset period includes: Get the pending video images of the current cycle; Fitting the plurality of key points in the pending video picture with a preset hand model; According to the fitting result, the position of the target key point in the to-be-determined video picture is determined, and the coordinate point corresponding to the current period is obtained.

2. The method according to claim 1, characterized in that After the step of obtaining the coordinate points according to the preset period, the method further includes: According to the obtained coordinate point of the current cycle, confirm whether the first coordinate point is an inflection point; the first coordinate point is a coordinate point obtained in a cycle before the current cycle; If so, the first coordinate point is connected to the last determined inflection point straight line to obtain a handwriting trajectory.

3. The method according to claim 2, characterized in that The step of confirming whether the first coordinate point is an inflection point according to the obtained coordinate point of the current cycle includes: According to the obtained coordinate point of the current cycle, confirm whether the angle between the first vector and the second vector is within a preset range; the first vector is a vector starting from the second coordinate point and ending at the first coordinate point; the second vector is a vector starting from the first coordinate point and ending at the coordinate point of the current cycle; the second coordinate point is a coordinate point obtained at a period interval from the current cycle; If so, the first coordinate point is an inflection point; If not, the first coordinate point is not an inflection point.

4. The method according to claim 1, characterized in that The method further comprises: When a preset number of coordinate points are obtained within a preset radius range centered at the starting point, the acquisition of the coordinate points stops; the starting point is the coordinate point obtained in the first cycle.

5. The method according to claim 1, characterized in that The method further comprises: When the video screen includes a preset end gesture, the coordinate point is stopped from being obtained.

6. A handwriting trajectory recognition device, characterized in that: The device comprises: An acquisition module, used for acquiring a plurality of key points of the hand in the video picture according to the collected video picture; A processing module, configured to determine a target key point from the plurality of key points when the video screen includes a preset writing gesture; the target key point represents a fingertip used for writing; Obtaining coordinate points according to a preset period; the coordinate points represent the positions of the target key points in the video picture; A connection module, used for connecting the coordinate points obtained in two adjacent cycles to obtain a handwriting trajectory; The processing module is also used to: obtain the pending video picture of the current period; fit the multiple key points in the pending video picture with the preset hand model; determine the position of the target key point in the pending video picture according to the fitting result, and obtain the coordinate point corresponding to the current period.

7. The device according to claim 6, characterized in that The connection module is also used for: According to the obtained coordinate point of the current cycle, confirm whether the first coordinate point is an inflection point; the first coordinate point is a coordinate point obtained in a cycle before the current cycle; If so, the first coordinate point is connected to the last determined inflection point straight line to obtain a handwriting trajectory.

8. A mobile terminal, characterized in that: The method comprises a processor and a memory, wherein the memory stores a computer program, and when the processor executes the computer program, the method according to any one of claims 1 to 5 is implemented.

9. A storage medium, characterized in that: The storage medium stores a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 5.

Images