Stroke detection method and device, readable medium and electronic device

Abstract

The present disclosure relates to a stroke detection method and device, readable medium and electronic equipment. The method comprises: obtaining inputted to-be-recognized strokes in a user writing a target character, the target character being composed of at least two standard strokes; determining a similarity between the to-be-recognized strokes and a target standard stroke as a first similarity probability, the target standard stroke being at least one of the standard strokes; determining a probability that the to-be-recognized strokes are the target standard strokes as a second similarity probability according to input content before the to-be-recognized strokes; and determining whether a stroke order of writing the to-be-recognized strokes is correct according to the first similarity probability and the second similarity probability. Thus, the accuracy of stroke detection can be effectively improved, the efficiency of stroke detection can be improved, and real-time detection can be realized during user writing.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and more specifically, to a stroke order detection method, apparatus, readable medium, and electronic device. Background Technology

[0002] Currently, with the development of the internet, users are increasingly using electronic devices to perform various functions. In the process of using electronic devices, for convenience, users often use the handwriting function to write text in the input area. In some scenarios, it is necessary to judge and evaluate the user's writing, such as evaluating the quality and accuracy of the writing. One aspect of evaluating and judging writing accuracy is recognizing the stroke order of the characters. Summary of the Invention

[0003] This section is provided to briefly introduce the concepts, which will be described in detail in the Detailed Description section later. This section is not intended to identify key or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0004] Firstly, this disclosure provides a stroke order detection method, the method comprising:

[0005] The system acquires the strokes to be recognized input by the user during the writing of target characters, wherein the target characters consist of at least two standard strokes.

[0006] Determine the similarity between the stroke to be identified and the target standard stroke as the first similarity probability, wherein the target standard stroke is at least one of the standard strokes;

[0007] Based on the user's input preceding the stroke to be identified, determine the probability that the stroke to be identified is the target standard stroke, and use it as the second similarity probability;

[0008] Based on the first similarity probability and the second similarity probability, determine whether the stroke order of the stroke to be identified is correct.

[0009] Secondly, a stroke order detection device is provided, the device comprising:

[0010] The first acquisition module is used to acquire the strokes to be recognized input by the user during the writing of the target text, wherein the target text consists of at least two standard strokes;

[0011] The first determining module is used to determine the similarity between the stroke to be identified and the target standard stroke as a first similarity probability, wherein the target standard stroke is at least one of the standard strokes.

[0012] The second determining module is used to determine the probability that the stroke to be identified is the target standard stroke based on the user's input content before the stroke to be identified, as the second similarity probability;

[0013] The third determining module is used to determine whether the stroke order of the stroke to be identified is correct based on the first similarity probability and the second similarity probability.

[0014] Thirdly, a computer-readable medium is provided having a computer program stored thereon, which, when executed by a processing device, implements the steps of the method described in the first aspect of this disclosure.

[0015] Fourthly, an electronic device is provided, comprising:

[0016] A storage device having at least one computer program stored thereon;

[0017] At least one means for executing the at least one computer program in the storage device to implement the steps of the method described in the first aspect of this disclosure.

[0018] The above technical solution acquires the strokes to be recognized input by the user during the writing of target text; determines the similarity between the strokes to be recognized and the target standard strokes as the first similarity probability; determines the probability that the strokes to be recognized are the target standard strokes based on the user's input content before the strokes to be recognized, as the second similarity probability; and determines whether the stroke order leading to the strokes to be recognized is correct based on the first and second similarity probabilities. The target text consists of at least two standard strokes, and the target standard strokes are at least one of the standard strokes. Therefore, stroke order detection not only considers the similarity between strokes but also the input content before the strokes to be recognized, and utilizes the first similarity probability determined based on the former and the second similarity probability determined based on the latter for stroke order detection. This effectively improves the accuracy and efficiency of stroke order detection and enables real-time detection during the user's writing process.

[0019] Other features and advantages of this disclosure will be described in detail in the following detailed description section. Attached Figure Description

[0020] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale. In the drawings:

[0021] Figure 1 This is a flowchart of a stroke order detection method provided according to one embodiment of the present disclosure;

[0022] Figure 2 This is a block diagram of a stroke order detection device provided according to one embodiment of the present disclosure;

[0023] Figure 3 A schematic diagram of the structure of an electronic device suitable for implementing embodiments of the present disclosure is shown. Detailed Implementation

[0024] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0025] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0026] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0027] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0028] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0029] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0030] Figure 1 This is a flowchart of a stroke order detection method provided according to one embodiment of the present disclosure. Figure 1 As shown, the method may include steps 11 to 14.

[0031] In step 11, the strokes to be recognized are obtained by the user during the writing of the target text.

[0032] The target text consists of at least two standard strokes. The target text is the text that the user is currently writing or is about to write. The purpose of this disclosure is to detect whether the stroke order is correct during the user's writing of the target text. Since the purpose of this disclosure is stroke order detection, text containing only one stroke is not within the scope of this disclosure. Therefore, the target text needs to include at least two strokes, that is, the target text consists of at least two standard strokes.

[0033] The strokes to be identified can be acquired in real time during the user's writing process, or they can be acquired according to the user's writing order after the user has finished writing all the characters for the target text. This disclosure does not limit either approach. Of the two approaches, the former corresponds to the real-time stroke order detection process, while the latter corresponds to the stroke order detection after the user has finished inputting the text. Its real-time performance is slightly weaker than the former.

[0034] In some application scenarios, the strokes to be recognized can be acquired in various ways. For example, an image acquisition device can be used to photograph or film the area where the strokes to be recognized are located, obtaining an image of the user writing the strokes. Another example is that the user can write on the screen of an electronic device, and the strokes to be recognized can be acquired through the screen's sensors.

[0035] In step 12, the similarity between the stroke to be identified and the target standard stroke is determined as the first similarity probability.

[0036] The target standard stroke is at least one of the standard strokes. The target standard stroke can be all or part of the standard strokes contained in the target character.

[0037] For example, in each stroke order detection, all standard strokes of the target character can be used as target standard strokes, and their similarity can be calculated with the strokes to be identified.

[0038] For example, in each stroke order detection, since the user may have already entered other strokes before inputting the stroke to be recognized, and all previously entered strokes have already been recognized in stroke order, the standard strokes in the target text that have not yet been confirmed as entered (partial standard strokes of the target text) can be used as the target standard strokes. This can further improve the efficiency of stroke order detection.

[0039] In one possible implementation, step 12 may include the following steps:

[0040] Based on the pre-trained vector extraction model, determine the corresponding stroke vector to be identified;

[0041] Determine the target standard stroke vector corresponding to each target standard stroke;

[0042] The similarity between the stroke vector to be identified and each target standard stroke vector is calculated to obtain the first similarity probability.

[0043] For example, a vector extraction model can be obtained in the following way:

[0044] Obtain the first training sample;

[0045] The model is trained by using the first training stroke image as the input of the model and the first training stroke corresponding to the first training stroke image as the target output of the model, so as to obtain the trained classification model.

[0046] Obtain the image of the second training stroke corresponding to the second training stroke in the second training text;

[0047] The second training stroke image is input into the classification model, and the output of each intermediate layer of the classification model is extracted.

[0048] From the output content, determine the target output content with the highest similarity to the second training stroke;

[0049] The local part of the classification model, which starts with the input layer of the classification model and ends with the intermediate layer corresponding to the target output content, is defined as the vector extraction model.

[0050] The first training sample includes the first training stroke image corresponding to the first training stroke in the first training text.

[0051] In the process of training the classification model using the first training samples, the first training stroke image is used as the input to the model, and the first training stroke corresponding to the first training stroke image is used as the target output to train the model. During each training process, the loss function is calculated using the actual output and the target output of the model, and the internal parameters of the model are updated based on the loss function until the condition for stopping the model training is met, thus obtaining the completed classification model.

[0052] The trained classification model exhibits superior classification performance. This model comprises multiple intermediate layers, each outputting a vectorized result (embedding) from the input content. Furthermore, due to the different internal parameters of each intermediate layer, the features extracted from the input content vary, resulting in different feature extraction effects in the output embeddings. Therefore, further analysis of the classification model is needed to identify the intermediate layer that is most helpful for classification.

[0053] Therefore, the classification model can be analyzed as described above using the second training strokes and their corresponding images in the second training text. For example, the analysis of the classification model can be performed on the development set; that is, the second training strokes and their corresponding images can be obtained from the development set.

[0054] The second training stroke image is input into the classification model, and the output content of each intermediate layer of the classification model is extracted. From the output content, the target output content with the highest similarity to the second training stroke is determined.

[0055] For example, determining the similarity between the output content and the second training stroke can be achieved by calculating the loss value between the output content and the target output corresponding to the second training stroke.

[0056] Therefore, the similarity between the output of each intermediate layer and the second training stroke can be calculated, and the intermediate layer with the highest similarity can be selected. Thus, the local part of the classification model, which starts with the input layer of the classification model and ends with the intermediate layer corresponding to the target output content, is defined as the vector extraction model.

[0057] Optionally, after obtaining the first training sample, the method provided in this disclosure may further include the following steps:

[0058] Image enhancement is performed on the first training stroke image, and the enhanced image is associated with the first training stroke corresponding to the first training stroke image, so that the enhanced image can be used to train the classification model.

[0059] Typically, the first training stroke image obtained when acquiring the first training sample corresponds to a relatively standard-written first training stroke. Considering that the content actually written by the user may not be standard during the recognition process, in order to improve the robustness of recognition, it is also possible to expand based on the first training stroke image to increase the diversity of training data.

[0060] In other words, after obtaining the first training stroke image corresponding to the first training stroke, image enhancement can be performed on the first training stroke image, and the enhanced image can also be used as the corresponding stroke image during the training of the first training stroke for model training. For example, image enhancement can include random distortion, random tilting, random thickening, random narrowing, etc.

[0061] Based on the above description, after obtaining the vector extraction model, the stroke image of the stroke to be identified can be input into the vector extraction model to obtain the stroke vector to be identified corresponding to the stroke to be identified output by the vector extraction model.

[0062] Similarly, the target standard stroke vector for each target standard stroke can also be obtained through the vector extraction model described above.

[0063] Furthermore, the similarity between the stroke vector to be identified and each target standard stroke vector can be calculated separately to obtain the first similarity probability. For example, the similarity between the stroke vector to be identified and the target standard stroke vector can be determined by calculating Euclidean distance, pre-similarity, etc.

[0064] In step 13, based on the user's input before the stroke to be identified, the probability that the stroke to be identified is the target standard stroke is determined as the second similarity probability.

[0065] In one possible implementation, step 13 may include the following steps:

[0066] Based on the user's input before the stroke to be identified, a pre-trained probability prediction model is used to determine the probability that the stroke to be identified is the target standard stroke, which is then used as the second similarity probability.

[0067] The probabilistic prediction model can predict the probability of the current stroke corresponding to each of the preceding strokes (which could be every stroke of all strokes or every stroke among a specified number of strokes). Therefore, among the multiple probabilities predicted by the probabilistic prediction model, it is easy to locate the probability corresponding to the target standard stroke. In other words, the probabilistic prediction model uses the user's input preceding the stroke to be identified to predict the probability that the stroke to be identified is the target standard stroke, serving as a second similarity probability.

[0068] For example, a probabilistic prediction model can be obtained in the following way:

[0069] Obtain the second training sample;

[0070] The model is trained by using a subsequence of the training pen sequence as input to the language model and the next element of the subsequence in the training pen sequence as the target output of the language model, so as to obtain a trained probability prediction model.

[0071] The second training sample includes a training stroke sequence corresponding to the third training character. The training stroke sequence is composed of stroke vectors corresponding to the third training strokes contained in the third training character, and the order of the stroke vectors corresponding to the third training strokes in the training stroke sequence conforms to the standard stroke order of the third training character.

[0072] A Language Model (LM) can be understood as being able to take the current word as input and predict the next word, that is, there is a probability belonging to a certain word. The language model can model semantic information. For example, for the text "eat rice", when the word "eat" is input into the language model, the probability that the language model outputs "rice" will be very high because "eat rice" has semantic information. Considering that the stroke order of Chinese characters also has semantic information, for example, the single-person radical is very common in the training set and the order is strictly to write the left-falling stroke first and then the vertical stroke. Then, if the left-falling stroke is input currently, the probability that the language model outputs the vertical stroke will also be relatively high. Based on this principle, the language model can be introduced into stroke order detection.

[0073] That is to say, the Chinese characters in the second training sample can be disassembled into strokes according to the standard writing order. For example, the character "tian" is disassembled into {horizontal stroke, horizontal stroke, left-falling stroke, right-falling stroke}, and special characters are added before and after respectively. <s> and< / s> , where <s> Indicates the beginning,< / s> represents the end. Thus, input { <s> The character {horizontal stroke, horizontal stroke, left-falling stroke, right-falling stroke} enables the language model to predict the meaning of {horizontal stroke, horizontal stroke, left-falling stroke, right-falling stroke}.< / s>} into the language model.

[0074] According to the above principle, the prediction of stroke probability can be realized by using the language model.

[0075] It should be noted that for the training stroke order sequences in the second training sample, the stroke vectors therein can be determined by the vector extraction model provided in this disclosure.

[0076] After obtaining the second training sample, it can be used for the training of the probability prediction model. Each time during training, a subsequence of the training stroke order sequence is input into the language model, and the next element of this subsequence in the training stroke order sequence is used as the target output of the model. After obtaining the actual output of the language model, the loss function is calculated between the actual output and the target output, and the internal parameters of the language model are updated based on the loss function. Thus, when the model stop training conditions are met, the trained probability prediction model can be obtained.

[0077] Therefore, according to the trained probability prediction model, probability determination can be performed. Obtain the input content before the stroke to be recognized by the user. Among them, if the stroke to be recognized is the first stroke of the target character, the input content can be a preset starting identifier (that is, <s>)。For example, for the target character being "天", if the stroke to be recognized is the first stroke (horizontal stroke) of the character "天", the content to be input into the probability prediction model is the start identifier { <s>}, or, if the stroke to be recognized is the third stroke (left-falling stroke) of the character "天", the content to be input into the probability prediction model is { <s>, horizontal, horizontal}.

[0078] Therefore, after determining the user's input content before the stroke to be recognized, it can be input into the probability prediction model. Then, the similarity probability corresponding to the target standard stroke can be found from the output of the probability prediction model as the second similarity probability.

[0079] In step 14, the stroke order of writing to the stroke to be identified is determined based on the first similarity probability and the second similarity probability.

[0080] In one possible implementation, step 14 may include the following steps:

[0081] For each target standard stroke, the target similarity probability between the stroke to be identified and the target standard stroke is calculated based on the first similarity probability and the second similarity probability corresponding to the target standard stroke.

[0082] The target stroke with the highest target similarity probability is identified as the target stroke corresponding to the stroke to be identified.

[0083] Based on the target strokes and the standard stroke order corresponding to the target characters, determine whether the stroke order of the strokes to be identified is correct.

[0084] For example, the first similarity probability and the second similarity probability can be weighted to obtain the target similarity probability between the stroke to be identified and the target standard stroke.

[0085] Based on this, it can be confirmed what kind of stroke the stroke to be identified is (target stroke). Then, based on the identified target stroke, it can be further identified whether the stroke currently entered by the user conforms to the standard stroke order of the target text input.

[0086] The above technical solution acquires the strokes to be recognized input by the user during the writing of target text; determines the similarity between the strokes to be recognized and the target standard strokes as the first similarity probability; determines the probability that the strokes to be recognized are the target standard strokes based on the user's input content before the strokes to be recognized, as the second similarity probability; and determines whether the stroke order leading to the strokes to be recognized is correct based on the first and second similarity probabilities. The target text consists of at least two standard strokes, and the target standard strokes are at least one of the standard strokes. Therefore, stroke order detection not only considers the similarity between strokes but also the input content before the strokes to be recognized, and utilizes the first similarity probability determined based on the former and the second similarity probability determined based on the latter for stroke order detection. This effectively improves the accuracy and efficiency of stroke order detection and enables real-time detection during the user's writing process.

[0087] Figure 2 This is a block diagram of a stroke order detection device according to one embodiment of the present disclosure. Figure 2 As shown, the device 20 may include:

[0088] The first acquisition module 21 is used to acquire the strokes to be recognized input by the user during the writing of the target text, wherein the target text is composed of at least two standard strokes;

[0089] The first determining module 22 is used to determine the similarity between the stroke to be identified and the target standard stroke as a first similarity probability, wherein the target standard stroke is at least one of the standard strokes.

[0090] The second determining module 23 is used to determine the probability that the stroke to be identified is the target standard stroke based on the user's input content before the stroke to be identified, as a second similarity probability;

[0091] The third determining module 24 is used to determine whether the stroke order of the stroke to be identified is correct based on the first similarity probability and the second similarity probability.

[0092] Optionally, the first determining module 22 includes:

[0093] The first determining submodule is used to determine the stroke vector to be identified corresponding to the stroke to be identified based on a pre-trained vector extraction model.

[0094] The second determining submodule is used to determine the target standard stroke vector corresponding to each of the target standard strokes.

[0095] The third determining submodule is used to calculate the similarity between the stroke vector to be identified and each of the target standard stroke vectors to obtain the first similarity probability.

[0096] Optionally, the vector extraction model is obtained through the following modules:

[0097] The second acquisition module is used to acquire the first training sample, the first training sample including the first training stroke image corresponding to the first training stroke in the first training text.

[0098] The first training module is used to train the model by taking the first training stroke image as the input of the model and taking the first training stroke corresponding to the first training stroke image as the target output of the model, so as to obtain a trained classification model. The classification model includes multiple intermediate layers, and each intermediate layer is used to output a vectorized processing result for the content input to the classification model.

[0099] The third acquisition module is used to acquire the image of the second training stroke corresponding to the second training stroke in the second training text.

[0100] An extraction module is used to input the second training stroke image into the classification model and extract the output content of each intermediate layer of the classification model;

[0101] The fourth determining module is used to determine, from the output content, the target output content with the highest similarity to the second training stroke;

[0102] The fifth determining module is used to determine a portion of the classification model that starts with the input layer of the classification model and ends with the intermediate layer corresponding to the target output content as the vector extraction model.

[0103] Optionally, after the second acquisition module acquires the first training sample, the device 20 further includes:

[0104] An image enhancement module is used to enhance the first training stroke image and associate the enhanced image with the first training stroke corresponding to the first training stroke image, so as to use the enhanced image to train a classification model.

[0105] Optionally, the second determining module 23 is used to determine the probability that the stroke to be identified is the target standard stroke based on the user's input content before the stroke to be identified, using a pre-trained probability prediction model, as a second similarity probability;

[0106] The probability prediction model is obtained through the following modules:

[0107] The fourth acquisition module is used to acquire a second training sample. The second training sample includes a training stroke sequence corresponding to the third training character. The training stroke sequence is composed of stroke vectors corresponding to the third training strokes contained in the third training character, and the order of the stroke vectors corresponding to the third training strokes in the training stroke sequence conforms to the standard stroke order of the third training character.

[0108] The second training module is used to train the model by taking a subsequence of the training pen sequence as input to the language model and taking the next element of the subsequence in the training pen sequence as the target output of the language model, so as to obtain a trained probability prediction model.

[0109] Optionally, the second determining module 23 includes:

[0110] The calculation submodule is used to calculate the target similarity probability between the stroke to be identified and the target standard stroke for each target standard stroke, based on the first similarity probability and the second similarity probability corresponding to the target standard stroke.

[0111] The fourth determination submodule is used to determine the target standard stroke with the highest target similarity probability as the target stroke corresponding to the stroke to be identified;

[0112] The fifth determination submodule is used to determine whether the stroke order of the target stroke is correct based on the target stroke and the standard stroke order corresponding to the target character.

[0113] Optionally, the calculation submodule is used to perform a weighted calculation on the first similarity probability and the second similarity probability to obtain the target similarity probability between the stroke to be identified and the target standard stroke.

[0114] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0115] The following is for reference. Figure 3 This diagram illustrates a structural schematic of an electronic device 600 suitable for implementing embodiments of the present disclosure. The terminal devices in the embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 3 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0116] like Figure 3 As shown, electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 601, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 602 or a program loaded from storage device 608 into random access memory (RAM) 603. RAM 603 also stores various programs and data required for the operation of electronic device 600. Processing device 601, ROM 602, and RAM 603 are interconnected via bus 604. Input / output (I / O) interface 605 is also connected to bus 604.

[0117] Typically, the following devices can be connected to I / O interface 605: input devices 606 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 607 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 608 including, for example, magnetic tapes, hard disks, etc.; and communication devices 609. Communication device 609 allows electronic device 600 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 3 An electronic device 600 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0118] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 609, or installed from a storage device 608, or installed from a ROM 602. When the computer program is executed by the processing device 601, it performs the functions defined in the methods of embodiments of this disclosure.

[0119] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0120] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol, such as HTTP (Hypertext Transfer Protocol), and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks ("LANs"), wide area networks ("WANs"), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0121] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0122] The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to: acquire a stroke to be recognized input by a user during the writing of target text, wherein the target text consists of at least two standard strokes; determine the similarity between the stroke to be recognized and the target standard strokes as a first similarity probability, wherein the target standard stroke is at least one of the standard strokes; determine the probability that the stroke to be recognized is the target standard stroke based on the user's input preceding the stroke to be recognized, as a second similarity probability; and determine whether the stroke order leading to the stroke to be recognized is correct based on the first similarity probability and the second similarity probability.

[0123] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0124] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0125] The modules described in the embodiments of this disclosure can be implemented in software or in hardware. The names of the modules do not necessarily limit the module itself; for example, the first acquisition module can also be described as "a module for acquiring the strokes to be recognized input by the user during the writing of target text".

[0126] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0127] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0128] According to one or more embodiments of this disclosure, a stroke order detection method is provided, the method comprising:

[0129] The system acquires the strokes to be recognized input by the user during the writing of target characters, wherein the target characters consist of at least two standard strokes.

[0130] Determine the similarity between the stroke to be identified and the target standard stroke as the first similarity probability, wherein the target standard stroke is at least one of the standard strokes;

[0131] Based on the user's input preceding the stroke to be identified, determine the probability that the stroke to be identified is the target standard stroke, and use it as the second similarity probability;

[0132] Based on the first similarity probability and the second similarity probability, determine whether the stroke order of the stroke to be identified is correct.

[0133] According to one or more embodiments of this disclosure, a stroke order detection method is provided, wherein determining the similarity between the stroke to be identified and a target standard stroke, as a first similarity probability, includes:

[0134] Based on the pre-trained vector extraction model, determine the vector of the stroke to be identified;

[0135] Determine the target standard stroke vector corresponding to each of the aforementioned target standard strokes;

[0136] The similarity between the stroke vector to be identified and each of the target standard stroke vectors is calculated to obtain the first similarity probability.

[0137] According to one or more embodiments of this disclosure, a stroke order detection method is provided, wherein the vector extraction model is obtained in the following manner:

[0138] Obtain a first training sample, which includes the first training stroke image corresponding to the first training stroke in the first training text;

[0139] The model is trained by using the first training stroke image as the input of the model and the first training stroke corresponding to the first training stroke image as the target output of the model. The trained classification model includes multiple intermediate layers, each of which is used to output a vectorized processing result for the content input to the classification model.

[0140] Obtain the image of the second training stroke corresponding to the second training stroke in the second training text;

[0141] The second training stroke image is input into the classification model, and the output content of each intermediate layer of the classification model is extracted;

[0142] From the output content, determine the target output content with the highest similarity to the second training stroke;

[0143] The vector extraction model is defined as a local part of the classification model that starts with the input layer of the classification model and ends with the intermediate layer corresponding to the target output content.

[0144] According to one or more embodiments of this disclosure, a stroke order detection method is provided, wherein after the step of obtaining a first training sample, the method further includes:

[0145] Image enhancement is performed on the first training stroke image, and the enhanced image is associated with the first training stroke corresponding to the first training stroke image, so that the enhanced image can be used to train the classification model.

[0146] According to one or more embodiments of this disclosure, a stroke order detection method is provided, wherein determining the probability that the stroke to be identified is the target standard stroke, as a second similarity probability, based on user input preceding the stroke to be identified, includes:

[0147] Based on the user's input preceding the stroke to be identified, a pre-trained probability prediction model is used to determine the probability that the stroke to be identified is the target standard stroke, which is then used as the second similarity probability.

[0148] The probability prediction model is obtained in the following way:

[0149] Obtain a second training sample, which includes a training stroke sequence corresponding to the third training character. The training stroke sequence is composed of stroke vectors corresponding to the third training strokes contained in the third training character, and the order of the stroke vectors corresponding to the third training strokes in the training stroke sequence conforms to the standard stroke order of the third training character.

[0150] A probabilistic prediction model is obtained by training a language model by using a subsequence of the training pen sequence as input and the next element of the subsequence in the training pen sequence as the target output of the language model.

[0151] According to one or more embodiments of this disclosure, a stroke order detection method is provided, wherein determining whether the stroke order of the stroke to be identified is correct based on a first similarity probability and a second similarity probability includes:

[0152] For each target standard stroke, the target similarity probability between the stroke to be identified and the target standard stroke is calculated based on the first similarity probability and the second similarity probability corresponding to the target standard stroke.

[0153] The target stroke with the highest target similarity probability is determined as the target stroke corresponding to the stroke to be identified.

[0154] Based on the target stroke and the standard stroke order corresponding to the target character, determine whether the stroke order of the stroke to be identified is correct.

[0155] According to one or more embodiments of this disclosure, a stroke order detection method is provided, wherein calculating the target similarity probability between the stroke to be identified and the target standard stroke based on a first similarity probability and a second similarity probability corresponding to the target standard stroke includes:

[0156] The first similarity probability and the second similarity probability are weighted and calculated to obtain the target similarity probability between the stroke to be identified and the target standard stroke.

[0157] According to one or more embodiments of this disclosure, a stroke order detection device is provided, the device comprising:

[0158] The first acquisition module is used to acquire the strokes to be recognized input by the user during the writing of the target text, wherein the target text consists of at least two standard strokes;

[0159] The first determining module is used to determine the similarity between the stroke to be identified and the target standard stroke as a first similarity probability, wherein the target standard stroke is at least one of the standard strokes.

[0160] The second determining module is used to determine the probability that the stroke to be identified is the target standard stroke based on the user's input content before the stroke to be identified, as the second similarity probability;

[0161] The third determining module is used to determine whether the stroke order of the stroke to be identified is correct based on the first similarity probability and the second similarity probability.

[0162] According to one or more embodiments of the present disclosure, a computer-readable medium is provided having a computer program stored thereon that, when executed by a processing device, implements the steps of the stroke order detection method described in any embodiment of the present disclosure.

[0163] According to one or more embodiments of this disclosure, an electronic device is provided, comprising:

[0164] A storage device on which one or more computer programs are stored;

[0165] One or more processing devices are configured to execute the one or more computer programs in the storage device to implement the steps of the stroke order detection method according to any embodiment of the present disclosure.

[0166] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0167] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0168] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative forms of implementing the claims. Regarding the apparatus in the above embodiments, the specific manner in which the various modules perform their operations has been described in detail in the embodiments relating to the method, and will not be elaborated upon here.< / s> < / s> < / s>

Claims

1. A stroke order detection method, characterized in that, The method includes: The system acquires the strokes to be recognized input by the user during the writing of target characters, wherein the target characters consist of at least two standard strokes. Determine the similarity between the stroke to be identified and the target standard stroke as the first similarity probability, wherein the target standard stroke is at least one of the standard strokes; Based on the user's input preceding the stroke to be identified, a pre-trained probability prediction model is used to determine the probability that the stroke to be identified is the target standard stroke, which is then used as the second similarity probability. Based on the first similarity probability and the second similarity probability, determine whether the stroke order of the stroke to be identified is correct; The probability prediction model is obtained in the following way: Obtain a second training sample, which includes a training stroke sequence corresponding to the third training character. The training stroke sequence is composed of stroke vectors corresponding to the third training strokes contained in the third training character, and the order of the stroke vectors corresponding to the third training strokes in the training stroke sequence conforms to the standard stroke order of the third training character. A probabilistic prediction model is obtained by training a language model by using a subsequence of the training pen sequence as input and the next element of the subsequence in the training pen sequence as the target output of the language model.

2. The method according to claim 1, characterized in that, Determining the similarity between the stroke to be identified and the target standard stroke, as the first similarity probability, includes: Based on the pre-trained vector extraction model, determine the vector of the stroke to be identified; Determine the target standard stroke vector corresponding to each of the aforementioned target standard strokes; The similarity between the stroke vector to be identified and each of the target standard stroke vectors is calculated to obtain the first similarity probability.

3. The method according to claim 2, characterized in that, The vector extraction model is obtained in the following way: Obtain a first training sample, which includes the first training stroke image corresponding to the first training stroke in the first training text; The model is trained by using the first training stroke image as the input of the model and the first training stroke corresponding to the first training stroke image as the target output of the model. The trained classification model includes multiple intermediate layers, each of which is used to output a vectorized processing result for the content input to the classification model. Obtain the image of the second training stroke corresponding to the second training stroke in the second training text; The second training stroke image is input into the classification model, and the output content of each intermediate layer of the classification model is extracted; From the output content, determine the target output content with the highest similarity to the second training stroke; The vector extraction model is defined as a local part of the classification model that starts with the input layer of the classification model and ends with the intermediate layer corresponding to the target output content.

4. The method according to claim 3, characterized in that, After the step of obtaining the first training sample, the method further includes: Image enhancement is performed on the first training stroke image, and the enhanced image is associated with the first training stroke corresponding to the first training stroke image, so that the enhanced image can be used to train the classification model.

5. The method according to claim 1, characterized in that, Determining whether the stroke order of the stroke to be identified is correct based on the first similarity probability and the second similarity probability includes: For each target standard stroke, the target similarity probability between the stroke to be identified and the target standard stroke is calculated based on the first similarity probability and the second similarity probability corresponding to the target standard stroke. The target stroke with the highest target similarity probability is determined as the target stroke corresponding to the stroke to be identified. Based on the target stroke and the standard stroke order corresponding to the target character, determine whether the stroke order of the stroke to be identified is correct.

6. The method according to claim 5, characterized in that, The step of calculating the target similarity probability between the stroke to be identified and the target standard stroke based on the first similarity probability and the second similarity probability corresponding to the target standard stroke includes: The first similarity probability and the second similarity probability are weighted and calculated to obtain the target similarity probability between the stroke to be identified and the target standard stroke.

7. A stroke order detection device, characterized in that, The device includes: The first acquisition module is used to acquire the strokes to be recognized input by the user during the writing of the target text, wherein the target text consists of at least two standard strokes; The first determining module is used to determine the similarity between the stroke to be identified and the target standard stroke as a first similarity probability, wherein the target standard stroke is at least one of the standard strokes. The second determining module is used to determine the probability that the stroke to be identified is the target standard stroke based on the user's input content before the stroke to be identified, using a pre-trained probability prediction model, as the second similarity probability; The third determining module is used to determine whether the stroke order of the stroke to be identified is correct based on the first similarity probability and the second similarity probability. The probability prediction model is obtained through the following modules: The fourth acquisition module is used to acquire a second training sample. The second training sample includes a training stroke sequence corresponding to the third training character. The training stroke sequence is composed of stroke vectors corresponding to the third training strokes contained in the third training character, and the order of the stroke vectors corresponding to the third training strokes in the training stroke sequence conforms to the standard stroke order of the third training character. The second training module is used to train the model by taking a subsequence of the training pen sequence as input to the language model and taking the next element of the subsequence in the training pen sequence as the target output of the language model, so as to obtain a trained probability prediction model.

8. A computer-readable medium having a computer program stored thereon, characterized in that, When executed by the processing device, the program implements the steps of the method according to any one of claims 1-6.

9. An electronic device, characterized in that, include: A storage device having at least one computer program stored thereon; At least one processing device is configured to execute the at least one computer program in the storage device to implement the steps of the method according to any one of claims 1-6.

Images