Handwriting character detection method, device and electronic equipment

Abstract

This disclosure provides a method, apparatus, and electronic device for handwritten character detection, belonging to the field of text recognition technology. The method includes: acquiring a first image comprising a single handwritten character; performing text recognition on the first image to obtain a first character set and the confidence score of each character therein; if the maximum confidence score in the first character set is greater than a false confidence threshold but less than a true confidence threshold, matching the first image with a second image associated with each character in the first character set to obtain an image matching result, wherein each character in the first character set is associated with an image of at least one handwritten character; and determining the detection result of the handwritten character based on the image matching result. Using this disclosure, handwritten character detection can be achieved.

Description

Technical Field

[0001] This disclosure relates to the field of text recognition technology, and in particular to a method, apparatus and electronic device for detecting handwritten characters. Background Technology

[0002] In handwriting scenarios, the written content may contain glaring errors, possibly not even being a normal character; this is commonly referred to as a "misspelling." No solutions for detecting handwritten characters have been found in relevant technologies.

[0003] In OCR (Optical Character Recognition) scenarios, the goal in most cases is to recognize symbols more accurately, rather than to solve the problem of miswritten characters in the image information itself, so this issue has been neglected.

[0004] In conclusion, there is currently no effective solution for detecting handwritten characters. Summary of the Invention

[0005] According to one aspect of this disclosure, a method for detecting handwritten characters is provided, comprising:

[0006] Obtain the first image containing a single handwritten character;

[0007] Perform text recognition on the first image to obtain the first character set and the confidence score of each character in it;

[0008] If the maximum confidence score in the first character set is greater than the false confidence score threshold but less than the true confidence score threshold, the first image is matched with the second image associated with each character in the first character set to obtain the image matching result, wherein each character in the first character set is associated with at least one image of its handwritten character.

[0009] The detection result of handwritten characters is determined based on the image matching results.

[0010] According to one aspect of this disclosure, a handwritten character detection device is provided, comprising:

[0011] The acquisition module is used to acquire a first image including a single handwritten character;

[0012] The recognition module is used to perform text recognition on the first image to obtain a first character set and the confidence score of each character in it;

[0013] The matching module is used to match the first image with the second image associated with each character in the first character set when the maximum confidence score in the first character set is greater than the false confidence score threshold and less than the true confidence score threshold, so as to obtain the image matching result. In this case, each character in the first character set is associated with the image of at least one handwritten character.

[0014] The determination module is used to determine the detection result of handwritten characters based on the image matching results.

[0015] According to another aspect of this disclosure, an electronic device is provided, comprising:

[0016] Processor; and

[0017] Stored program memory,

[0018] The program includes instructions that, when executed by the processor, cause the processor to perform the methods of this disclosure.

[0019] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause the computer to perform the methods of this disclosure.

[0020] One or more technical solutions provided in the embodiments of this disclosure can realize handwritten character detection based on text recognition and image comparison. Attached Figure Description

[0021] Further details, features, and advantages of this disclosure are disclosed in the following description of exemplary embodiments in conjunction with the accompanying drawings, in which:

[0022] Figure 1 A flowchart of a handwritten character detection method according to an exemplary embodiment of the present disclosure is shown;

[0023] Figure 2 A schematic diagram showing an image including a single handwritten character is provided.

[0024] Figure 3 A schematic diagram showing the image of a handwritten character associated with another character is provided.

[0025] Figure 4 This diagram illustrates the image difference processing between the image of a handwritten character and the image of the handwritten character associated with the character.

[0026] Figure 5 Another flowchart of a handwritten character detection method according to an exemplary embodiment of the present disclosure is shown;

[0027] Figure 6 A schematic block diagram of a handwritten character detection process according to an exemplary embodiment of the present disclosure is shown;

[0028] Figure 7 A schematic block diagram of a misspelling detection process according to an exemplary embodiment of the present disclosure is shown;

[0029] Figure 8 A schematic block diagram illustrating an image comparison process according to an exemplary embodiment of the present disclosure is shown;

[0030] Figure 9 A schematic block diagram of a handwritten character detection apparatus according to an exemplary embodiment of the present disclosure is shown;

[0031] Figure 10 A structural block diagram of an exemplary electronic device that can be used to implement embodiments of the present disclosure is shown. Detailed Implementation

[0032] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0033] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0034] The term "comprising" and its variations as used herein are open-ended, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below. It should be noted that the concepts of "first", "second", etc., used in this disclosure are only used to distinguish different devices, modules, or units, and are not intended to limit the order of functions performed by these devices, modules, or units or their interdependencies.

[0035] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0036] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0037] The solutions of the present disclosure are described below with reference to the accompanying drawings.

[0038] Figure 1 A flowchart of a handwritten character detection method according to an exemplary embodiment of the present disclosure is shown. As Figure 1 shown, the handwritten character detection method includes steps S101 to step S104.

[0039] Step S101: Obtain a first image including a single handwritten character.

[0040] In this embodiment, the first image including a single handwritten character can be extracted from an image including multiple characters.

[0041] In this embodiment, the image of a single handwritten character can be extracted from a text image by various methods. For example, the image of a handwritten character can be obtained from a text image based on text detection, and further file detection based on semantic segmentation, etc. This embodiment does not limit this.

[0042] In this embodiment, the handwritten character can be Chinese, English letters, numbers, etc. Taking Chinese as an example, an example of the first image including a single handwritten character is as Figure 2 shown. The image includes a text part (also called the handwriting part), that is, the position where the character "文" is located, and a background part (the part other than the handwriting), that is, the white area outside the position where the character "文" is located.

[0043] Step S102: Perform text recognition on the first image to obtain a first character set and the confidence of each character therein.

[0044] In this embodiment, the first image can be text-recognized by various methods, and this embodiment does not limit the text recognition method.

[0045] In this embodiment, a false confidence threshold and a true confidence threshold are set. When the confidence of a character in the first character set is less than or equal to the false confidence threshold, the character on the first image is not the character in the first character set. When the confidence of a character in the first character set is greater than or equal to the true confidence threshold, the character on the first image is the character in the first character set. When the maximum confidence in the first character set is greater than the false confidence threshold and less than the true confidence threshold, proceed to step S103 for further processing.

[0046] Step S103: When the maximum confidence in the first character set is greater than the false confidence threshold and less than the true confidence threshold, match the first image with a second image associated with each character in the first character set to obtain an image matching result.

[0047] Among them, each character in the first character set is associated with an image of at least one handwritten character.

[0048] In this embodiment, images of at least one handwritten character for each character are collected in advance to form an association relationship between the character and the images of at least one handwritten character. According to the association relationship between the character and the images of at least one handwritten character, the images (second images) associated with the character can be obtained.

[0049] The images of the handwritten characters associated with the character are as Figure 3 shown. In Figure 3 the characters "six", "big", and "too" are taken as examples. Among them, the character "six" is associated with images of three different handwritten characters. See the right side of "six" in Figure 3 ; the character "big" is associated with images of two different handwritten characters. See the right side of "big" in Figure 3 ; the character "too" is associated with images of three different handwritten characters. See the right side of "too" in Figure 3 .

[0050] In some embodiments, characters that meet the preset conditions are selected from the first character set. The preset conditions include: the confidence level is greater than the preset threshold and / or the confidence level ranks among the top preset names, where the preset threshold is greater than the false confidence threshold and less than the true confidence threshold. Among them, in step S103, the first image is matched with the second images associated with each of the selected characters. For the case where the first character set is relatively large, further selection is performed by setting the preset threshold, and the characters in the first character set with a confidence level less than or equal to the preset threshold are excluded from the candidate characters on the first image, reducing the computational amount for subsequent processing.

[0051] As a possible implementation manner, matching the first image with the second images associated with each character in the first character set to obtain an image matching result, including:

[0052] Performing image difference processing between the first image and each second image associated with each character in the first character set to obtain a difference image between the first image and each second image associated with each character in the first character set;

[0053] Processing each difference image using a pre-trained neural network model to obtain an image matching result between the first image and each second image associated with each character in the first character set.

[0054] Among them, the pre-trained neural network model may include a convolutional neural network. This embodiment does not limit the neural network model.

[0055] The purpose of image difference processing is to highlight the differences between two images. For images of handwritten characters, if the writing is consistent, then the same position on both images represents the same part, meaning that the same image area is either all text or all background outside the text.

[0056] As an example, in order to enable the neural network model to pay attention to the differences in writing, a graph difference processing is performed between the first image and each second image associated with each character in the first character set, including: for each pixel position on the first image, if it is a different part from the corresponding pixel position on each second image, the corresponding pixel position on the difference image is determined as a first value, wherein the first image and the second image are divided into a text part and a background part.

[0057] Furthermore, for each pixel position in the first image, if it is the same as the corresponding pixel position in the second image and is a text portion, the corresponding pixel position in the difference image is determined as the second value.

[0058] Furthermore, for each pixel position in the first image, if it is the same as the corresponding pixel position in the second image and is part of the background, the corresponding pixel position in the difference image is determined as the third value.

[0059] The images of handwritten characters (first image) and characters (second image) typically have a significant color distinction between the text and background areas. Therefore, the color value of a pixel can be used to distinguish whether it belongs to the text or background area. For example, handwritten character images and character images are usually white background with black text or white background with colored text (e.g., blue text).

[0060] As an example, when converting images of handwritten characters and other characters to black and white, the color value of the text portion of the image is 1, and the color value of the background portion is 0. (Reference) Figure 4 As shown, for each pixel position of the difference image: if the color value on both the handwritten character image and the character image is 0, the color value of the pixel position in the difference image is set to 0; if the color value on both the handwritten character image and the character image is 1, the color value of the pixel position in the difference image is set to 0.5; if the color values ​​on the handwritten character image and the character image are different, the pixel position in the difference image is set to 1.

[0061] Step S104: Determine the detection result of handwritten characters based on the image matching result.

[0062] As one possible implementation, step S104 includes: determining the characters in the first character set that match the handwritten characters based on the image matching results; and determining the detection result of the handwritten characters based on the characters in the first character set that match the handwritten characters.

[0063] As an example, determining the characters in the first character set that match the handwritten characters based on the image matching results includes: determining a first number of second images in which each character in the first character set matches the first image; and determining the character matching result of each character with the handwritten characters based on the first number corresponding to each character in the first character set.

[0064] As an example, the character matching result of each character with the handwritten character is determined based on the first number corresponding to each character in the first character set, including: for each character in the first character set, if the first number corresponding to the character is greater than the first preset number, the character is determined to match the handwritten character.

[0065] Optionally, considering the different levels of complexity of characters (e.g., the number of strokes), each character in the first character set is preset with the aforementioned first preset number.

[0066] As an example, determining the detection result of a handwritten character based on the characters that match the handwritten character in the first character set includes: determining the detection result of the handwritten character based on a second number of characters that match the handwritten character in the first character set.

[0067] In some embodiments, the detection result of handwritten characters can be determined based on the image matching result, which can enable the recognition of handwritten characters that cannot be recognized by text recognition.

[0068] In some embodiments, the detection result of handwritten characters can be determined based on the image matching result, thereby enabling the detection of the handwritten characters.

[0069] Figure 5 Another flowchart of a handwritten character detection method according to an exemplary embodiment of the present disclosure is shown, such as Figure 5 As shown, the handwritten character detection method includes steps S501 to S506.

[0070] Step S501: Obtain a first image including a single handwritten character.

[0071] Step S502: Perform text recognition on the first image to obtain the first character set and the confidence score of each character in it.

[0072] Step S503: Determine the relationship between the maximum confidence level in the first character set and the false confidence threshold and the true confidence threshold.

[0073] If the maximum confidence score in the first character set is less than or equal to the false confidence score threshold, the handwritten character in the first image is determined to be incorrect. If the maximum confidence score in the first character set is greater than or equal to the true confidence score threshold, the handwritten character in the first image is determined to be correct, and the recognized character is output. If the maximum confidence score in the first character set is greater than the false confidence score threshold but less than the true confidence score threshold, proceed to step S504.

[0074] Step S504: If the maximum confidence score in the first character set is greater than the false confidence score threshold but less than the true confidence score threshold, match the first image with the second image associated with each character in the first character set to obtain the image matching result.

[0075] As one possible implementation, step S504 includes:

[0076] Perform image difference processing between the first image and each second image associated with each character in the first character set to obtain the difference image between the first image and each second image associated with each character in the first character set;

[0077] Each difference image is processed using a pre-trained neural network model to obtain image matching results between the first image and each second image associated with each character in the first character set.

[0078] The purpose of image difference processing is to highlight the differences between two images. For images of handwritten characters, if the writing is consistent, then the same position on both images represents the same part, meaning that the same image area is either all text or all background outside the text.

[0079] As an example, to enable the neural network model to focus on differences in writing, image difference processing is performed between the first image and each second image associated with each character in the first character set, including:

[0080] For each pixel position in the first image, if it is a different part from the corresponding pixel position in each second image, the corresponding pixel position in the difference image is determined as the first value, wherein the first image and the second image are divided into text part and background part;

[0081] For each pixel position in the first image, if it is the same as the corresponding pixel position in the second image and is a text part, the corresponding pixel position in the difference image is determined as the second value.

[0082] For each pixel position in the first image, if it is the same as the corresponding pixel position in the second image and is part of the background, the corresponding pixel position in the difference image is determined as the third value.

[0083] Step S505: Based on the image matching results, determine the characters in the first character set that match the handwritten characters.

[0084] As one implementation method, step S505 includes:

[0085] Determine the first number of second images in which each character in the first character set matches the first image;

[0086] Based on the first number corresponding to each character in the first character set, determine the character matching result between each character and the handwritten character.

[0087] As an example, the character matching result of each character with the handwritten character is determined based on the first number corresponding to each character in the first character set, including: for each character in the first character set, if the first number corresponding to the character is greater than the first preset number, the character is determined to match the handwritten character.

[0088] Optionally, considering the different levels of complexity of characters (e.g., the number of strokes), each character in the first character set is preset with the aforementioned first preset number.

[0089] Step S506: Determine the detection result of the handwritten character based on the characters that match the handwritten character in the first character set.

[0090] As one implementation, step S506 includes: determining the detection result of the handwritten character based on a second number of characters that match the handwritten character in the first character set.

[0091] Handwritten characters are very common in intelligent homework grading scenarios. The following is an example of this disclosure, using homework grading as an example.

[0092] In this example, such as Figure 6 As shown, it includes a semantic segmentation module, an OCR module, and a misspelling detection module.

[0093] Semantic segmentation module

[0094] The semantic segmentation module performs image semantic segmentation on the handwritten homework image, dividing the written content in the "handwritten homework image" into a group of images, represented as T, based on the characters.

[0095] OCR module

[0096] The OCR module identifies each image in T and converts it into the character with the highest confidence level.

[0097] In this process, for each small image in T, a confidence score is generated for each character in the OCR-supported recognizable character set C (or, in some OCR schemes, characters with higher confidence scores are selected to calculate probabilities, while the confidence scores of other unselected characters are set to 0). The set of confidence scores is denoted by S, where S = [s1, s2, ..., s...]. n ], where s i Represents the character c i The confidence level of a certain image in T.

[0098] The acquisition of the confidence set S is not limited to the OCR process; it can also be achieved by combining high-dimensional textual information from the preceding and following contexts for confidence scoring. This example will not elaborate on this aspect.

[0099] The following operations are performed on a unit of small images in T, which are represented by t.

[0100] Typo detection module

[0101] like Figure 7 As shown, the misspelling detection module is further divided into three modules: detection and filtering module, image comparison module, and comprehensive judgment module.

[0102] Detection and filtering module

[0103] For t, in the above process, a set of confidence scores S for each element in the supported character set C is obtained, and the score s with the highest confidence score is selected. max The value is determined by formula (1):

[0104] f(s max The possible values ​​are as follows:

[0105] 1,s max Greater than or equal to th r,i ;

[0106] 0,th f,i Less than s max Less than th r,i ;

[0107] -1,s max Less than or equal to th f,i ;

[0108] Among them, th r,i Represents the character c in C i The truth confidence threshold, if a certain character c i The corresponding confidence level s i It is the highest value in S, and s i Greater than or equal to th r,i If the character in t is correctly written, then the result in formula (1) is 1; thf,i This represents the false confidence threshold for each character in C. If ci has a false confidence threshold, then s i It is the highest value in S, and s i Less than or equal to th f,i If the character in t is incorrect, the result in the formula is -1; otherwise, the result is 0, and the process proceeds to the image comparison module.

[0109] th r,i and th f,i It can be obtained from the OCR test set statistics in specific tasks.

[0110] In C++, the truth confidence threshold can be the same for each character, or it can be set separately for each character. Similarly, the false confidence threshold can be the same for each character, or it can be set separately for each character. Because each character is different—for example, characters with complex strokes differ from simple characters—setting the threshold separately allows for differentiated processing of different characters.

[0111] When the formula output is 0, select the k characters corresponding to the top k confidence levels in S (k is a hyperparameter, set between 3 and 10 depending on the scenario), and add the characters corresponding to the elements in S that have a confidence level greater than the false value threshold. Let the union of the two parts be H, and the number of characters be m. Then H = [h1, h2, ..., h m ].

[0112] Image comparison module

[0113] In the image comparison module, a set of handwritten images is collected for each character (a recognizable character) in C. This image library is manually collected. The collection standard is the writing style of various norms in the usage scenario. However, the standard of norms is difficult to quantify, so it can be selected through a vote by multiple people. The number of images in the sub-set corresponding to each character does not need to be the same. The image comparison determines whether t is consistent with the content of the handwritten image corresponding to each character in H.

[0114] The images from the m sets of handwritten images corresponding to t and H are grouped into image pairs (original image t and comparison image d), and each pair is processed as follows. The smaller images here are all black and white images of uniform size.

[0115] like Figure 8 As shown, the image comparison module includes an image difference module, a feature extraction module, and a classifier module.

[0116] Map Difference Module

[0117] The image difference module receives the original image t and the comparison image d (the image of the characters in H), and calculates the difference image q using formula (2):

[0118] qij =0,t ij =d ij =0;

[0119] q ij =0.5, t ij =d ij =1;

[0120] q ij =1,t ij Not equal to d ij ;

[0121] Here, i and j represent the row and column indices of the image pixels. Setting the positions of inequalities to 1 allows the model to focus more on the differences between pixels. Then, the difference image q enters the feature extraction module.

[0122] Feature extraction module

[0123] The feature extraction module extracts high-dimensional features from the difference image q. This can be done using a convolutional neural network (CNN) or some novel network structures.

[0124] Classifier module

[0125] The classifier module uses a fully connected network layer.

[0126] After passing through the classifier module, each image of each character in H yields a recognition result. The number of positive examples (the number of matched images) for each character in H is then counted, forming the positive example count: Z = [z1, z2, ..., z m For example, if h1 has 3 images that match t, ​​then z1 takes the value 3.

[0127] Comprehensive judgment module

[0128] Z is used to obtain M through formula (3):

[0129] m i =1, zi > 0;

[0130] m i =0, zi=0.

[0131] For t, O can be obtained through formula (4):

[0132] O = sum(M).

[0133] The evaluation of the characters written in t is given. There are 5 levels: A, B, C, D, and E.

[0134] in:

[0135] Formula (1) (i.e., f(s)max If the output result is 1, then it is set to grade A;

[0136] If the output of formula (4) (i.e., O) is 1, then it is set to level B;

[0137] If the output of formula (4) is greater than 1, it is set to grade C;

[0138] If the output of formula (4) is equal to 0, then the grade is set to D.

[0139] If the output of formula (1) is -1, then we define it as level E;

[0140] Among them, grades A and B are considered to be correctly written, grade C is considered to be non-standard written, and grades D and E are considered to be incorrectly written.

[0141] In this example, handwriting error detection is achieved by combining OCR and image comparison. The two images are fused using image difference analysis to highlight differences and downplay similarities, effectively solving the problem of handwritten character recognition. To enhance user experience in the context of homework grading, a tiered handwriting error detection strategy is provided. This strategy uses comparisons of different character sub-images to determine the severity of errors and provides varying levels of feedback.

[0142] Figure 9 A schematic diagram of a handwritten character detection device according to an exemplary embodiment of the present disclosure is shown, such as... Figure 9 As shown, the handwritten character detection device includes:

[0143] Acquisition module 910 is used to acquire a first image including a single handwritten character;

[0144] The recognition module 920 is used to perform text recognition on the first image to obtain a first character set and the confidence level of each character therein;

[0145] The matching module 930 is used to match the first image with the second image associated with each character in the first character set when the maximum confidence score in the first character set is greater than the false confidence score threshold and less than the true confidence score threshold, so as to obtain the image matching result. In this case, each character in the first character set is associated with the image of at least one handwritten character.

[0146] The determination module 940 is used to determine the detection result of handwritten characters based on the image matching result.

[0147] As one implementation method, the determining module 940 determines the detection result of handwritten characters based on the image matching result, specifically including:

[0148] Based on the image matching results, determine the characters in the first character set that match the handwritten characters;

[0149] The detection result of the handwritten character is determined based on the characters that match the handwritten character in the first character set.

[0150] As one implementation method, the determining module 940 determines, based on the image matching results, the characters in the first character set that match the handwritten characters, specifically including:

[0151] Determine the first number of second images in which each character in the first character set matches the first image;

[0152] Based on the first number corresponding to each character in the first character set, determine the character matching result between each character and the handwritten character.

[0153] As one implementation method, the determining module 940 determines the character matching result between each character and the handwritten character based on the first number corresponding to each character in the first character set, specifically including:

[0154] For each character in the first character set, if the first number corresponding to the character is greater than the first preset number, the character is determined to match the handwritten character.

[0155] In one implementation method, each character in the first character set is preset with a first preset number.

[0156] As one implementation method, the determining module 940 determines the detection result of the handwritten character based on the characters that match the handwritten character in the first character set. Specifically, it determines the detection result of the handwritten character based on the second number of characters that match the handwritten character in the first character set.

[0157] In one implementation, the matching module 930 matches the first image with the second image associated with each character in the first character set to obtain an image matching result, specifically including:

[0158] Perform image difference processing between the first image and each second image associated with each character in the first character set to obtain the difference image between the first image and each second image associated with each character in the first character set;

[0159] Each difference image is processed using a pre-trained neural network model to obtain image matching results between the first image and each second image associated with each character in the first character set.

[0160] As one implementation, the matching module 930 performs image difference processing between the first image and each second image associated with each character in the first character set, specifically including:

[0161] For each pixel position in the first image, if it is a different part from the corresponding pixel position in each second image, the corresponding pixel position in the difference image is determined as a first value, wherein the first image and the second image are divided into a text part and a background part.

[0162] As one implementation, the matching module 930 performs image difference processing between the first image and each second image associated with each character in the first character set, further including:

[0163] For each pixel position in the first image, if it is the same as the corresponding pixel position in the second image and belongs to text, determine the corresponding pixel position in the difference image as the second value; and / or

[0164] For each pixel position in the first image, if it is the same as the corresponding pixel position in the second image and is part of the background, the corresponding pixel position in the difference image is determined as the third value.

[0165] As one implementation, it further includes: selecting characters from a first character set that meet preset conditions, the preset conditions including: confidence level greater than a preset threshold and / or confidence level ranked first preset name, wherein the preset threshold is greater than the false value confidence threshold and less than the true value confidence threshold; wherein, the matching module 930 is used to match the first image with the second image associated with each selected character.

[0166] Exemplary embodiments of this disclosure also provide an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor. The memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to cause the electronic device to perform a method according to an embodiment of this disclosure.

[0167] Exemplary embodiments of this disclosure also provide a non-transitory computer-readable storage medium storing a computer program, wherein the computer program, when executed by a computer's processor, is used to cause the computer to perform a method according to embodiments of this disclosure.

[0168] Exemplary embodiments of this disclosure also provide a computer program product, including a computer program, wherein, when executed by a processor of a computer, the computer program is used to cause the computer to perform a method according to an embodiment of this disclosure.

[0169] refer to Figure 10The present invention describes a structural block diagram of an electronic device 1000 that can serve as a server or client of the present disclosure, which is an example of a hardware device that can be applied to various aspects of the present disclosure. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0170] like Figure 10 As shown, the electronic device 1000 includes a computing unit 1001, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1002 or a computer program loaded from a storage unit 1008 into a random access memory (RAM) 1003. The RAM 1003 may also store various programs and data required for the operation of the device 1000. The computing unit 1001, ROM 1002, and RAM 1003 are interconnected via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0171] Multiple components in electronic device 1000 are connected to I / O interface 1005, including: input unit 1006, output unit 1007, storage unit 1008, and communication unit 1009. Input unit 1006 can be any type of device capable of inputting information to electronic device 1000. Input unit 1006 can receive input digital or character information and generate key signal inputs related to user settings and / or function control of electronic device. Output unit 1007 can be any type of device capable of presenting information and may include, but is not limited to, a display, speaker, video / audio output terminal, vibrator, and / or printer. Storage unit 1008 may include, but is not limited to, disk and optical disk. Communication unit 1009 allows electronic device 1000 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication transceivers, and / or chipsets, such as Bluetooth devices, WiFi devices, WiMax devices, cellular communication devices, and / or the like.

[0172] The computing unit 1001 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1001 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1001 performs each of the methods and processes described above. For example, in some embodiments, the handwritten character detection method can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 1008. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 1000 via ROM 1002 and / or communication unit 1009. In some embodiments, the computing unit 1001 can be configured to perform the handwritten character detection method by any other suitable means (e.g., by means of firmware).

[0173] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0174] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0175] As used in this disclosure, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or apparatus (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including machine-readable media that receive machine instructions as machine-readable signals. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0176] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0177] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0178] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other.

Claims

1. A method for detecting handwritten characters, characterized in that, include: Obtain the first image containing a single handwritten character; Perform text recognition on the first image to obtain a first character set and the confidence score of each character in it; When the maximum confidence score in the first character set is greater than the false confidence score threshold but less than the true confidence score threshold, image difference processing is performed between the first image and each second image associated with each character in the first character set to obtain a difference image between the first image and each second image associated with each character in the first character set. Each difference image is then processed using a pre-trained neural network model to obtain an image matching result between the first image and each second image associated with each character in the first character set, wherein each character in the first character set is associated with an image of at least one handwritten character. The detection result of the handwritten characters is determined based on the image matching result; The process of performing image difference processing between the first image and each second image associated with each character in the first character set includes: For each pixel position in the first image, if it is a different part from the corresponding pixel position in each second image, the corresponding pixel position in the difference image is determined as a first value, wherein the first image and the second image are divided into text part and background part; For each pixel position in the first image, if it is the same as the corresponding pixel position in the second image and is a text portion, the corresponding pixel position in the difference image is determined as a second value; and / or, for each pixel position in the first image, if it is the same as the corresponding pixel position in the second image and is a background portion, the corresponding pixel position in the difference image is determined as a third value.

2. The handwritten character detection method as described in claim 1, characterized in that, Determining the detection result of the handwritten character based on the image matching result includes: Based on the image matching results, determine the characters in the first character set that match the handwritten characters; The detection result of the handwritten character is determined based on the characters that match the handwritten character in the first character set.

3. The handwritten character detection method as described in claim 2, characterized in that, Based on the image matching results, the characters in the first character set that match the handwritten characters are determined, including: Determine the first number of second images that match each character in the first character set with the first image; Based on the first number corresponding to each character in the first character set, determine the character matching result between each character and the handwritten character.

4. The handwritten character detection method as described in claim 2, characterized in that, Based on the characters in the first character set that match the handwritten character, the detection result of the handwritten character is determined, including: The detection result of the handwritten character is determined based on the second number of characters that match the handwritten character in the first character set.

5. The handwritten character detection method according to any one of claims 1 to 4, characterized in that, Also includes: Select characters that meet preset conditions from the first character set. The preset conditions include: confidence level greater than a preset threshold and / or confidence level ranked first in a preset name, wherein the preset threshold is greater than the false value confidence threshold and less than the true value confidence threshold. The process of matching the first image with the second image associated with each character in the first character set includes: matching the first image with the second image associated with each selected character.

6. The handwritten character detection method as described in claim 3, characterized in that, Based on the first number corresponding to each character in the first character set, determine the character matching result between each character and the handwritten character, including: For each character in the first character set, if the first number corresponding to the character is greater than the first preset number, the character is determined to match the handwritten character.

7. The handwritten character detection method as described in claim 6, characterized in that, Each character in the first character set is preset with the first preset number.

8. A handwritten character detection device, characterized in that, include: The acquisition module is used to acquire a first image including a single handwritten character; The recognition module is used to perform text recognition on the first image to obtain a first character set and the confidence score of each character therein; The matching module is used to perform image difference processing between the first image and each second image associated with each character in the first character set when the maximum confidence score in the first character set is greater than the false confidence score threshold and less than the true confidence score threshold, to obtain a difference image between the first image and each second image associated with each character in the first character set, and to process each difference image using a pre-trained neural network model to obtain an image matching result between the first image and each second image associated with each character in the first character set, wherein each character in the first character set is associated with an image of at least one handwritten character; The determining module is used to determine the detection result of the handwritten character based on the image matching result; Specifically, when performing image difference processing between the first image and each second image associated with each character in the first character set, the matching module is used for: For each pixel position in the first image, if it is a different part from the corresponding pixel position in each second image, the corresponding pixel position in the difference image is determined as a first value, wherein the first image and the second image are divided into text part and background part; For each pixel position in the first image, if it is the same as the corresponding pixel position in the second image and is a text portion, the corresponding pixel position in the difference image is determined as a second value; and / or, for each pixel position in the first image, if it is the same as the corresponding pixel position in the second image and is a background portion, the corresponding pixel position in the difference image is determined as a third value.

9. An electronic device, comprising: processor; as well as Stored program memory, The program includes instructions that, when executed by the processor, cause the processor to perform the method according to any one of claims 1-7.

10. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-7.

Images