Method and device for identifying handwritten name in scanned paper, computing equipment and storage medium

Abstract

The embodiment of the invention relates to the technical field of machine learning, in particular to an identification method and device for a handwritten name in a scanned paper, computing equipment and a storage medium. The method comprises the steps that a handwritten name image is acquired and preprocessed to obtain an image I; printing the candidate name on an image J with the same size as the image I according to a printed form; inputting I into a neural network 1 and a neural network 2 to obtain v1 and v2, inputting J into the neural network 1 and a neural network 3 to obtain u1 and u2, merging I and J in a channel dimension, and inputting the merged I and J into a neural network 4 to obtain p3; calculating a similarity value p1 of the v1 and the u1, calculating a similarity value p2 of the v2 and the u2, and calculating the similarity between the candidate name and the name to be identified according to p1, p2 and p3; and finding a name with the maximum similarity with the handwritten name from all candidate names as an identification result. According to the scheme, the recognition problem of the handwritten name in the closed set is solved by means of whole word comparison, and the handwritten name recognition performance in a special scene can be improved.

Description

TECHNICAL FIELD

[0001] The embodiment of the present application relates to the technical field of machine learning, in particular to a handwritten name recognition method in scanned volume surface, device, computing equipment and storage medium. BACKGROUND

[0002] Because there are often rare Chinese characters in Chinese names, and there is no obvious semantic information, combined with the structure and shape change in writing names, the general handwritten Chinese character recognition system is not good at recognizing names, even if the name is recognized and compared with the candidate name set, due to the short length of the name, the edit distance difference is not obvious, resulting in unsatisfactory final recognition performance. SUMMARY

[0003] In view of the problem that the existing handwritten name recognition method in scanned volume surface does not design for the special problem of handwritten name recognition and the special application scene of examination volume, the embodiment of the present application provides a handwritten name recognition method in scanned volume surface to improve the above problems. In order to achieve the above purpose, the technical scheme adopted by the present application is as follows:

[0004] In the first aspect, a handwritten name recognition method in scanned volume surface is provided, comprising:

[0005] obtaining a handwritten name region image;

[0006] preprocessing the handwritten name region image to obtain a preprocessed image I;

[0007] For each candidate name s, print the name s on the image J as a printed body, and the image J is a candidate name image, then input I into convolutional neural network 1 and convolutional neural network 2 respectively to obtain output vectors v1 and vector v2, input image J into convolutional neural network 1 and convolutional neural network 3 respectively to obtain output vectors u1 and vector u2, then input I and J in the channel dimension after merging to input convolutional neural network 4 to obtain similarity value p3, then calculate the similarity value p1 of vector v1 and vector u1, the similarity value p2 of vector v2 and vector u2, and finally calculate the similarity of the candidate name s and the handwritten name to be recognized according to the values of p1, p2 and p3; the convolutional neural network 1, the convolutional neural network 2, the convolutional neural network 3 and the convolutional neural network 4 are all obtained by using multiple groups of data through deep learning training, and the multiple groups of data include training images and expected similarity values;

[0008] Find the name with the largest similarity to the handwritten name from all candidate names as the recognition result.

[0009] Further, the step of obtaining a handwritten name region image comprises:

[0010] According to the pre-labeled position of the handwritten name region, a handwritten name region image is cut from the scanned document image.

[0011] Further, the preprocessing operation step comprises:

[0012] If the scanned document image is white background and black text, the handwritten name region image is first inverted to obtain a handwritten image A with black background and white text; otherwise, the handwritten name region image is directly assigned to image A; the background value of the obtained image A is 0 and the foreground value is 1.

[0013] The image A is accumulated by row to obtain a vector r and accumulated by column to obtain a vector c, the starting row and ending row numbers of the handwritten name region are found according to r>0, the absolute value d of the change of the adjacent column accumulation value is calculated according to c, and the starting column and ending column numbers of the handwritten name region are found according to d>1. The image block B is cut from the image A according to the starting row and column and the ending row and column.

[0014] After the image B is normalized to a predefined size, a preprocessed image I is obtained.

[0015] Further, the convolutional neural network 1 is a branch network in a twin neural network structure, the twin neural network structure comprises two network branches, the two network branches share network structure and parameters, and the inputs of the two network branches are respectively a handwritten name image and a candidate name image, and the outputs of the two network branches are both feature vectors.

[0016] Further, the convolutional neural network 2 and the convolutional neural network 3 are two branch networks in a twin neural network structure, but the convolutional neural network 2 and the convolutional neural network 3 do not share parameters, the inputs of the two network branches are respectively a handwritten name image and a candidate name image, and the outputs of the two network branches are both feature vectors.

[0017] Further, the convolutional neural network 4 comprises a convolutional neural network structure and a fully connected network structure, the input of the convolutional neural network 4 is a double-channel image obtained by fusing a handwritten name image and a candidate name image in a channel dimension, the output of the convolutional neural network structure is a feature vector, and the similarity of the handwritten name image and the candidate name image is obtained after the fully connected network structure.

[0018] Further, the step of calculating the similarity of the vector u and the vector v comprises: first calculating a difference vector w of the vector u and the vector v, and then inputting the w into a fully connected network, and the output of the fully connected network represents the similarity of the vector u and the vector v.

[0019] In a second aspect, an embodiment of the present application provides a handwritten name recognition device in a scanned document, comprising:

[0020] An acquisition unit is configured to acquire an image to be recognized.

[0021] an input unit configured to pre-process the image and input the pre-processed image into the pre-trained recognition model;

[0022] an output unit configured to process the model output and output a recognition result.

[0023] In a third aspect, an embodiment of the present application further provides a computing device, comprising:

[0024] a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method in any embodiment of the present application.

[0025] In a fourth aspect, an embodiment of the present application further provides a computer readable storage medium, which stores a computer program, and the computer program, when executed in a computer, causes the computer to execute the method in any embodiment of the present application.

[0026] The present application has the following beneficial effects:

[0027] The present application solves the problems that are difficult to be solved by traditional algorithms, such as the inclusion of rare characters in the name and the random writing of the name, by using the idea of whole-word comparison to realize the recognition of handwritten Chinese names in a closed set, and is beneficial to improving the performance of handwritten Chinese name recognition in special scenarios. BRIEF DESCRIPTION OF DRAWINGS

[0028] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor based on these drawings.

[0029] Figure 1 is a flow chart of a handwritten name recognition method in a scanned volume surface provided by an embodiment of the present application;

[0030] Figure 2 is a pre-processing schematic diagram provided by an embodiment of the present application;

[0031] Figure 3 is a model architecture diagram of a handwritten name recognition method in a scanned volume surface provided by an embodiment of the present application;

[0032] Figure 4 is a hardware architecture diagram of an electronic device provided by an embodiment of the present application;

[0033] Figure 5It is an embodiment of the present application to provide a handwritten name recognition device structure diagram in a scanned volume. DETAILED DESCRIPTION

[0034] To make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0035] As described above, in the related art, the conventional handwritten name recognition method usually does not consider the particularity of handwritten Chinese name recognition in a volume, resulting in poor effect of a general handwritten Chinese character recognition system in recognizing names. In order to solve the above technical problem, the inventors consider using a similarity measurement network to compare the similarity between a handwritten name and a candidate name, avoiding the problem of directly recognizing rare characters and other special problems, indirectly realizing the name recognition problem, and being beneficial to improving the handwritten name recognition performance in a scanned volume.

[0036] The specific implementation of the above concept will be described below.

[0037] Reference Figure 1 The embodiments of the present application provide a handwritten name recognition method in a scanned volume, which comprises the following steps.

[0038] Step 100: According to the pre-calibrated handwritten name region position, a handwritten name region image is cut out from a scanned volume image;

[0039] Step 101: The handwritten name region image is preprocessed, and a preprocessing schematic diagram is as shown in Figure 2 The step comprises the following steps.

[0040] If the scanned volume image is white background and black characters, the handwritten name region image is first subjected to color inversion processing to obtain a handwritten image A with black background and white characters; otherwise, the handwritten name region image is directly assigned to image A; the background value of the obtained image A is 0, and the foreground value is 1;

[0041] The image A is accumulated by row to obtain a vector r and accumulated by column to obtain a vector c, the starting row and ending row numbers of the handwritten name region are found according to r>0, the change absolute value d of the adjacent column accumulation value is calculated according to c, and the starting column and ending column numbers of the handwritten name region are found according to d>1. The image block B is cut out from the image A according to the starting row and column and the ending row and column.

[0042] The image B is normalized to a pre-defined size to obtain a preprocessed image I.

[0043] Step 102: For each candidate name s, print the name s onto image J in print form, where image J is the candidate name image. Then, input I into convolutional neural network 1 and convolutional neural network 2 respectively to obtain output vectors v1 and v2, input image J into convolutional neural network 1 and convolutional neural network 3 respectively to obtain output vectors u1 and u2, then merge I and J in the channel dimension and input them into convolutional neural network 4 to obtain a similarity value p3, then calculate the similarity value p1 between vector v1 and vector u1, and the similarity value p2 between vector v2 and vector u2, and finally calculate the similarity between the candidate name s and the handwritten name to be recognized based on the values ​​of p1, p2 and p3; the convolutional neural network 1, convolutional neural network 2, convolutional neural network 3 and convolutional neural network 4 are all obtained by deep learning training using multiple sets of data, and the multiple sets of data include training images and expected similarity values; in this embodiment of the present invention, the network structure of the recognition model is as follows Figure 3 shown.

[0044] Figure 3 Convolutional Neural Networks 1, 2, 3, and 4 all use ResNet18 as the underlying feature extraction network architecture. The twin neural network in which Convolutional Neural Network 1 resides shares network structure and parameters. The inputs of the two network branches are the handwritten name image and the candidate name image, respectively, and both branches output feature vectors. Convolutional Neural Networks 2 and 3 also have two branches in a similar twin neural network architecture, but they do not share parameters. The inputs of the two network branches are the handwritten name image and the candidate name image, respectively, and both branches output feature vectors. The vectors output by the two twin neural networks are first subtracted and then input into a fully connected network. The output of the fully connected network is sigmoid-transformed to represent the similarity between the two vectors. Convolutional Neural Network 4 comprises a feature extraction module and a fully connected module. The feature extraction module also uses ResNet18 as its underlying architecture. The fully connected module is a three-layer fully connected network, and the output of the fully connected network is a floating-point value. The input of the convolutional neural network 4 is a dual-channel image obtained by fusion of the handwritten name image and the candidate name image in the channel dimension. The output of the feature extraction module is a feature vector. The output of the fully connected module is converted by Sigmoid to obtain the similarity between the handwritten name image and the candidate name image.

[0045] Step 103: After the similarities between all candidate names and the handwritten name to be recognized are obtained in step 102, the candidate name with the greatest similarity is selected as the final recognition result.

[0046] like Figure 4 、 Figure 5As shown in the figure, the embodiment of the present application provides a handwritten name recognition device in a scanned document surface. The device embodiment can be realized by software, or realized by hardware or a combination of software and hardware. From the hardware layer, as shown in the figure, a hardware architecture diagram of a computing device where the handwritten name recognition device in a scanned document surface provided by the embodiment of the present application is located, in addition to the processor, the memory, the network interface, the computing device where the device in the embodiment usually can also include other hardware, such as a forwarding chip responsible for processing a message and the like. Taking the software implementation as an example, as shown in the figure, as a logically meaningful device, it is formed by the CPU of the computing device where it is located to read the corresponding computer program in the non-volatile memory into the memory and run. Figure 4 As shown in the figure, a hardware architecture diagram of a computing device where the handwritten name recognition device in a scanned document surface provided by the embodiment of the present application is located, in addition to the processor, the memory, the network interface, the computing device where the device in the embodiment usually can also include other hardware, such as a forwarding chip responsible for processing a message and the like. Taking the software implementation as an example, as shown in the figure, as a logically meaningful device, it is formed by the CPU of the computing device where it is located to read the corresponding computer program in the non-volatile memory into the memory and run. Figure 4 As shown in the figure, a hardware architecture diagram of a computing device where the handwritten name recognition device in a scanned document surface provided by the embodiment of the present application is located, in addition to the processor, the memory, the network interface, the computing device where the device in the embodiment usually can also include other hardware, such as a forwarding chip responsible for processing a message and the like. Taking the software implementation as an example, as shown in the figure, as a logically meaningful device, it is formed by the CPU of the computing device where it is located to read the corresponding computer program in the non-volatile memory into the memory and run. Figure 5 As shown in the figure, a hardware architecture diagram of a computing device where the handwritten name recognition device in a scanned document surface provided by the embodiment of the present application is located, in addition to the processor, the memory, the network interface, the computing device where the device in the embodiment usually can also include other hardware, such as a forwarding chip responsible for processing a message and the like. Taking the software implementation as an example, as shown in the figure, as a logically meaningful device, it is formed by the CPU of the computing device where it is located to read the corresponding computer program in the non-volatile memory into the memory and run.

[0047] As shown in the figure, the embodiment provides a handwritten name recognition device in a scanned document surface, which comprises: Figure 5 As shown in the figure, the embodiment provides a handwritten name recognition device in a scanned document surface, which comprises:

[0048] The acquisition unit 301 is configured to acquire an image to be recognized.

[0049] The input unit 302 is configured to pre-process the image and input the image into a pre-trained recognition model.

[0050] The output unit 303 is configured to process the model output and output a recognition result.

[0051] It can be understood that the structure shown in the embodiment of the present application does not constitute a specific limitation on the handwritten name recognition method in a scanned document surface. In another embodiment of the present application, the handwritten name recognition method in a scanned document surface can include more or fewer modules than the figure, or combine certain modules, or split certain modules, or different module arrangement. The modules shown in the figure can be realized by hardware, software or a combination of software and hardware.

[0052] The information interaction, execution process and the like between the modules in the device are based on the same concept as the method embodiment of the present application, and the specific content can be referred to the description in the method embodiment of the present application, which will not be described here.

[0053] The embodiment of the present application also provides a computing device comprising a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, the method for recognizing handwritten names in a scanned document surface in any embodiment of the present application is realized.

[0054] The embodiment of the present application also provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program enables a processor to execute the method for recognizing a handwritten name in a volume surface according to any one of the embodiments of the present application when the computer program is executed by the processor.

[0055] Specifically, a system or an apparatus provided with a storage medium storing software program codes for realizing the functions of any one of the above embodiments can be provided, and a computer (or CPU or MPU) of the system or the apparatus reads out and executes the program codes stored in the storage medium.

[0056] In this case, the program codes read from the storage medium can realize the functions of any one of the above embodiments by themselves, and thus the program codes and the storage medium storing the program codes constitute a part of the present application.

[0057] The embodiments of the storage medium for providing the program codes include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), magnetic tapes, nonvolatile memory cards, and ROMs. Alternatively, the program codes can be downloaded from a server computer via a communication network.

[0058] In addition, it should be clear that not only the program codes read by the computer can be executed, but also part or all of the actual operations can be completed by operating systems and the like operating on the computer based on the instructions of the program codes, so as to realize the functions of any one of the above embodiments.

[0059] In addition, it should be understood that the program codes read from the storage medium can be written into memories provided in extension boards inserted into the computer or memories provided in extension modules connected to the computer, and then part or all of the actual operations can be executed by CPUs and the like installed in the extension boards or the extension modules based on the instructions of the program codes, so as to realize the functions of any one of the above embodiments.

[0060] It is to be noted that, in the present text, the relational terms such as first and second, and the like, are used solely to distinguish one entity or action from another, without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element preceded by "comprises... a" does not, without more constraints, foreclose the existence of additional identical elements in the process, method, article, or apparatus that comprises the recited element.

[0061] It is to be understood by those skilled in the art that all or part of the steps of the above-mentioned method embodiments can be completed by program instruction-related hardware, and the aforementioned program can be stored in a computer-readable storage medium, which, when executed, performs steps including the above-mentioned method embodiments; and the aforementioned storage medium includes ROM, RAM, magnetic disc or optical disc, and various storage media that can store program codes.

[0062] Finally, it is to be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit the same; even though the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements to some technical features thereof; and such modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for recognizing handwritten names in scanned documents, characterized in that: include: Get the handwritten name area image; Preprocessing the handwritten name area image to obtain a preprocessed image I; For each candidate name s, the name s is printed on an image J, which is the candidate name image. Then, I is input into the convolutional neural network 1 and the convolutional neural network 2 respectively to obtain output vectors v1 and v2. Image J is input into the convolutional neural network 1 and the convolutional neural network 3 respectively to obtain output vectors u1 and u2. I and J are then merged in the channel dimension and input into the convolutional neural network 4 to obtain a similarity value p3. Then, the similarity value p1 between the vector v1 and the vector u1 and the similarity value p2 between the vector v2 and the vector u2 are calculated. Finally, the similarity between the candidate name s and the handwritten name to be recognized is calculated based on the values ​​of p1, p2 and p3. The convolutional neural network 1, the convolutional neural network 2, the convolutional neural network 3 and the convolutional neural network 4 are all obtained by deep learning training using multiple sets of data, and the multiple sets of data include training images and expected similarity values. From all candidate names, find the name that has the greatest similarity to the handwritten name as the recognition result.

2. The method according to claim 1, characterized in that The step of obtaining the handwritten name area image comprises: According to the pre-calibrated handwritten name area position, the handwritten name area image is cut out from the scanned paper image.

3. The method according to claim 1, characterized in that The pre-processing steps include: If the scanned image is black text on a white background, the handwritten name area image is first inverted to obtain a handwritten image A with white text on a black background; otherwise, the handwritten name area image is directly assigned to image A; the background value of the obtained image A is 0, and the foreground value is 1; Accumulate image A by rows to obtain vector r, and by columns to obtain vector c. If r > 0, find the starting and ending row numbers of the handwritten name area. Calculate the absolute change d of the accumulated values ​​of adjacent columns based on c. If d > 1, find the starting and ending column numbers of the handwritten name area. Based on the starting and ending rows and columns, cut image block B from image A. The preprocessed image I is obtained by normalizing the image B to a predefined size.

4. The method according to claim 1, wherein The convolutional neural network 1 is a branch network in a twin neural network structure. The twin neural network structure includes two network branches. The two network branches share network structure and parameters. The inputs of the two network branches are handwritten name images and candidate name images, respectively. The outputs of the two network branches are both feature vectors.

5. The method according to claim 1, wherein The convolutional neural network 2 and the convolutional neural network 3 are two branch networks in the twin neural network structure, but the convolutional neural network 2 and the convolutional neural network 3 do not share parameters. The inputs of the two network branches are the handwritten name image and the candidate name image respectively, and the outputs of the two network branches are both feature vectors.

6. The method according to claim 1, characterized in that The convolutional neural network 4 includes a convolutional neural network structure and a fully connected network structure. The input of the convolutional neural network 4 is a dual-channel image after the handwritten name image and the candidate name image are fused in the channel dimension. The output of the convolutional neural network structure is a feature vector. After the fully connected network structure, the similarity between the handwritten name image and the candidate name image is obtained.

7. The method according to claim 1, characterized in that The step of calculating the similarity between vector u and vector v includes: first calculating the difference vector w between vector u and vector v, and then sending w to a fully connected network, and the output of the fully connected network represents the similarity between vector u and vector v.

8. A device for recognizing handwritten names in a scanned document, characterized in that: include: an acquisition unit, configured to acquire an image to be recognized; An input unit, configured to pre-process the image and input the image into a pre-trained recognition model; The output unit is used to process the model output and output the recognition result.

9. A computing device comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute the method according to any one of claims 1 to 7.

Images