Matrix character key point detection method and device

Through supervised learning and unsupervised clustering combined with deep learning neural network, the problem of inaccurate key points of complex structures and large font characters in OCR character recognition is solved, and more accurate key point recognition is achieved, simplifying the process and improving the effect.

CN118918594APending Publication Date: 2024-11-08ZHEJIANG UNIV
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202410905664.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-08
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

The existing OCR character key point recognition technology is prone to errors when dealing with complex structures and large font characters. The generated key points are not centered enough and it is difficult to accurately identify them.

Method used

Supervised learning is used to obtain the range of character key points, and the key point coordinates are accurately obtained through unsupervised clustering algorithm, combining deep learning neural network training and unsupervised clustering methods, including steps such as obtaining pixel matrix graphs, labeling key points, training models, Predict the coordinates of key points and clustering centers.

Benefits of technology

The accuracy of the recognition of key points of complex structural characters and large font characters is improved, ensuring that the key points are closer to the real position, the method is simple and the effect is better than the existing technology, and it is versatile.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118918594A_ABST
    Figure CN118918594A_ABST
Patent Text Reader

Abstract

The invention discloses a matrix type character key point detection method and device, and the method comprises the steps: training a plurality of key point clusters based on supervised learning, and obtaining the center coordinates of each key point of a character through unsupervised clustering, and comprises the steps: (1) mapping an OCR character graph to a pixel matrix of a fixed specification; (2) setting a threshold parameter to binarize the pixel matrix to obtain a character pixel map; (3) marking a plurality of pixels at a stroke starting point, a stroke ending point and a stroke intersection point in the character pixel map, and recording the number of clusters; (4) learning a plurality of cluster key points of the pixel matrix by using a deep learning neural network; and (5) obtaining a key point center of each cluster by using an unsupervised clustering algorithm. The method has the advantages of being high in accuracy, low in delay and high in robustness, can be used for a conventional character type braille board, can assist the blind in skipping learning braille and directly learning and understanding conventional characters and other product function applications, and has good commercialization prospects.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of computer vision target recognition, and in particular to a matrix-type character key point detection method and device. Background Art

[0002] The existing OCR character key point recognition technology still relies on the principles of computer vision and analyzes the structure of the characters. This type of technology will produce errors when learning characters with more complex structures. The current deep learning algorithm is trained based on edge features and inflection point features, but for large-size characters, the key points it generates are still not centered enough. Therefore, the problem of dealing with complex structure characters and large-size characters still needs to be solved. With the development of supervised learning and unsupervised learning, supervised learning can be used to obtain a small area prediction range and then unsupervised precise clustering can be performed to obtain accurate key point coordinates. Summary of the invention

[0003] The purpose of the present invention is to provide a matrix character key point detection method and device to address the deficiencies of the prior art. The present invention adopts supervised learning of key point ranges and then performs unsupervised clustering to obtain the key point coordinates of the characters.

[0004] A first aspect of the present invention relates to a matrix character key point detection method, comprising the following steps:

[0005] (1) Obtaining the pixel matrix of OCR characters. Call the computer's built-in font, place it in the center of the preset matrix size, and generate a pixel matrix;

[0006] (1.1) Call computer character images through the PIL library in Python. The fonts include but are not limited to Kaiti, Songti, Times New Roman, etc., and the font sizes include [5,72];

[0007] (1.2) The value range of the rectangle side length of the preset matrix size is [10,100]. The rectangle side length value must be larger than the font size, and the matrix values ​​are all 0;

[0008] (1.3) Get the length and width of the called character image and calculate its center coordinates;

[0009] (1.4) placing the center coordinates of step (1.3) at the center of the preset matrix, and superimposing its character image value to obtain a pixel matrix image;

[0010] (2) Character pixel map production. The default value of the binarization threshold is 255, and the value range is [128, 255]. The pixel matrix of step (1.4) Figure 2 Binarization: the pixel value greater than the binarization threshold is set to 0;

[0011] (3) Marking several cluster key points. Based on the character pixel map obtained in step (2), mark several pixel points at the starting point, end point, and intersection of all strokes of the character, and record the coordinates of the marked pixel points and the number of clusters as labels;

[0012] (3.1) Mark several clusters of key points, and mark all the starting points, end points,

[0013] The random position pixel at the intersection position, the random position number range is [1,10], the default value is 3;

[0014] (3.2) Record the matrix coordinates of several pixel points and the number of clusters in step (3.1) and save them as a label document;

[0015] (4) Training character pixel images and several cluster key points. The character pixel image of step (2) and the label of step (3.2) are put into the deep learning neural network training to obtain the trained model.

[0016] (4.1) Use a deep learning network to train character pixel maps and several cluster key points.

[0017] The network includes but is not limited to convolutional neural networks, Transformer networks, etc.;

[0018] (4.2) Simultaneously train the positions of several key points and their cluster numbers to obtain the network training model;

[0019] (4.3) For the key point character to be detected, the steps (1) to (2) are performed to obtain a pixel character map to be detected, and the model trained in step (4.2) is used to predict the key point coordinates and the number of clusters of the pixel character map to be detected;

[0020] (5) Unsupervised clustering to obtain the center of each key point. The model obtained in step (4.3) is tested on the key point characters to obtain a number of key pixel points, and then an unsupervised clustering algorithm is used to obtain the center coordinates of the key points of the characters.

[0021] (5.1) Unsupervised clustering methods include but are not limited to K-means, hierarchical clustering, density clustering, etc.;

[0022] (5.2) Use the cluster center and several coordinates from step (5.3) and the clustering method used in step (6.1) to obtain the coordinates of the character key points.

[0023] The second aspect of the present invention relates to a matrix-type character key point detection device, comprising a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement a matrix-type character key point detection method of the present invention.

[0024] A third aspect of the present invention relates to a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements a matrix-type character key point detection method of the present invention.

[0025] The beneficial effect of the present invention is that supervised learning of the key point range through a deep learning neural network can ensure that the key points are covered to deal with the key point recognition problem of complex structured characters. Unsupervised clustering can ensure that the key points obtained by the present invention when dealing with large characters are more in line with the real key point positions. The present invention proposes to first perform supervised learning to obtain a small area target, and then obtain accurate key point coordinates through unsupervised clustering. The method is simple, the effect is better than the existing method, and it has the characteristics of versatility. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 is a step diagram of the method of the present invention;

[0027] Figure 2 is a pixel matrix diagram of the method of the present invention;

[0028] Figure 3 is a character pixel map of the method of the present invention;

[0029] Figure 4 is a supervised learning result graph of the method of the present invention;

[0030] Figure 5 It is the unsupervised result graph of the method of the present invention. DETAILED DESCRIPTION

[0031] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.

[0032] Example 1

[0033] The core technology of the present invention is to use deep learning neural network supervised learning and unsupervised learning to obtain the coordinates of character key points.

[0034] The present invention proposes a matrix character key point detection method, comprising the following steps: Figure 1 ):

[0035] (1) Obtaining the pixel matrix of OCR characters. Call the computer's built-in font and place it in the center of the preset matrix size to generate a pixel matrix ( Figure 2 );

[0036] (1.1) Call computer character images through the PIL library in Python. The fonts include but are not limited to Kaiti, Songti, Times New Roman, etc. The font size includes [5,72], and the default value is 10;

[0037] (1.2) The value range of the rectangle side length of the preset matrix size is [10,100], the default value is 20, the rectangle side length value must be larger than the font size, and the matrix values ​​are all 0;

[0038] (1.3) Get the length and width of the called character image and calculate its center coordinates;

[0039] (1.4) placing the center coordinates of step (1.3) at the center of the preset matrix, and superimposing its character image value to obtain a pixel matrix image;

[0040] (2) Character pixel map production. The default value of the binarization threshold is 255, and the value range is [128, 255]. The pixel matrix of step (1.4) Figure 2 The pixel value greater than the binary threshold is set to 0 ( Figure 3 );

[0041] Mark several cluster key points. Based on the character pixel map obtained in step (2), mark several pixel points at the starting point, end point, and intersection of all strokes of the character, and record the coordinates of the marked pixel points and the number of clusters as labels;

[0042] (3.1) Mark several cluster key points, and mark the random position pixels at all starting points, end points, and intersections of the character pixel map in step (2). The number of random positions ranges from [1,10], and the default value is 3;

[0043] (3.2) Record the matrix coordinates of several pixel points and the number of clusters in step (3.1) and save them as a label document;

[0044] Training character pixel map and several cluster key points. Put the character pixel map of step (2) and the label of step (3.2) into the deep learning neural network training to obtain the trained model.

[0045] (4.1) Use a deep learning network to train a character pixel map and a number of cluster key points, wherein the deep learning network includes but is not limited to a convolutional neural network, a Transformer network, etc.;

[0046] (4.2) Simultaneously train the positions of several key points and their cluster numbers to obtain the network training model;

[0047] (4.3) Perform steps (1) to (2) to obtain a pixel character map to be detected for the key point character to be detected, and use the model trained in step (4.2) to predict the key point coordinates and cluster number of the pixel character map to be detected ( Figure 4 );

[0048] (5) Unsupervised clustering to obtain the center of each key point. The model obtained in step (4.3) is tested on the key point characters to obtain a number of key pixel points, and then an unsupervised clustering algorithm is used to obtain the center coordinates of the key points of the characters.

[0049] (5.1) Unsupervised clustering methods include but are not limited to K-means, hierarchical clustering, density clustering, etc.;

[0050] (5.2) Use the cluster center and several coordinates from step (5.3) and the clustering method used in step (6.1) to obtain the coordinates of the key points of the characters. Figure 5 )

[0051] Example 2

[0052] The present embodiment relates to a matrix-type character key point detection device, including a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement a matrix-type character key point detection method of embodiment 1.

[0053] Example 3

[0054] The present embodiment relates to a computer-readable storage medium having a program stored thereon. When the program is executed by a processor, a matrix-type character key point detection method of embodiment 1 is implemented.

[0055] The contents described in the embodiments of this specification are merely an enumeration of the implementation forms of the inventive concept. The protection scope of the present invention should not be regarded as limited to the specific forms described in the embodiments. The protection scope of the present invention also extends to equivalent technical means that can be conceived by those skilled in the art based on the inventive concept.

Claims

1. A matrix character key point detection method, characterized in that: The steps include: (1) Obtaining the pixel matrix image of OCR characters; calling the computer's built-in font, placing it in the center of the preset matrix size, and generating a pixel matrix image; (2) Creating a character pixel map: Setting a threshold value, binarizing the pixel matrix map in step (1) to obtain a character pixel map; (3) marking a number of cluster key points; based on the character pixel map obtained in step (2), marking a number of pixel points at the starting point, end point, and intersection of all strokes of the character, and recording the coordinates of the marked pixel points and the number of clusters as labels; (4) training the character pixel map and several cluster key points; putting the character pixel map of step (2) and the label of step (3) into the deep learning neural network training to obtain the trained model; (5) Unsupervised clustering is used to obtain the centers of each key point; the trained model obtained in step (4) is used to test the key point characters to be detected to obtain a number of key pixel points, and then an unsupervised clustering algorithm is used to obtain the center coordinates of the key points of the characters.

2. A matrix character key point detection method according to claim 1, characterized in that: The OCR character pixel matrix image acquisition described in step (1) includes the following sub-steps: (1.1) Call computer character images through the PIL library in Python. The fonts include but are not limited to Kaiti, Songti, Times New Roman, etc. The font sizes include [5,72]; (1.2) The value range of the rectangle side length of the preset matrix size is [10,100]. The rectangle side length value must be larger than the font size, and the matrix values ​​are all 0; (1.3) Get the length and width of the called character image and calculate its center coordinates; (1.4) Place the center coordinates at the center of the preset matrix and superimpose its character image value to obtain a pixel matrix image.

3. A matrix character key point detection method according to claim 1, characterized in that: The character pixel map production described in step (2) includes: The default value of the binarization threshold is 255, and the value range is [128, 255]. The pixel matrix image is binarized, and the pixel values ​​greater than the binarization threshold are set to 0.

4. A matrix character key point detection method according to claim 1, characterized in that: The step (3) of marking a plurality of cluster key points includes the following sub-steps: (3.1) Mark several clusters of key points, and mark the random position pixels at all starting points, end points, and intersections of the character pixel map. The number of random positions ranges from [1,10], and the default value is 3; (3.2) Record the matrix coordinates of the pixel points and the number of clusters and save them as a label document.

5. A matrix character key point detection method according to claim 1, characterized in that: The step (4) of training the character pixel map and a plurality of cluster key points comprises the following sub-steps: (,4.1) Use a deep learning network to train character pixel maps and several cluster key points, and its deep learning network includes convolutional neural network and Transformer network; (4.2) Simultaneously train the positions of several key points and their cluster numbers to obtain the network training model; (4.3) For the key point characters to be detected, steps (1) to (3) are performed to obtain a pixel character map to be detected, and the trained model is used to predict the key point coordinates and the number of clusters of the pixel character map to be detected.

6. A matrix character key point detection method according to claim 5, characterized in that: The step (5) of obtaining the centers of each key point by unsupervised clustering includes the following sub-steps: (5.1) Unsupervised clustering methods include but are not limited to K-means, hierarchical clustering, density clustering, etc.; (5.2) Use the cluster center and several coordinates from step (4.3) and the clustering method used in step (5.1) to obtain the coordinates of the character key points.

7. A matrix character key point detection device, characterized in that: It comprises a memory and one or more processors, wherein the memory stores executable codes, and when the one or more processors execute the executable codes, they are used to implement a matrix character key point detection method according to any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that: A program is stored thereon, and when the program is executed by a processor, a matrix character key point detection method according to any one of claims 1 to 6 is implemented.

Citation Information

Patent Citations

  • Character recognition method and device and electronic equipment

    CN111414913A

  • Text recognition method and device, equipment and storage medium

    CN115063809A

  • Stroke similarity evaluation method and system for hard-tipped regular script imitated works

    CN115984875A

  • Handwriting recognition pipelines for genealogical records

    US20230010202A1

  • Model training method and apparatus, device, medium, and program product

    US20230290003A1