Tiny character recognition analysis method, system, equipment and medium
By combining character segmentation and deep learning models based on RGB values, the problem of low accuracy of micro character recognition in the existing technology is solved, and efficient recognition and semantic analysis of 10x10 or even 5x5 characters is achieved, which improves recognition accuracy and efficiency .
Patent Information
- Application Number
- CN202411076637.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-07
- Publication Date
- 2024-11-22
AI Technical Summary
The prior art is difficult to identify and interpret tiny characters below 10x10 pixels, the recognition accuracy is low, the semantics cannot be restored, and it cannot be widely used in manufacturing, quality control, safety, medical care and other fields.
Using a character segmentation method based on RGB values, combined with deep learning models with different convolution kernel sizes, analyze character spatial location and semantics, recognize and logical combination of tiny characters, and train the model using the adam optimizer and MSE loss function to realize the end To-end tiny character recognition.
The accuracy of tiny character recognition is improved, and the inefficiency and error problems of traditional optical character recognition and object detection algorithms are solved in recognition of tiny characters, and efficient recognition and semantic analysis of 10x10 or even 5x5 characters are achieved.
Smart Images

Figure CN119007223A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of character recognition, and in particular relates to a method, system, device and medium for analyzing tiny character recognition. Background Art
[0002] Micro character recognition technology is a sophisticated image processing and computer vision technology that is specifically designed to recognize and interpret tiny characters in images with a maximum size of 10x10 pixels. By using high-resolution camera technology, advanced image processing algorithms and deep learning models, this technology can achieve extremely high performance and accuracy in complex backgrounds and character deformation. Its core technologies include optical character recognition (OCR), image preprocessing, feature extraction and classification, making micro character recognition widely used in manufacturing, quality control, security, medical care and logistics.
[0003] In the early days, people used their naked eyes to distinguish tiny characters. With the development of computers, deep learning was proposed to recognize characters, but the minimum recognition range was limited to about 30*30 pixels. These methods could not recognize characters of 10*10 or even 5*5 on a large scale. The recognition accuracy was very low, and the semantics could not be restored, so they were basically unusable. Summary of the invention
[0004] In order to overcome the above problems existing in the prior art, the present invention proposes a method, system, device and medium for tiny character recognition and analysis. The technical solution is as follows:
[0005] A method for identifying and analyzing tiny characters comprises the following steps:
[0006] S1. Obtain different pictures containing tiny characters;
[0007] S2. Based on the page RGB value with 255 as the boundary, split the characters into the smallest symbols in the x and y directions, and record the coordinates of the upper left and lower right points of the character space;
[0008] S3. Analyze the pixel size of tiny characters, set the pixel size of tiny characters to be less than Q*Q pixels, and fill the surrounding of tiny characters less than Q*Q pixels with 0;
[0009] S4. Divide the data into training set, validation set, and test set, and preprocess the data;
[0010] S5. Build a deep learning model;
[0011] S6. Based on the adam model optimizer and the MSEloss loss function, the model constructed in S5 is trained using the training set obtained in step 4, and the model is verified using the validation set, and the model performance is tested using the test set;
[0012] S7. Based on the spatial position of the characters obtained in S2, analyze the semantics of the blank character and combine the characters according to the correct logic.
[0013] Preferably, in S2, character segmentation is performed: its characteristic is that under the RGB feature, the RGB value of the pure white background is 255, the characters are arranged horizontally, and the position of the characters in the overall image is unknown. After horizontal segmentation by 255, vertical segmentation is performed line by line, and the coordinate positions of the two points in the upper left and lower right of the entire page are recorded.
[0014] Preferably, in S2, the conditions for determining whether segmentation is required are as follows: the horizontal and vertical lines are segmented by 255 bright lines, the minimum is 1, the maximum is 10, and the characters need to be absolutely segmented; after black and white conversion, it can be determined by the following formula:
[0015]
[0016] Where img is the image; i is the index row, and j is the index column; all row and column segmentation lines are stored in an array, and the x and y coordinate positions after segmentation are recorded for semantic re-synthesis.
[0017] Preferably, in S3, the pixels constituting the image are padded with 0 on the periphery to form a 10*10 two-dimensional array; the numbers need to be padded evenly on both sides, and odd numbers are preferentially padded on the left side.
[0018] Preferably, the formula for constructing a deep learning model is as follows:
[0019]
[0020] Where Y(i,j) is the convolution Y at position i,j, img(i+u,j+v) represents the pixel value of the image at position i+u,j+v, and K(u,v) represents the weight of the convolution kernel at position (u,v);
[0021] S51. Use different convolution kernel sizes to collect weights of different kernel sizes under the same image, use three convolution kernels of kernel_size=1, kernel_size=3, kernel_size=5 to collect the convolution information Y1, Y2, Y3 of the data, and then use concat to connect them into a whole, that is:
[0022] R = concat([Y1,Y2,Y3]);
[0023] S52. Use the reshape function to convert the data into a one-dimensional array, plus the batch dimension, a total of 2 dimensions. The process is: suppose the three-dimensional array is R[b,q,r], and the converted array is R'[b,q*r], where b is the batchsize, q and r are the dimensions of the image after passing through the convolution layer;
[0024] S53. Use MLP multi-layer perceptron to transform the reduced one-dimensional features to obtain the target result, wherein the loss function uses mean square error MSE to determine the error between the true value and the predicted value;
[0025]
[0026] N = b;
[0027]
[0028] Where W1-W4 is the weight of each layer, b1-b4 is the bias of each layer; Predicted value; y r is the true value, The feature vector value for each layer input.
[0029] Preferably, in S7, the character pronunciation is obtained based on the character spatial position, and the specific processing process is: based on the x and y coordinates obtained by the character, its row affiliation is determined in the y direction, the character size is judged by the left and right distance of the x coordinate, the character spacing is calculated by the x coordinate, and based on the spacing relationship, it is determined whether it belongs to the context of the same sentence, or the relationship between different phrases or different sentences.
[0030] Preferably, in S7, according to the correct logic combination characters, the steps are as follows:
[0031] arr_tmp=[x1, y1, x2, y2, ind_n, pred]
[0032] arr = concat(arr, arr_tmp)
[0033] res = ana(arr)
[0034] Among them, x1, y1, x2, y2 are the data of the upper left and lower right corners of the original document, representing the image location information, ind_n is the predicted index of the image, representing the predicted index, and pred is the predicted demerit;
[0035] ana is a prediction algorithm, and its principle is:
[0036] Using a q_state variable, based on the values of x1, y1, x2, and y2, dynamically adjust the state and set q_state to two different semantic states, 'a_state' and 'b_state'.
[0037] ind_n is an auxiliary judgment information: Set two variables, a_pred and b_pred, to store the cumulative character results in the two states of a_state and b_state; a_pred is the label and b_pred is the result character, and store these two results in the res result array.
[0038] With V tmp Represent the current position information (x1, y1, x2, y2), and its detailed logic is as follows:
[0039] q_state = a_state and |x2 - v tmp [i + 1][x1]| ≤ 6, q_state remains unchanged
[0040] q_state = a_state and |x2 - v tmp [i + 1][x1]| > 6, q_state changes to b_state
[0041] q_state = b_state and |x2 - c tmp [i + 1][x1]| ≤ 6, q_state remains unchanged
[0042] q_state = b_state and |x2 - v tmp [i + 1][x1]| >, q_state changes to a_state
[0043] When the state b_state changes to a_state, it means that the character semantic recognition of a group is completed. Save the two values of a_pred and b_pred to res and clear them.
[0044] A micro character recognition and analysis system includes a data acquisition module, a data processing module, a model preparation module, a data inference module, and a data integration module.
[0045] The data acquisition module is used to acquire pictures containing micro character data; the data processing module is used to preprocess the pictures to obtain micro character data; the model preparation module is used to build a deep learning model and train and verify the model; the data inference module is used to analyze and process the preprocessed micro character data; the data integration module is used to output the analysis results.
[0046] A computer device includes at least one processor and a computer-readable medium storing a computer program. When the computer program is read and executed by the processor, the method of the present application is implemented.
[0047] A computer-readable medium stores a computer program, and when the computer program is read and executed by a processor, the method of the present application is implemented.
[0048] Compared with the prior art, the present invention has the following beneficial effects:
[0049] The beneficial effects of the present invention are that it introduces the features extracted by different convolution kernels into the recognition of tiny characters based on RGB255 character segmentation, and analyzes the meaning of tiny characters by spatial position and empty character semantics. Compared with the traditional optical character recognition, the present invention adopts a deep learning model more suitable for tiny characters and a special image segmentation method, which solves the problem of low recognition rate, large error, difficult analysis and slow speed of tiny characters on large panels by traditional algorithms and traditional target detection algorithms from end to end. BRIEF DESCRIPTION OF THE DRAWINGS
[0050] The present invention is further described below in conjunction with the accompanying drawings and embodiments.
[0051] Figure 1 This is a diagram of the deep model architecture of the present invention;
[0052] Figure 2(a) is a segmentation map based on RGB;
[0053] Figure 2(b) shows the comparison between the inference results of the deep learning model and the actual results;
[0054] Figure 2(c) Semantic summary graph based on coordinate points. DETAILED DESCRIPTION
[0055] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention is described in detail below in conjunction with the accompanying drawings and specific implementation methods.
[0056] This embodiment discloses a method for tiny character recognition and analysis based on deep learning, comprising the following steps:
[0057] Step 1, obtain different pictures containing tiny characters, the character size accounts for a very low proportion of the page, and the number is large.
[0058] Images with tiny characters refer to images with a size of 1080*1080 that contain dozens or hundreds of characters of a size of about 8*8 or even 5*5. Usually, a 12pt small character occupies about 20*20 pixels.
[0059] If the characters are too small in size, several problems will occur:
[0060] The first reason is that the characters are small and the spacing between them is also small. Usually the minimum spacing is only 1 pixel. This is because the traditional YOLO series (target detection, target segmentation model) or traditional OCR (optical character recognition) are based on a combination of multiple layers of convolutions. The target is determined by comprehensive judgment in large and small fields of view. The minimum recognition accuracy is generally 20*20, and convolution can easily confuse 1-pixel character segmentation when extracting features. Therefore, it is difficult to find and complete tiny character recognition through traditional YOLO series and traditional OCR models.
[0061] Second, different characters may look similar in macroscopic view, but may be different when viewed from close up. The pixels of tiny characters may shift due to font, handwriting conversion, scaling, etc. However, for some closely aligned characters, the shift of a single pixel may cause recognition errors, so a unified and accurate method is needed to authenticate the characters.
[0062] Step 2: Based on the page RGB value with 255 as the boundary, split the characters into the smallest symbols in the x and y directions, and record the coordinates of the upper left and lower right points of the character space, which are (x1, y1) and (x2, y2) respectively;
[0063] Generally, in the RGB system, white is fully bright (255, 255, 255) and black is fully dark (0, 0, 0). Half of the character is black, and the blank is white. But when we need to recognize the character, we need to convert the character from (0, 0, 0) to (255, 255, 255). Convert white from (255, 255, 255) to (0, 0, 0). This is because when the data is passed into the deep learning model, the matrix multiplication will reduce the influence of the 0 value part. Therefore, in some pictures, there will be a lot of black lines, so we need to do very accurate recognition of the characters based on 255.
[0064] We identify and delete black lines by determining that the row value of the black line is 0. That is, when sum(img[:_k,:]) == 0 is satisfied, it is considered to be a black line and the character is deleted. The three dimensions of the img matrix are [x, y, channel], where xy represents the horizontal and vertical coordinates, and channel represents the RGB dimension. Because the image is black and white, RGB can be replaced by gray level, and any RGB dimension is taken as the gray level dimension. That is, channel = 1.
[0065] The determination of whether segmentation is needed can be solved by a set of mathematical algorithms. The core is that the horizontal and vertical lines are segmented by 255 bright lines, with a minimum of 1 and a maximum of 10. We need to absolutely segment the characters. After black and white conversion, it can be determined by the following formula, where img is the image, i and j are rows and columns, n is the maximum number of columns, and m is the maximum number of rows.
[0066]
[0067] Similarly, the column dividing lines can also be obtained, all row and column dividing lines are stored in an array, and the xy coordinate positions after segmentation are recorded for semantic re-synthesis.
[0068] Step 3, analyze the pixel size of tiny characters. In this example, the pixels occupied by tiny characters are all less than 10 pixels * 10 pixels. The tiny characters less than 10 * 10 pixels are filled with 0 using the filling method.
[0069] When the input content of the deep learning model is fixed, its output will be more stable and more convenient for model design. Because in this example, 255 and 0 are swapped in step 2 to ensure that the valid information is 255, adding 0 to fill in the back will not change the information. The specific operation is:
[0070] img[i,j] is each pixel in the image, m is the maximum number of rows, n is the maximum number of columns, i is the index row, j is the index column, img: original image.
[0071] target_height=10: target height; target_width=10: target width.
[0072] padding_leftpadding: padding on the left; padding_right: padding on the right.
[0073] padding_top: top padding; padding_bottom: bottom padding.
[0074]
[0075] padding_right=target_width-n-padding_left
[0076]
[0077] padding_bottom=target_height-m-padding_top
[0078]
[0079] Step 4: Divide the data into training set, validation set, and test set, and preprocess the data;
[0080] Construct the training machine, validation machine, and test set in the 7:1:2 manner. Set up the prepare(args) function to pass parameters and map specific labels and characters in the dataset one by one.
[0081] Step 5: Build a deep learning model. The model architecture is shown in Figure 1 ;
[0082] In this implementation, the deep learning model includes convolution operation modules with different convolution kernel sizes, feature dimension reduction modules after convolution, feature fusion modules, MLP multi-layer perceptron modules, etc. The convolution operation modules with different convolution kernel sizes output cognitive features within different visual fields of the same image; the feature dimension reduction module reduces the features to one dimension for easy fusion, and the feature fusion module fuses the features of each visual field; the MLP multi-layer perceptron module converts the features into the numerical value of the target result through two fully connected layers, where the loss function is Relu.
[0083] The mathematical formula of convolution operation CNN is as follows:
[0084]
[0085] Where Y(i,j) is the convolution Y at position i,j, img(i+u,j+v) represents the pixel value of the image at position i+u,j+v, K(u,v) represents the weight of the convolution kernel at position (u,v); n,m are the maximum values of the rows and columns of img.
[0086] First, use different convolution kernel sizes to collect weights of different kernel sizes under the same image, use three convolution kernels of kernel_size=1, kernel_size=3, kernel_size=5 to collect the convolution information Y1, Y2, Y3 of the data, and then use concat to connect them as a whole, that is:
[0087] R = concat([Y1,Y2,Y3]);
[0088] Use the reshape function to convert the data into a one-dimensional array, plus the batch dimension for a total of 2 dimensions. The process is: suppose the original three-dimensional array is R[b,q,r], and the converted array is R'[b,q*r], where b is the batchsize, q and r are the dimensions of the image after passing through the convolution layer.
[0089] Finally, the MLP multi-layer perceptron is used to transform the reduced one-dimensional features into the target result, where the loss function uses the mean square error MSE to determine the error between the true value and the predicted value.
[0090]
[0091] N = b;
[0092] The activation function is the rectified linear unit ReLU
[0093] ReLU(x)=max(0,x)
[0094] MLP adopts a 4-layer structure. The matrix dimensions of the fully connected layer are:
[0095] (104,4096),(4096,512),(512,32),(32,1)
[0096]
[0097] Where W1-W4 is the weight of each layer, b1-b4 is the bias of each layer; Predicted value; y r is the true value, The feature vector value for each layer input.
[0098] In this embodiment, the specific parameters of the model are as follows:
[0099] inputsize=(10,10)
[0100] kernel_size1=1
[0101] kernel_size2=3
[0102] kernel_size3=5
[0103] hiddensize1=4096
[0104] hiddensize2=512
[0105] hiddensize3=32
[0106] output:1
[0107] Step 6: Based on the adam model optimizer and the MSEloss loss function, the model constructed in step 5 is trained using the training set obtained in step 4, and the model is verified using the validation set, and the model performance is tested using the test set.
[0108] After testing, the model's accuracy on all known characters is close to 100%. We used multiple rounds of data supplementation to cover all characters and all forms in the training set.
[0109] Step 7: Based on the character space positions obtained in Step 2, analyze the semantics of the null characters and combine the characters according to the correct logic.
[0110] arr_tmp = [x1, y1, x2, y2, ind_n, pred]
[0111] arr = concat(arr, arr_tmp)
[0112] res = ana(arr)
[0113] Among them, x1, y1, x2, and y2 are the upper left and lower right corner data of the numbers in the original document, representing the picture position information. ind_n is the predicted index quantity of the picture, representing the predicted index, and pred is the prediction result.
[0114] ana is a prediction algorithm. Its core mathematical principle is to use a q_state state quantity and dynamically adjust the state based on the values of x1, y1, x2, and y2. The q_state can be set to
[0115] two different semantic states of 'a_state' and 'b_state'. ind_n is an auxiliary judgment information. Set a_pred
[0116] and b_pred two variables to store the cumulative character results in the two states of a_state and b_state. a_pred is the label, and b_pred is the result character. Finally, store these two results in the res result array for convenient later use. Let Vtmp represent the current position information (x1, y1, x2, y2). Its detailed logic is as follows:
[0117] q_state = a_state and |x2 - v tmp |[i + 1][x1]| ≤ 6, q_state remains unchanged
[0118] q_state = a_state and |x2 - v tmp [i + 1][x1]| > 6, q_state changes to b_state
[0119] q_state = b_state and |x2 - v tmp [i + 1][x1]| ≤ 6, q_state remains unchanged
[0120] q_state = b_state and |x2 - v tmp [i + 1][x1]| >, q_state changes to a_state
[0121] When the state b_state changes to a_state, it means that the semantic recognition of a group of characters is completed, and the values of a_pred and b_pred are saved in res and cleared to zero.
[0122] The above embodiments are only exemplary embodiments of the present invention and are not intended to limit the present invention. The protection scope of the present invention is defined by the claims. Those skilled in the art may make various modifications or equivalent substitutions to the present invention within the essence and protection scope of the present invention, and such modifications or equivalent substitutions shall also be deemed to fall within the protection scope of the present invention.
Claims
1. A method for identifying and analyzing tiny characters, characterized in that: The following steps are involved: S1. Obtain different pictures containing tiny characters; S2. Based on the page RGB value with 255 as the boundary, split the characters into the smallest symbols in the x and y directions, and record the coordinates of the upper left and lower right points of the character space; S3. Analyze the pixel size of tiny characters, set the pixel size of tiny characters to be less than Q*Q pixels, and fill the surrounding of tiny characters less than Q*Q pixels with 0; S4. Divide the data into training set, validation set, and test set, and preprocess the data; S5. Build a deep learning model; S6. Based on the adam model optimizer and the MSEloss loss function, the model constructed in S5 is trained using the training set obtained in step 4, and the model is verified using the validation set, and the model performance is tested using the test set; S7. Based on the spatial position of the characters obtained in S2, analyze the semantics of the blank character and combine the characters according to the correct logic.
2. The micro character recognition and analysis method according to claim 1, characterized in that: In S2, character segmentation is performed: under the RGB feature, the RGB value of the pure white background is 255, the characters are arranged horizontally, and their position in the overall image is unknown. After horizontal segmentation by 255, vertical segmentation is performed line by line, and the coordinate positions of the upper left and lower right points of the entire page are recorded.
3. The method for identifying and analyzing tiny characters according to claim 1, characterized in that: In S2, the conditions for determining whether segmentation is required are as follows: the horizontal and vertical lines are segmented by 255 bright lines, the minimum is 1, the maximum is 10, and the characters need to be absolutely segmented; after black and white conversion, it can be determined by the following formula: Where img is the image; i is the index row, j is the index column; All row and column segmentation lines are stored in an array, and the x and y coordinate positions after segmentation are recorded for semantic re-synthesis.
4. The method for identifying and analyzing tiny characters according to claim 1, characterized in that: In S3, the pixels that make up the image are padded with 0 on the periphery to form a 10*10 two-dimensional array; the numbers need to be padded evenly on both sides, and odd numbers are first padded on the left side.
5. The method for identifying and analyzing micro characters according to claim 1, characterized in that: In S5, the formula for building a deep learning model is as follows: Where Y(i,j) is the convolution Y at position i,j, img(i+u,j+v) represents the pixel value of the image at position i+u,j+v, and K(u,v) represents the weight of the convolution kernel at position (u,v); S51. Use different convolution kernel sizes to collect weights of different kernel sizes under the same image, use three convolution kernels of kernel_size=1, kernel_size=3, kernel_size=5 to collect the convolution information Y1, Y2, Y3 of the data, and then use concat to connect them into a whole, that is: R = concat([Y1,Y2,Y3]); S52. Use the reshape function to convert the data into a one-dimensional array, plus the batch dimension, a total of 2 dimensions. The process is: suppose the three-dimensional array is R[b,q,r], and the converted array is R'[b,q*r], where b is the batchsize, q and r are the dimensions of the image after passing through the convolution layer; S53. Use MLP multi-layer perceptron to transform the reduced one-dimensional features to obtain the target result, wherein the loss function uses mean square error MSE to determine the error between the true value and the predicted value; N = b; Where W1-W4 is the weight of each layer, b1-b4 is the bias of each layer; Predicted value; y r is the true value, The feature vector value for each layer input.
6. The method for identifying and analyzing tiny characters according to claim 1, characterized in that: In S7, the character pronunciation is obtained based on the character spatial position, and the specific processing process is: based on the x and y coordinates obtained by the character, its row affiliation is determined in the y direction, the character size is judged by the left and right distance of the x coordinate, the character spacing is calculated by the x coordinate, and based on the spacing relationship, it is determined whether it belongs to the context of the same sentence, or the relationship between different phrases and different sentences.
7. The method for identifying and analyzing tiny characters according to claim 1, characterized in that: In S7, the steps to combine characters according to the correct logic are as follows: arr_tmp=[x1,y1,x2,y2,ind_n,pred] arr=concat(arr,arr_tmp) res = ana(arr) Among them, x1, y1, x2, y2 are the numbers in the upper left and lower right corners of the original document, representing the image location information, ind_n is the predicted index of the image, representing the predicted index, and pred is the predicted demerit; ana is a prediction algorithm, and its principle is: Using a q_state state variable, based on the values of x1, y1, x2, y2, dynamically adjust the state and set q_state to two different semantic states, 'a_state' and 'b_state'; ind_n is an auxiliary judgment information: set two variables a_pred and b_pred to store the accumulated character results in the two states a_state and b_state; a_pred is the label, b_pred is the result character, and store these two results in the res result array; V tmp Represents the current position information (x1, y1, x2, y2). Its detailed logic is: q_state = a_state and |x2 - v tmp [i + 1][x1]| ≤ 6, q_state remains unchanged q_state = a_state and |x2 - v tmp [i + 1][x1]| > 6, q_state changes to b_state q_state = b_state and |x2-v tmp [i+1][x1]|≤6, q_state remains unchanged q_state = b_state and |x2-v tmp [i+1][x1]|>6, q_state becomes a_state When the state b_state changes to a_state, it means that the semantic recognition of a group of characters is completed, and the values of a_pred and b_pred are saved in res and cleared to zero.
8. A system according to any one of claims 1 to 7, characterized in that: It includes data acquisition module, data processing module, model preparation module, data reasoning module and data integration module; The data acquisition module is used to acquire images containing tiny character data; the data processing module is used to preprocess the images to obtain tiny character data; the model preparation module is used to construct a deep learning model and train and verify the model; The data inference module is used to analyze and process the pre-processed tiny character data; The data integration module is used to output the analysis results.
9. A computer device, characterized in that: The invention comprises at least one processor and a computer-readable medium storing a computer program, wherein when the computer program is read and executed by the processor, the method according to any one of claims 1 to 7 is implemented.
10. A computer-readable medium, characterized in that The computer-readable medium stores a computer program, and when the computer program is read and executed by a processor, the method according to any one of claims 1 to 7 is implemented.
