An English letter recognition method, device and medium based on machine vision

Abstract

The application discloses an English letter recognition method and device based on machine vision and a medium, relates to the technical field of image recognition, and comprises the following steps: collecting an original image, performing grayscale and denoising processing to obtain a denoised grayscale image, correcting the denoised grayscale image and performing binaryzation processing, delimiting a connected domain, and obtaining a binary letter image; performing iterative pruning on the binary letter image, setting nodes and edges, summarizing a skeleton graph structure, calculating the path length of the edges and the curvature and bifurcation degree of the nodes, and labeling the skeleton graph structure to obtain a skeleton graph structure with topological filtering labels; setting a topological complex according to the skeleton graph structure with topological filtering labels, constructing the input of a graph neural network model, and completing the training of the graph neural network model; and inputting a node feature matrix into the graph neural network model to obtain a letter recognition result. Through the training of the graph neural network model, the recognition accuracy of complex, careless or deformed letters is improved.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and in particular to a method, device and medium for English letter recognition based on machine vision. Background Technology

[0002] In the fields of machine vision and pattern recognition, letter recognition is a fundamental and crucial task, widely applied in document digitization, automatic scoring systems, scene text recognition, and assisted robot interaction. Traditional machine vision-based letter recognition methods typically follow classic image processing and feature extraction paradigms. The typical process begins with preprocessing the acquired raw image, such as grayscale conversion and filtering to eliminate interference. Then, binarization segmentation is performed using global or local thresholding methods to extract the foreground regions of the letters. Further feature engineering is then essential, with common methods including extracting character contour features, geometric moment features, projected histogram features, or skeleton-based keypoint features. These hand-designed feature vectors are then fed into classifiers such as Support Vector Machines (SVM), Random Forests, or Multilayer Perceptrons for training and recognition. This technical approach is mature and stable, laying a solid foundation for the development of early Optical Character Recognition (OCR) systems.

[0003] While the aforementioned conventional methods have been widely adopted, they still face challenges when dealing with complex backgrounds and variations in font shape. First, the effectiveness of feature design heavily relies on prior knowledge and experience. The extracted contours, rectangles, and other features may exhibit decreased discriminative power and robustness when faced with severe deformation, partial occlusion, or stroke overlap, leading to increased intra-class differences for similar letters and decreased inter-class differences for dissimilar letters. Second, traditional methods typically treat images as a grid of pixels, failing to fully utilize the inherent topological structure and graph relationship information of letter shapes. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides a machine vision-based English letter recognition method to solve the problems of insufficient feature robustness and inadequate utilization of character topology in complex scenarios.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] In a first aspect, the present invention provides a machine vision-based method for English letter recognition, comprising: acquiring an original image; performing grayscale conversion and denoising processing to obtain a denoised grayscale image; correcting and binarizing the denoised grayscale image; defining connected components to obtain a binary letter image; iteratively pruning the binary letter image; setting nodes and edges; summarizing the results into a skeleton graph structure; calculating the path length of the edges and the curvature and fork degree of the nodes and labeling them on the skeleton graph structure to obtain a skeleton graph structure with topological filtering annotations; constructing the input of a graph neural network model based on the set topological complex of the skeleton graph structure with topological filtering annotations; completing the training of the graph neural network model; and inputting the node feature matrix into the graph neural network model to obtain the letter recognition result.

[0008] As a preferred embodiment of the machine vision-based English letter recognition method of the present invention, the correction of the denoised grayscale image includes the following steps: acquiring the original image, performing grayscale processing on the original image to obtain a grayscale image; extracting a neighborhood window centered on each pixel of the grayscale image, and calculating the median of the grayscale values ​​of all pixels in the neighborhood window as the new grayscale value to obtain a denoised grayscale image; performing binarization processing on the denoised grayscale image to obtain a binary image; selecting a rectangular structuring element according to the size of the neighborhood window, and performing an erosion operation on the binary image using the rectangular structuring element; performing a dilation operation on the eroded image to obtain a background estimation image, and correcting the denoised grayscale image to a background-corrected grayscale image using the background estimation image.

[0009] As a preferred embodiment of the machine vision-based English letter recognition method of the present invention, the steps for obtaining the binary letter image are as follows: Calculate the offset coefficient based on the corrected grayscale value of the background corrected grayscale image, and then obtain the binarization threshold for each pixel; compare the corrected grayscale value with the binarization threshold to obtain the corrected binarized image; scan each pixel of the corrected binarized image and delineate the connected components to obtain the binary letter image.

[0010] As a preferred embodiment of the machine vision-based English letter recognition method of the present invention, the steps of iteratively pruning the binary letter image and setting nodes and edges to form a skeleton graph structure are as follows: iteratively pruning the binary letter image until the result of the iterative pruning no longer changes to generate a skeleton graph; extracting the foreground pixels of the skeleton graph and setting nodes; starting from the nodes, connecting the foreground pixels along the skeleton graph to obtain edges, and summarizing them with the nodes to form a skeleton graph structure.

[0011] As a preferred embodiment of the machine vision-based English letter recognition method of the present invention, the steps for obtaining the skeleton graph structure with topological filtering annotation are as follows: setting the origin of the coordinate system and calculating the path length of the edges based on the node position and the number of nodes; extracting the position coordinates of the node and two adjacent nodes, calculating the triangle area of ​​the three nodes, and thus obtaining the curvature of the node; counting the number of nodes in the eight positions around each node as the fork degree; annotating the path length of each edge and the curvature of each node to obtain the skeleton graph structure with topological filtering annotation.

[0012] As a preferred embodiment of the machine vision-based English letter recognition method of the present invention, the setting of the topological complex refers to combining the nodes of the skeleton graph structure with topological filtering labels and the topological filtering labels into the topological complex of the nodes, counting the number of nodes on each edge, calculating the weight of the edge according to the curvature of each node, combining the node topological complex of each edge with the weight of the edge to form the topological complex of the edge, and forming a complete topological complex with the topological complex of all nodes.

[0013] As a preferred embodiment of the machine vision-based English letter recognition method of the present invention, the steps for constructing the input of the graph neural network model are as follows: train the graph neural network model, construct node feature vectors according to the node position coordinates, node curvature and node fork degree and summarize them into a node feature matrix; construct an adjacency matrix according to the number of nodes; count the number of edges connecting each node to other nodes and construct a degree matrix; perform symmetric normalization on the adjacency matrix according to the degree matrix to obtain a normalized adjacency matrix.

[0014] As a preferred embodiment of the machine vision-based English letter recognition method of the present invention, the steps for training the graph neural network model are as follows: randomly generate an initial weight matrix, perform the first layer graph convolution calculation to obtain the first layer output node features; calculate the second layer graph convolution calculation in the same way until the output node features are obtained; calculate the loss function; calculate the partial derivative of the loss function with respect to the weights, set the weight update rule through the optimization algorithm, and update the weights of each layer; complete the training of the graph neural network model.

[0015] In a second aspect, the present invention provides a computer device including a memory and a processor, wherein the memory stores a computer program, wherein when the computer program is executed by the processor, it implements any step of the machine vision-based English letter recognition method described in the first aspect of the present invention.

[0016] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein, when the computer program is executed by a processor, it implements any step of the machine vision-based English letter recognition method described in the first aspect of the present invention.

[0017] The beneficial effects of this invention are as follows: by obtaining a skeleton graph structure labeled with topological filtering, the essential structural information of letter morphology is accurately captured, effectively reducing the interference of redundant details; by completing the training of the graph neural network model, the recognition accuracy of complex, illegible or deformed letters is improved. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart of a machine vision-based English letter recognition method.

[0020] Figure 2 The flowchart for obtaining the binary alphabet diagram.

[0021] Figure 3 A flowchart for obtaining the skeleton graph structure with topological filtering annotations.

[0022] Figure 4 This is a flowchart for training a graph neural network model.

[0023] Figure 5 The skeleton graph structure with topological filtering annotations for the letter "A".

[0024] Figure 6 For training loss curves. Detailed Implementation

[0025] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0026] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0027] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0028] Reference Figures 1-6 This is one embodiment of the present invention, which provides a machine vision-based English letter recognition method, including the following steps:

[0029] S1. Acquire the original image, perform grayscale conversion and denoising to obtain a denoised grayscale image, correct the denoised grayscale image and perform binarization, delineate connected components, and obtain a binary alphabet image.

[0030] The original image is acquired and then converted to grayscale to obtain a grayscale image.

[0031] Furthermore, the original image is acquired, and the pixel values ​​of each pixel in the red, green, and blue channels are linearly weighted to obtain the grayscale value of each pixel. This converts the original image into a grayscale image, as expressed by:

[0032] ;

[0033] in, Grayscale value The pixel value for the red channel. The pixel value is for the green channel. This represents the pixel value of the blue channel.

[0034] It should be noted that the weights corresponding to the three color channels are specified by the International Electrotechnical Commission.

[0035] The neighborhood window is extracted centered on each pixel of the grayscale image, and the median of the grayscale values ​​of all pixels in the neighborhood window is calculated as the new grayscale value to obtain the denoised grayscale image.

[0036] Furthermore, denoising is performed on the grayscale image. Each pixel in the grayscale image is treated as the pixel to be processed. A neighborhood window is taken centered on the pixel to be processed, and the grayscale values ​​of all pixels in the neighborhood window are obtained and sorted in ascending order to obtain a neighborhood grayscale sequence. The median of the neighborhood grayscale sequence is calculated as the new grayscale value of the pixel to be processed. When the pixel to be processed is at the edge of the grayscale image, the neighborhood window will extend beyond the grayscale image boundary. Only the grayscale values ​​within the neighborhood window are extracted and sorted in ascending order to obtain a neighborhood grayscale sequence. The median of the neighborhood grayscale sequence is calculated as the new grayscale value of the pixel to be processed, resulting in a denoised grayscale image.

[0037] The grayscale value in the denoised grayscale image ranges from 0 to 255. The midpoint of the grayscale value range, 128, is taken as the binarization threshold. The denoised grayscale image is binarized by setting pixels with grayscale values ​​less than 128 to 0 (background pixels) and pixels with grayscale values ​​not less than 128 to 1 (foreground pixels), thus obtaining a binary image.

[0038] A rectangular structuring element is selected based on the size of the neighborhood window, and the binary image is eroded using the rectangular structuring element.

[0039] Furthermore, a rectangular structuring element is selected based on the size of the neighborhood window, and the center point of the rectangular structuring element is used as the structuring anchor point. The structuring anchor point is aligned with the first pixel of the upper left corner of the binary image, and an erosion operation is performed. For the part of the rectangular structuring element that extends beyond the binary image, zeros are padded around the binary image to obtain the eroded image.

[0040] It should be noted that a rectangular structuring element refers to a set of pixels in a rectangular shape used to scan an image in morphological operations, where all elements in the pixel set are equal to 1. When selecting a rectangular structuring element, the selection is based on the size of the neighborhood window, and the size of the rectangular structuring element is the same as the size of the neighborhood window.

[0041] It should be noted that the erosion operation refers to the process of continuously comparing a rectangular structuring element with the binary image during the erosion process. The rectangular structuring element defines a range within the binary image, and each pixel of the rectangular structuring element is compared with the pixels within that range. If a 0 pixel exists within the range, the pixel corresponding to the structuring anchor is set to 0; otherwise, it is set to 1. The rectangular structuring element is then slid to the right, comparing its pixels with the pixels within the range defined by the shifted element, until the foreground and background pixels of the current row are set. The rectangular structuring element is then slid down, setting either foreground or background pixels, and so on, until the foreground and background pixels of all pixels in the binary image are set, resulting in the eroded image.

[0042] A dilation operation is performed on the eroded image to obtain a background estimation map, and the denoised grayscale image is corrected to a background-corrected grayscale image using the background estimation map.

[0043] Furthermore, a dilation operation is performed on the eroded image. The structural anchor point of the rectangular structuring element is aligned with the first pixel of the upper left corner of the eroded image, and the dilation operation is performed. For the part of the rectangular structuring element that exceeds the eroded image, zeros are padded around the eroded image to obtain the background estimation image.

[0044] It should be noted that the dilation operation involves continuously comparing the rectangular structuring element with the eroded image during the dilation process. The eroded image area is defined by the rectangular structuring element. Each pixel of the rectangular structuring element is compared with the pixels within the defined eroded image area. If a pixel exists within the eroded image area, the pixel corresponding to the structuring anchor is set to 1; otherwise, it is set to 0. This process is repeated pixel by pixel in the eroded image using the same sliding method as the erosion operation, resulting in the background estimation image.

[0045] Create a blank image of the same size as the background estimation image and the denoised grayscale image. Fill the blank image with the background pixels from the background estimation image according to their positions, and fill the remaining part of the blank image with the corresponding grayscale values ​​from the denoised grayscale image to obtain the background-corrected grayscale image.

[0046] Based on the corrected grayscale value of the background corrected grayscale image, the offset coefficient is calculated, and then the binarization threshold of each pixel is obtained. The corrected grayscale value is compared with the binarization threshold to obtain the corrected binarized image.

[0047] Furthermore, all corrected grayscale values ​​of the background corrected grayscale image are extracted, and the standard deviation of the corrected image is calculated. The offset coefficient is then calculated based on the standard deviation of the corrected image, expressed as:

[0048] ;

[0049] in, This is the offset coefficient. To correct the standard deviation of the image.

[0050] It should be noted that in the formula for calculating the offset coefficient, 0.1 is an adjustment coefficient. Using 0.1 as an adjustment coefficient is widely used in image processing tasks.

[0051] Extract the corrected grayscale value of each pixel in the background corrected grayscale image, calculate the average grayscale value of the neighborhood region of each pixel, and calculate the binarization threshold for each pixel. The expression is as follows:

[0052] ;

[0053] in, For binarization threshold, This represents the average gray level of the neighborhood.

[0054] The corrected gray value is compared with the binarization threshold to perform a binarization comparison and obtain a corrected binarized image. If the corrected gray value is greater than the binarization threshold, the gray value of the current pixel is set to 255 and it is used as a foreground pixel. If the corrected gray value is not greater than the binarization threshold, the gray value of the current pixel is set to 0 and it is used as a background pixel.

[0055] Each pixel of the scanned and corrected binarized image is used to define connected components for all pixels with a gray value of 255 using the 8-neighborhood connection method, resulting in a binary alphabetic image.

[0056] It should be noted that the 8-neighborhood connection method refers to taking the foreground pixel as the current pixel, determining whether the pixels in the eight surrounding directions (horizontal, vertical, and diagonal directions) of the current pixel are foreground pixels, and classifying the foreground pixels in the eight directions and the current pixel as a unified connected component.

[0057] S2. Iteratively prune the binary alphabet graph, set the nodes and edges, and summarize them into a skeleton graph structure. Calculate the path length of the edges and the curvature and fork degree of the nodes and label them in the skeleton graph structure to obtain a skeleton graph structure with topological filtering labels.

[0058] The binary alphabetic graph is iteratively pruned until the result of the iterative pruning no longer changes, thus generating a skeleton graph.

[0059] Furthermore, letter skeletons are extracted from the binary letter image, and pixels in the binary letter image are pruned. Each pixel in the binary letter image is designated as a pixel to be pruned, and the pixel to be pruned and its eight surrounding pixels are defined as the pruning neighborhood. Pruning is performed in step A, with the following pixel deletion conditions: 1. The pixel to be pruned is a foreground pixel; 2. At least one pixel in the pruning neighborhood is a background pixel; 3. Two or three pixels in the pruning neighborhood are foreground pixels; 4. None of the four adjacent pixels (top, bottom, left, and right) of the pixel to be pruned are foreground pixels. When the pixel to be pruned meets all four conditions, it is deleted and adjusted to a background pixel. Set the pixel deletion conditions in step B to be the same as in step A, and continue trimming through step B to complete one trimming iteration, obtaining a trimming iteration map. Compare the trimming iteration map with the binary alphabet map. If the two maps are the same, stop the iteration and use the trimming iteration map as the skeleton map; if the two maps are different, start the next round of iterations until the trimming iteration map generated in this round is the same as the trimming iteration map before this round of iterations. Then, use the trimming iteration map generated in this round of iterations as the skeleton map.

[0060] It should be noted that for pixels to be pruned that are located at the edge of the binary alphabet image, the pixel deletion conditions in step A and step B are adjusted as follows: 1. The pixel to be pruned is a foreground pixel; 2. There are at least two foreground pixels in the neighborhood of the pixel to be pruned; 3. There are at least two background pixels in the neighborhood of the pixel to be pruned; 4. Deleting the pixel to be pruned will not break the connectivity of the image.

[0061] Extract each foreground pixel from the skeleton graph sequentially, and then extract the 8 pixels surrounding each node. If there is only one foreground pixel among the 8 pixels surrounding the current foreground pixel, then the current foreground pixel is an endpoint; if there are three or more foreground pixels among the 8 pixels surrounding the current foreground pixel, then the current foreground pixel is a branch point. Starting from each endpoint or branch point, connect adjacent foreground pixels along the path in the skeleton graph to obtain edges, and add the same connection label to all nodes on the edges. Combine the nodes and edges to form the skeleton graph structure.

[0062] Set the origin of the coordinate system, and calculate the path length of the edges based on the node positions and the number of nodes.

[0063] Furthermore, a coordinate system is established with the bottom left corner of the binary alphabetic graph as the origin, the bottom edge as the horizontal axis, and the left side as the vertical axis. The position coordinates of each node on the edge are read, the number of nodes on the edge is counted, and the path length of the edge is calculated. The expression is:

[0064] ;

[0065] in, For path length, For the number of nodes, For node indexing, For the first The x-coordinate of each node For the first The x-coordinate of each node For the first The ordinate of each node, For the first The ordinate of each node.

[0066] Extract the position coordinates of the node and two adjacent nodes, calculate the area of ​​the triangle formed by the three nodes, and then obtain the curvature of the node.

[0067] Furthermore, extract the position coordinates of the current node and its two adjacent nodes, calculate the path length of the edge formed by the current node and its two adjacent nodes, and calculate the area of ​​the triangle formed by the three nodes. The expression is as follows:

[0068] ;

[0069] in, Let the area be the triangle. The x-coordinate of the current node. Let x be the x-coordinate of the first adjacent node. Let x be the x-coordinate of the second adjacent node. Let be the y-coordinate of the current node. Let be the ordinate of the first adjacent node. The ordinate of the second adjacent node.

[0070] Calculate the curvature of the current node based on the path length of the edge formed by the current node and its two adjacent nodes, and the area of ​​the triangle. The expression is:

[0071] ;

[0072] in, Let be the curvature of the current node. This is the path length of the edge formed by the current node and its first adjacent node. This is the path length of the edge formed by the current node and its second adjacent node.

[0073] Count the number of nodes in the 8 positions surrounding each node, and use this as the fork degree.

[0074] The path length of each edge is labeled as the topological filter label for the edge, and the curvature and fork degree of each node are labeled as the topological filter label for the node, resulting in a skeleton graph structure with topological filter labels.

[0075] Taking the letter "A" as an example, generate Figure 5 , Figure 5 This diagram displays the skeleton structure of the letter "A" with topological filtering annotations. A two-dimensional coordinate system is used to represent the node positions of the letter skeleton, with dots representing nodes. Green dashed lines connect the nodes to form edges, and the path length of each edge is labeled. Curvature values ​​(e.g., 2.82) and forks are labeled next to the nodes, reflecting the geometric features and topological structure of the letter skeleton. Figure 5 The text also shows the area of ​​triangles formed by the edges (e.g., area = 12.7). This geometric data is used to calculate the curvature of nodes, providing structured information about letter shapes to assist in subsequent letter recognition and analysis.

[0076] S3. Based on the defined topological complex of the skeleton graph structure with topological filtering annotation, construct the input of the graph neural network model and complete the training of the graph neural network model.

[0077] The nodes in the skeleton graph structure and their topological filtering labels are combined into a topological complex of the node, and then summarized into a vertex set. The topological complex of a node includes the node's position coordinates, curvature, and branching degree. The number of nodes in each edge is counted, and the edge weight is calculated based on the curvature of each node in the edge, expressed as:

[0078] ;

[0079] in, The weight of the edge. Let be the number of nodes on the edge. The node index of the edge. Nodes that are edges The curvature.

[0080] Combine the topological complexes of all nodes in the edge with the weights of the edge to form the topological complex of the edge, and summarize them into an edge set. Summarize the vertex set and the edge set into a topological complex.

[0081] The graph neural network model is trained, and node feature vectors are constructed according to the node's position coordinates, node curvature, and node fork degree, and then summarized into a node feature matrix.

[0082] Furthermore, the graph neural network model is trained based on the historical topological complex. Node feature vectors are constructed based on the historical vertex set according to the node's position coordinates, curvature, and fork degree. All node feature vectors are then aggregated to obtain the node feature matrix.

[0083] Based on the number of nodes, construct an adjacency matrix. The number of rows and columns of the adjacency matrix are the same as the number of nodes. If there is an edge between two nodes, fill in 1 in the corresponding position; if there is no edge between two nodes, fill in 0 in the corresponding position.

[0084] Count the number of edges connecting each node to other nodes and construct a degree matrix.

[0085] It should be noted that the degree matrix is ​​a diagonal matrix, and the diagonal elements represent the degree of a node, that is, the number of edges connecting a node to other nodes.

[0086] The adjacency matrix is ​​symmetrically normalized using the degree matrix to obtain the normalized adjacency matrix, expressed as:

[0087] ;

[0088] in, To normalize the adjacency matrix, For degree matrix, It is an adjacency matrix.

[0089] An initial weight matrix is ​​generated randomly using a uniform distribution. The number of columns in the weight matrix represents the feature dimension of each node, and the number of rows represents the output feature dimension. The ReLU activation function is used to perform the first layer of graph convolution, yielding the first layer of output node features. The expression is as follows:

[0090] ;

[0091] in, The first layer output node features, For activation function, The node feature matrix, This is the initial weight matrix.

[0092] It should be noted that the output feature dimension is a hyperparameter with a value range of 64 to 1024. Based on experiments, the impact of different dimensions on letter recognition accuracy and training efficiency was evaluated. The experimental results show that when the output feature dimension is 256, the model can maintain high recognition accuracy while avoiding excessive consumption of computing resources, and it also shows good convergence during training. Therefore, the output feature dimension is set to 256.

[0093] The second layer graph convolution is calculated using the same method, up to the Lth layer, to obtain the Lth layer output node features.

[0094] Extract the position coordinates of each node in the image, the node's fork degree, and calculate the node's curvature as the true features of the image nodes. Then, obtain the true labels of the image nodes and represent their categories using one-hot encoding. Finally, transform the output node features into predicted values ​​using a multi-classification function, expressed as:

[0095] ;

[0096] in, For the first The predicted value of each node, For multi-class classification functions, This is a bias term.

[0097] The loss function is calculated using the cross-entropy loss function, and its expression is:

[0098] ;

[0099] in, For loss function, For the first The true characteristics of each node.

[0100] By calculating the partial derivative of the loss function with respect to the weights, we obtain the gradient of the weights and the gradient of the node features. The expression is as follows:

[0101] ;

[0102] in, For the first The gradient of the layer weights, For the first Gradient of layer node features, This represents the gradient of the node features with respect to the weight matrix.

[0103] It should be noted that the expression for the gradient of the node features with respect to the weight matrix is ​​as follows:

[0104] ;

[0105] in, For the first Layer output node features.

[0106] The weights of each layer are updated by optimizing the algorithm, with the following update rule:

[0107] ;

[0108] ;

[0109] ;

[0110] ;

[0111] in, For the first The first moment of the gradient in the next iteration For the first The second moment of the gradient in the next iteration. For the first Momentum after bias correction in the next iteration. For the first The squared gradient after bias correction in the next iteration. For learning rate, The first momentum decay coefficient, This is the second momentum decay coefficient. The minimum value is added to prevent division by zero errors. For the number of iterations, For the first The weight matrix of the next iteration. For the first The weight matrix for the next iteration.

[0112] It should be noted that the learning rate is typically set to 0.001 or 0.01.

[0113] It should be noted that the first momentum decay coefficient is used to control the update rate of the first moment estimate, and is usually set to 0.9; the second momentum decay coefficient is used to control the update rate of the second moment estimate, and is usually set to 0.999.

[0114] It should be noted that the partial derivative of the loss function with respect to the bias term is calculated to obtain the gradient of the bias term, and the bias term of each layer is updated through the optimization algorithm. The expression is as follows:

[0115] ;

[0116] in, For the first The bias term of the next iteration For the first The bias term of the next iteration This is the gradient of the bias term.

[0117] The maximum number of iterations is set to 10,000, and the minimum loss threshold is set to 0.01. When the maximum number of iterations is reached or the loss value is less than the minimum loss threshold, the iteration stops and the graph neural network model training is completed.

[0118] It should be noted that the training process of a graph neural network model involves a large number of samples and requires multiple iterations to achieve the desired effect. Setting 10,000 iterations as the maximum number of iterations ensures that the graph neural network model can fully adjust its parameters during training. Setting 0.01 as the minimum threshold for loss ensures that the graph neural network model achieves high performance while avoiding overtraining and resource waste. Further reducing the loss value will not significantly improve the training of the graph neural network model.

[0119] To verify the impact of different learning rates on model training performance, we conducted... Figure 6 The training loss curves are shown for different learning rates (0.001 and 0.01). As the number of training epochs increases, the loss values ​​for all learning rates gradually decrease, indicating that the model is continuously optimizing and gradually converging. Higher learning rates (0.01) converge faster in the early stages but exhibit greater fluctuations, while lower learning rates (0.001) show a more stable convergence trend, with the loss value tending to stabilize.

[0120] S4. Input the node feature matrix into the graph neural network model to obtain the letter recognition results.

[0121] The node feature matrix is ​​input into the graph neural network model to obtain the class probability distribution of each node. The class with the highest probability is selected as the predicted class of the node. The predicted classes of nodes with the same connection label are summarized, and the proportion of each predicted class is calculated. The class with the highest proportion is the letter class, and the letter recognition result is obtained.

[0122] It should be noted that since it is identifying English letters, there are 26 categories.

[0123] This embodiment also provides a computer device applicable to the English letter recognition method based on machine vision, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the English letter recognition method based on machine vision as proposed in the above embodiment.

[0124] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0125] This embodiment also provides a storage medium storing a computer program that, when executed by a processor, implements the machine vision-based English letter recognition method as proposed in the above embodiments. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0126] In summary, this invention achieves the effect of accurately capturing the essential structural information of letter shapes and effectively reducing the interference of redundant details by obtaining a skeleton graph structure with topological filtering annotation; and improves the recognition accuracy of complex, illegible or deformed letters by completing the training of the graph neural network model.

[0127] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A machine vision-based method for English letter recognition, characterized in that, include: The original image is acquired, grayscale and denoising are performed to obtain a denoised grayscale image, the denoised grayscale image is corrected and binarized, connected components are delineated, and a binary alphabet image is obtained. The binary alphabet graph is iteratively pruned, and nodes and edges are set to form a skeleton graph structure. The path length of the edges and the curvature and fork degree of the nodes are calculated and labeled on the skeleton graph structure to obtain a skeleton graph structure with topological filtering labels. Based on the defined topological complex of the skeleton graph structure with topological filtering annotation, the input of the graph neural network model is constructed, and the training of the graph neural network model is completed. The node feature matrix is ​​input into the graph neural network model to obtain the class probability distribution of each node. The class with the highest probability is selected as the predicted class of the node. The predicted classes of nodes with the same connection label are summarized and the proportion of each predicted class is calculated. The class with the highest proportion is the letter class, and the letter recognition result is obtained. The steps to obtain the skeleton graph structure with topological filtering annotations are as follows: Set the origin of the coordinate system, and calculate the path length of the edges based on the node positions and the number of nodes; Extract the position coordinates of the node and two adjacent nodes, calculate the area of ​​the triangle of the three nodes, and then obtain the curvature of the node; Count the number of nodes in the 8 positions surrounding each node, and use this as the fork degree; Label the path length of each edge and the curvature of each node to obtain the skeleton graph structure with topological filtering annotation; The path length of each edge is labeled as the topological filter label of the edge, and the curvature and fork degree of each node are labeled as the topological filter label of the node, resulting in a skeleton graph structure with topological filter labels. The aforementioned setting of the topological complex refers to combining the nodes of the skeleton graph structure with topological filter annotations and the topological filter annotations into a topological complex of the nodes, counting the number of nodes for each edge, calculating the weight of the edge based on the curvature of each node, combining the node topological complex of each edge with the edge weight to form the topological complex of the edge, and combining it with the topological complex of all nodes to form a complete topological complex. The steps for constructing the input to the graph neural network model are as follows: The graph neural network model is trained, and node feature vectors are constructed according to the node's position coordinates, node curvature, and node fork degree, and then summarized into a node feature matrix. Construct an adjacency matrix based on the number of nodes; Count the number of edges connecting each node to other nodes and construct a degree matrix; The adjacency matrix is ​​symmetrically normalized based on the degree matrix to obtain the normalized adjacency matrix.

2. The machine vision-based English letter recognition method as described in claim 1, characterized in that, The steps for correcting the denoised grayscale image are as follows: Acquire the original image, and perform grayscale processing on the original image to obtain a grayscale image; Extract a neighborhood window centered on each pixel of the grayscale image, and calculate the median of the grayscale values ​​of all pixels in the neighborhood window as the new grayscale value to obtain a denoised grayscale image. The denoised grayscale image is binarized to obtain a binary image; A rectangular structuring element is selected based on the size of the neighborhood window, and the binary image is eroded using the rectangular structuring element. A dilation operation is performed on the eroded image to obtain a background estimation map, and the denoised grayscale image is corrected to a background-corrected grayscale image using the background estimation map.

3. The machine vision-based English letter recognition method as described in claim 2, characterized in that, The steps to obtain the binary alphabet image are as follows: Based on the corrected grayscale value of the background corrected grayscale image, the offset coefficient is calculated, and then the binarization threshold of each pixel is obtained. The corrected grayscale value is compared with the binarization threshold to obtain the corrected binarized image. Scan each pixel of the corrected binarized image and delineate connected components to obtain a binary alphabetic image.

4. The machine vision-based English letter recognition method as described in claim 1, characterized in that, The steps for iteratively pruning the binary alphabetic graph and setting nodes and edges to summarize it into a skeleton graph structure are as follows: The binary alphabetic graph is iteratively pruned until the result of the iterative pruning no longer changes, thus generating a skeleton graph. Extract the foreground pixels of the skeleton graph and set nodes. Starting from the nodes, connect the foreground pixels along the skeleton graph to obtain edges, and combine them with the nodes to form the skeleton graph structure.

5. The machine vision-based English letter recognition method as described in claim 1, characterized in that, The steps to complete the training of the graph neural network model are as follows: An initial weight matrix is ​​randomly generated, and the first layer of graph convolution is performed to obtain the first layer output node features. The second layer of graph convolution is calculated using the same method until the output node features are obtained; Calculate the loss function; The partial derivative of the loss function with respect to the weights is calculated, and the weight update rules are set through the optimization algorithm to update the weights of each layer. Complete the training of the graph neural network model.

6. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the machine vision-based English letter recognition method according to any one of claims 1 to 5.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the machine vision-based English letter recognition method according to any one of claims 1 to 5.

Images