Industrial part number extraction and recognition method based on adaptive graph convolution and transformer

Abstract

The application discloses an industrial part number extraction and recognition method based on adaptive graph convolution and a Transformer, and comprises the following steps: S1, acquiring RGB part image data containing an industrial part number, sequentially performing grayscale processing, normalization processing and image enhancement processing on the RGB part image data, and acquiring preprocessed image data; S2, constructing an industrial part number extraction and recognition model; S3, performing model training on the constructed industrial part number extraction and recognition model according to the preprocessed image data, and acquiring an optimal industrial part number extraction and recognition model, so that the industrial part number extraction and recognition is realized according to the optimal industrial part number extraction and recognition model. The application solves the problem that the existing number extraction and recognition technology cannot effectively deal with the number area with complex structure and diversity, resulting in insufficient industrial part number extraction and recognition precision.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and artificial intelligence, and in particular to a method for extracting and recognizing industrial part numbers based on adaptive graph convolution and Transformer. Background Technology

[0002] With the rapid development of industrial automation and intelligent manufacturing, the automatic extraction and identification technology of part numbers has become a key link in industrial production management. Part numbers are used to identify different types of parts and are an important basis for multiple stages such as production, material tracking, warehouse management, and quality control. In order to improve production efficiency and automation level, accurate identification of part numbers has become an important technical requirement in modern manufacturing. However, current part number identification methods still face significant challenges when dealing with complex backgrounds, changes in lighting, uneven textures on part surfaces, and raised numbering features.

[0003] Traditional part number recognition methods primarily rely on image processing techniques, such as edge detection and morphological processing. These methods often depend on manually designed features to extract the numbering region. However, these traditional methods are frequently affected by environmental changes in practical applications. For example, uneven lighting, complex backgrounds, and potential stains or obstructions on the part surface all make number region extraction extremely difficult. Furthermore, part numbers vary widely in font, size, and layout, and are sometimes deformed or occluded, making it difficult for traditional methods to achieve efficient and accurate recognition.

[0004] In recent years, deep learning, especially convolutional neural networks (CNNs), has made significant progress in the field of image recognition. By automatically learning features from data, CNNs have achieved excellent results in many visual tasks, but they still have certain limitations when dealing with complex backgrounds and non-planar numbering (such as raised numbering). This is because traditional CNN networks often rely on fixed convolutional kernels, which cannot effectively handle numbering regions with complex structures and diversity. Summary of the Invention

[0005] This invention provides a method for extracting and recognizing industrial part numbers based on adaptive graph convolution and Transformer, in order to overcome the above-mentioned technical problems.

[0006] To achieve the above objectives, the technical solution of the present invention is as follows:

[0007] A method for extracting and recognizing industrial part numbers based on adaptive graph convolution and Transformer, specifically including the following steps:

[0008] S1: Obtain RGB part image data containing industrial part numbers;

[0009] The RGB part image data is then subjected to grayscale conversion, normalization, and image enhancement processing in sequence to obtain preprocessed image data.

[0010] S2: Construct an industrial parts number extraction and recognition model;

[0011] Furthermore, the industrial part number extraction and recognition model includes a feature extraction unit based on adaptive graph convolution, a global information modeling unit based on the Transformer module, and a positioning and recognition module for part number positions and characters;

[0012] A feature extraction unit based on adaptive graph convolution is used to extract feature vectors of local part number regions in preprocessed image data.

[0013] The global information modeling unit based on the Transformer module is used to perform global feature modeling based on the features of the local part number region, obtain the global part number region feature vector, and obtain the enhanced region feature vector by concatenating the global part number region feature vector with the local part number region feature vector.

[0014] The part number location and character positioning and recognition module is used to locate the industrial part number area and recognize the industrial part number characters based on the enhanced region feature vector;

[0015] S3: Train the constructed industrial part number extraction and recognition model based on the preprocessed image data to obtain the optimal industrial part number extraction and recognition model, and realize the extraction and recognition of industrial part numbers based on the optimal industrial part number extraction and recognition model.

[0016] Furthermore, S1 specifically includes the following steps:

[0017] S11: Obtain RGB part image data containing industrial part numbers, and perform grayscale processing on it to obtain a grayscale image;

[0018] S12: Normalize the grayscale image to obtain a normalized image;

[0019] S13: Perform image enhancement processing on the normalized image to obtain preprocessed image data; and the image enhancement processing includes: performing noise reduction processing on the normalized image through a median filter;

[0020] S14: Based on the Canny edge detection algorithm, edge features are extracted from the normalized image after denoising to obtain an edge feature map; and the edge feature map is converted into a binary image using the Otsu algorithm.

[0021] S15: Call the pre-built OpenCV function cv2.equalizeHist to perform histogram equalization on the binary image and obtain the histogram equalized image; and perform morphological operations on the histogram equalized image to obtain the morphological image.

[0022] S16: Call the cv2.findContours function in the default OpenCV to perform contour detection and extract the number region of the industrial part number in the morphological image;

[0023] S17: Call the cv2.drawContours function in the default OpenCV to draw the extracted numbered region onto the RGB part image to obtain the numbered region bounding box and mark the industrial part number characters, and save the image as a PNG format to obtain preprocessed image data.

[0024] Furthermore, the feature extraction unit based on adaptive graph convolution in S2 includes a graph structure acquisition module, an adaptive graph convolution module, a local feature enhancement module, a multi-scale feature extraction module, and a local feature fusion module;

[0025] The graph structure acquisition module is used to define the numbered region pixels as nodes, and use the pixel value similarity between the pixels in each numbered region as edges, so as to transform the numbered regions in the preprocessed image data into a node graph structure, and obtain the node feature vector based on the node graph structure.

[0026] And the expression for the node feature vector is:

[0027] X i =I(x) i ,y i )

[0028] In the formula: I(x) i ,y i ) represents the position (x) in the preprocessed image data. i ,y i Pixel value at (); X i Represents the feature vector of node i;

[0029] The adaptive graph convolution module is used to adjust the convolution kernel parameters of the adaptive graph convolutional network based on the node feature vectors; the adjustment formula for the convolution kernel parameters is as follows:

[0030] W i =f(X) i ,X j )

[0031] In the formula: W i Denotes the adaptive convolution kernel for node i; f(X) i ,X j) represents the feature similarity function between node i and its neighboring node j;

[0032] The node feature vectors are then subjected to graph convolution operations using an adjusted adaptive graph convolution network to obtain an optimized node graph structure. The expression for the graph convolution operation is as follows:

[0033]

[0034] In the formula: W represents the output feature vector of node i in the (l+1)th layer; N(i) represents the neighboring nodes of node i; ij This represents the parameter weights between node i and its neighboring node j; b represents the output feature vector of node i in the l-th layer; i This represents the bias term for node i;

[0035] The local feature enhancement module is used to adjust and optimize the attention weights between nodes in the node graph structure based on the self-attention mechanism to obtain feature maps of local numbered regions.

[0036] And the expression for the attention weights between nodes is:

[0037]

[0038] In the formula: α ij X represents the attention weight between node i and node j; i ,X j This represents the feature vectors of node i and node j in the optimized node graph structure;

[0039] The multi-scale feature extraction module includes a first-scale convolutional layer, a second-scale convolutional layer, and a third-scale convolutional layer set in parallel.

[0040] The first-scale convolutional layer is used to perform convolution operations on the feature maps of the local numbered regions to obtain the first feature map; the second-scale convolutional layer is used to perform convolution operations on the feature maps of the local numbered regions to obtain the second feature map; the third-scale convolutional layer is used to perform convolution operations on the feature maps of the local numbered regions to obtain the third feature map.

[0041] The local feature fusion module is used to stitch together the first feature map, the second feature map, and the third feature map to obtain the features of the local part number area.

[0042] Furthermore, the global information modeling unit based on the Transformer module in S2 includes a Transformer encoder module and a global feature fusion module;

[0043] The Transformer encoder module includes a multi-layered stacked Transformer structure;

[0044] The Transformer structure is used to obtain the global attention feature vector of the part number region based on the features of the local part number region using a self-attention mechanism.

[0045] And the formula for obtaining the global attention feature vector is:

[0046]

[0047]

[0048] Q p =W Q h p K q =W K h q V q =W V h q

[0049] In the formula: h′ p The global attention feature vector representing the part number region; α pq The attention weight between nodes p and q is represented by N(p); N(p) represents the set of neighboring nodes of node p; W Q W K W V Q represents the learnable weight matrix; p K represents the query vector for node p; q V represents the key vector of node q; q h represents the value vector of node q; p The feature representation of node p; h q The feature representation of node q is represented by T; T represents the transpose.

[0050] The global feature fusion module is used to fuse the global attention feature vectors output by each Transformer structure to obtain the global part number region feature vector.

[0051] The enhanced region feature vector is obtained by splicing the feature vectors of the local part number region with the feature vectors of the local part number region.

[0052] Furthermore, the part number position and character location recognition module in S2 includes a region positioning module and a character recognition module based on a convolutional neural network (CNN).

[0053] S21: The region positioning module locates the industrial part number region based on the enhanced region feature vector, specifically including:

[0054] S211: Obtain a set of candidate bounding boxes for the localized region based on the enhanced region feature vector through the region proposal network;

[0055] S212: Based on bounding box regression and non-maximum suppression (NMS), the optimal localization region candidate box is selected and obtained from the set of localization region candidate boxes;

[0056] S22: Using a character recognition module based on a convolutional neural network (CNN), identify and obtain the industrial part number characters in the candidate bounding box of the optimal positioning region, specifically including:

[0057] S221: Extract low-dimensional local character features from the candidate boxes of the optimal localization region using a convolutional neural network (CNN), and convert the low-dimensional local character features into high-dimensional local character features.

[0058] S222: Input the high-dimensional local character features into a pre-set recurrent neural network (RNN) to obtain the probability distribution of the character sequence, and then output the predicted industrial part number characters.

[0059] Furthermore, in S3, the method for training the constructed industrial part number extraction and recognition model based on preprocessed image data to obtain the optimal industrial part number extraction and recognition model includes the following steps:

[0060] S31: Divide the preprocessed image data into training and testing sets according to a preset ratio;

[0061] S32: Train the constructed industrial part number extraction and recognition model based on the training set to obtain the trained industrial part number extraction and recognition model:

[0062] S33: Use the CTC loss function as the model loss function, and evaluate the trained industrial part number extraction and recognition model based on the test set to determine whether the output of the trained industrial part number extraction and recognition model has converged.

[0063] If so, then the industrial part number extraction and recognition model trained at this time is the optimal industrial part number extraction and recognition model;

[0064] Otherwise, the hyperparameters of the trained industrial part number extraction and recognition model are optimized based on the backpropagation algorithm, and step S31 is repeated.

[0065] This invention provides a method for extracting and recognizing industrial part numbers based on adaptive graph convolution and Transformer, with the following advantages:

[0066] (1) Introduction of Adaptive Graph Convolutional Network (AGCN): In order to cope with the raised features of the part number area and the interference of complex background, the introduced adaptive graph convolutional network can dynamically adjust the weight of the convolution kernel based on the local features of the image, efficiently extract the geometric shape and texture features of the part number, and improve the adaptability to complex background and uneven lighting conditions.

[0067] (2) Global information modeling of the Transformer module: Combining the self-attention mechanism of the Transformer model, this invention can effectively process global information in the image, capture the complex relationship between the numbered region and the background, and the Transformer module helps to improve the model's ability to distinguish between the numbered region and the background, strengthens the semantic expression of the numbered features, and can effectively improve the robustness and accuracy of the model.

[0068] (3) Combination of adaptive graph convolution and Transformer: This invention combines adaptive graph convolution network with Transformer module to give full play to the advantages of both in local feature extraction and global information modeling. This combination enables the model to accurately extract and identify part number regions in complex industrial environments, especially in situations with strong background interference and large changes in lighting conditions.

[0069] (4) Multi-level feature extraction and fusion mechanism: This invention designs a multi-level feature extraction and fusion mechanism to process the numbering region features at different scales. Local features are extracted through graph convolution, and combined with the global modeling capability of Transformer, the model can effectively cope with the diverse forms of numbering and improve the recognition accuracy in a variable environment. Attached Figure Description

[0070] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0071] Figure 1 This is a flowchart of the industrial part number extraction and recognition method based on adaptive graph convolution and Transformer of the present invention;

[0072] Figure 2 In this embodiment, the first RGB part image data containing the industrial part number is obtained;

[0073] Figure 3 In this embodiment, second RGB part image data containing industrial part numbers is obtained;

[0074] Figure 4 In this embodiment, the third RGB part image data containing the industrial part number is obtained;

[0075] Figure 5 This is the recognition result diagram of the first RGB part image data in this embodiment;

[0076] Figure 6 This is the recognition result diagram of the second RGB part image data in this embodiment;

[0077] Figure 7 This is the recognition result diagram of the third RGB part image data in this embodiment. Detailed Implementation

[0078] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0079] This embodiment provides a method for extracting and recognizing industrial part numbers based on adaptive graph convolution and Transformer, such as... Figure 1 As shown, the specific steps include:

[0080] S1: Obtain RGB part image data containing industrial part numbers;

[0081] Specifically, images are acquired using an industrial camera with a resolution of 1024x1024 pixels. In this embodiment, a Baslerac A2440-35µm camera is used to ensure image clarity during acquisition and to avoid overexposure or uneven lighting. Each image is guaranteed to include the part number area and is captured under uniform lighting conditions. The acquired raw images are as follows: Figures 2 to 4 As shown;

[0082] The RGB part image data is sequentially processed through grayscale conversion, normalization, and image enhancement to obtain preprocessed image data. This process includes the following steps:

[0083] S11: Obtain RGB part image data containing industrial part numbers, and perform grayscale processing on it to obtain a grayscale image;

[0084] Specifically, the acquired color image is converted into a grayscale image using the following formula:

[0085] Igray (x,y)=0.2989·R(x,y)+0.5870·G(x,y)+0.1140·B(x,y)

[0086] In the formula: I gray (x,y) represents the pixel value of the image after grayscale conversion, and R(x,y), G(x,y), B(x,y) represent the pixel values ​​of the red, green, and blue channels in the RGB image. This conversion formula is visually close to the sensitivity of the human eye to color.

[0087] S12: Normalize the grayscale image to obtain a normalized image;

[0088] Specifically, the pixel values ​​are adjusted to the range of [0,1], using the normalization formula:

[0089]

[0090] In the formula: I norm (x,y) represents the normalized pixel value, I gray (x,y) represents the pixel values ​​of the original grayscale image. After normalization, all pixel values ​​are located in the range [0,1], which facilitates subsequent processing.

[0091] S13: Perform image enhancement processing on the normalized image to obtain preprocessed image data; and the image enhancement processing includes: performing noise reduction processing on the normalized image through a median filter;

[0092] Specifically, for the normalized image, a 3x3 median filter is used to denoise the image, removing noise caused by small particles such as sand grains, while preserving edge and texture information;

[0093] S14: Based on the Canny edge detection algorithm, edge features are extracted from the normalized image after denoising to obtain an edge feature map; and the edge feature map is converted into a binary image using the Otsu algorithm.

[0094] Specifically, the Canny edge detection algorithm is used to extract edges from the image, with high and low thresholds set to 100 and 200 respectively. The formula for Canny edge detection is as follows:

[0095]

[0096] In the formula: ▽I represents the gradient values ​​of the image in the x and y directions; and These represent the rates of change of the image in the horizontal and vertical directions, respectively; here, I refers to the normalized image I. norm(x,y); The gradient value calculated by this formula allows the Canny algorithm to effectively extract edge information in the image; then the Otsu algorithm is used to automatically determine the optimal threshold and convert the image into a binary image in order to further separate the numbered regions and the background. The Otsu algorithm selects the optimal threshold by maximizing the inter-class variance.

[0097] S15: Call the pre-built OpenCV function cv2.equalizeHist to perform histogram equalization on the binary image and obtain the histogram equalized image; and perform morphological operations on the histogram equalized image to obtain the morphological image.

[0098] Specifically, histogram equalization is performed using the cv2.equalizeHist function in OpenCV to enhance image contrast, especially when image contrast is low, the equalized image helps to improve the recognizability of numbered regions; then morphological operations are performed, using dilation to enhance the convex features of the numbered regions, and setting the kernel size to a rectangular structuring element of 5x5.

[0099] S16: Call the cv2.findContours function in the default OpenCV to perform contour detection and extract the number region of the industrial part number in the morphological image;

[0100] Specifically, the `cv2.findContours` function in OpenCV is used for contour detection to extract numbered regions from the image. This involves calling the `cv2.findContours` function and setting the return value to `RETR_EXTERNAL` mode to detect only external contours. The function call rules are as follows:

[0101] contours,_=cv2.findContours(binary_image,cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE); In this embodiment, the area of ​​each detected contour is calculated, and the contour with the larger area is selected as the numbered region. The possible numbered regions are filtered out by the area threshold of the contour (above 2000 pixels).

[0102] S17: Call the cv2.drawContours function in the default OpenCV to draw the extracted numbered region onto the RGB part image to obtain the numbered region bounding box and mark the industrial part number characters, and save the image as a PNG format to obtain preprocessed image data.

[0103] Specifically, the extracted numbered regions are drawn onto the image using the cv2.drawContours function, marking the specific locations of the numbers. The final image contains the marked numbered region boxes and the corresponding number characters, which helps subsequent steps to further identify and process the numbered regions.

[0104] S2: Construct an industrial parts number extraction and recognition model;

[0105] Furthermore, the industrial part number extraction and recognition model includes a feature extraction unit based on adaptive graph convolution, a global information modeling unit based on the Transformer module, and a positioning and recognition module for part number positions and characters;

[0106] The feature extraction unit based on adaptive graph convolution is used to extract feature vectors of local part number regions in preprocessed image data; the feature extraction unit based on adaptive graph convolution includes a graph structure acquisition module, an adaptive graph convolution module, a local feature enhancement module, a multi-scale feature extraction module, and a local feature fusion module.

[0107] The graph structure acquisition module is used to define the numbered region pixels as nodes, and use the pixel value similarity between the pixels in each numbered region as edges, so as to transform the numbered regions in the preprocessed image data into a node graph structure, and obtain the node feature vector based on the node graph structure.

[0108] Specifically, transforming the numbered regions in an image into a graph structure is a key step in graph convolutional networks. In this embodiment, each pixel in the numbered region of the image is regarded as a node in the graph, and the edges in the graph represent the spatial relationship or pixel value similarity between nodes. In this way, the numbered regions in the image (i.e., the regions containing part numbers) are represented as a part of the graph structure, where each pixel in the region participates in the graph construction as an independent node. The graph construction method is based on the spatial distance between nodes. If two pixels are relatively close in space, an edge can be established between them. The weight of the edge is set according to the difference in pixel values ​​between nodes to reflect the relationship between different regions. In graph convolutional networks, each node (i.e., a pixel in the image) needs a feature representation. For each node, this embodiment initializes its feature vector to the pixel value at the location of the node. In grayscale images, the initial feature vector of a node is the grayscale value of the pixel, with a value range of [0,1], which is the normalized pixel value in the image.

[0109] And the expression for the node feature vector is:

[0110] X i =I(x) i ,y i )

[0111] In the formula: I(x) i ,y i ) represents the position (x) in the preprocessed image data. i ,y i Pixel value at (); X i Represents the feature vector of node i;

[0112] An adaptive graph convolution module is used to adjust the convolution kernel parameters of the adaptive graph convolutional network based on the node feature vectors. Specifically, in traditional graph convolutional networks, the convolution kernel is fixed and shared among all nodes. However, part numbers often have irregular shapes and are affected by background noise, so traditional convolution kernels may struggle to effectively capture the features of the part numbers. Therefore, this embodiment introduces an adaptive graph convolution kernel, enabling the kernel to be dynamically adjusted based on the local context of each node. The design method of the adaptive convolution kernel is as follows: for each node, the weights of its convolution kernel are adjusted based on the features of that node and the features of its neighboring nodes. Specifically, the node's feature X... i It will be used to update the weights of the adaptive convolution kernel by learning the features X of node i. i Based on the features of its neighboring nodes, the weights of the convolution kernel can be dynamically updated, allowing the kernel to adapt more accurately to the local context. The formula for adjusting the kernel parameters is as follows:

[0113] W i =f(X) i ,X j )

[0114] In the formula: W i Denotes the adaptive convolution kernel for node i; f(X) i ,X j Let be a fully connected network, and let represent the feature similarity function between node i and its neighboring node j;

[0115] The optimized node graph structure is obtained by performing graph convolution operations on the node feature vectors through the adjusted adaptive graph convolution network. Specifically, graph convolution is the core part of the graph convolution network. Through graph convolution, the local information of the image can be effectively propagated to every node of the image, thereby enhancing the feature representation of each node. It is worth noting that the graph convolution operation is applied to the graph structure, rather than directly convolving on the image obtained in step one. After the numbered region is transformed into a graph structure in step one, each pixel is regarded as a node in the graph. The graph convolution operation is performed on this graph structure, so that the local information of the image is propagated to each node through the graph structure.

[0116] And the expression for graph convolution operation is:

[0117]

[0118] In the formula: Let N(i) represent the output feature vector of node i at layer l+1, i.e., the updated feature representation; N(i) represent the neighboring nodes of node i, i.e., the set of nodes directly connected to node i by edges; W ij This represents the weight matrix between node i and its neighboring node j. This weight matrix plays an important role in the graph convolution process and is dynamically adjusted according to the node features. b represents the output feature vector of node i in the l-th layer; i represents the bias term for node i, used to adjust the output of node features; l represents the number of layers in the graph convolutional network; σ represents the activation function; W ij Features of node j There is a multiplicative relationship between them; specifically, the features of node i and its neighbor node j are expressed through the weight matrix W. ij When multiplied, the weight matrix can be viewed as adaptive (dynamically adjusted according to the local features of the nodes);

[0119] The local feature enhancement module is used to adjust and optimize the attention weights between nodes in the node graph structure based on the self-attention mechanism to obtain feature maps of local numbered regions.

[0120] Specifically, the features extracted by the graph convolutional network in this embodiment mainly focus on the structural information of local regions. However, since the shape, size, and font of part numbers may vary, relying solely on these local features may not accurately express the semantic information of the number. Therefore, using a self-attention mechanism to enhance the extracted local features can help the model better capture important information in the numbering region, especially in complex backgrounds. The self-attention mechanism assigns different attention weights to each node by calculating the similarity between nodes.

[0121] And the expression for the attention weights between nodes is:

[0122]

[0123] In the formula: α ij X represents the attention weight between node i and node j; i ,X jLet i represent the feature vectors of node i and node j. In this embodiment, the model can dynamically adjust the weights between nodes to enhance the feature representation of the numbered regions. However, simply assigning attention weights is not enough to directly enhance the features. The role of the attention mechanism is to model and weight the relationships between nodes, so that the features of each node can be updated according to its importance in the local and global context. In this way, nodes will pay more attention to important regions when updating features, especially in complex backgrounds and with a lot of interference, which can effectively strengthen the key features of the local numbered regions. Finally, the features adjusted by the self-attention mechanism will enable the model to more accurately identify and understand the semantic information of the numbered regions.

[0124] The multi-scale feature extraction module includes a first-scale convolutional layer, a second-scale convolutional layer, and a third-scale convolutional layer set in parallel. The first-scale convolutional layer is used to perform convolution operations on the feature maps of local numbered regions to obtain a first feature map. The second-scale convolutional layer is used to perform convolution operations on the feature maps of local numbered regions to obtain a second feature map. The third-scale convolutional layer is used to perform convolution operations on the feature maps of local numbered regions to obtain a third feature map.

[0125] Specifically, due to the variety of sizes and fonts used for industrial part numbers, the network needs to be able to adapt to numbering areas of different sizes. Therefore, this embodiment designs a multi-scale feature extraction module. Specifically, to process part numbering regions of different scales, this invention selects three commonly used convolutional kernel sizes: 3×3, 5×5, and 7×7. These convolutional kernels are suitable for numbering regions of different sizes. In small-sized numbering regions, the 3×3 convolutional kernel can effectively capture the detailed features and local texture information of the numbering region. For medium-sized numbering regions, the 5×5 convolutional kernel can extract a wider range of local information, enhancing the ability to recognize the shape of the numbering region. In large-sized numbering regions, the 7×7 convolutional kernel can capture a large range of image structure and global background information. Through this multi-scale convolution operation, this module can extract rich image features from feature maps of different scales, thereby improving the model's ability to recognize numbering regions of different sizes. This embodiment processes the input image through parallel convolution operations. In each convolution operation, the convolutional kernel performs convolution through a sliding window to generate feature maps of different scales. Each convolutional kernel corresponds to a feature map of a specific scale, and these feature maps are merged in the subsequent feature fusion process to combine feature information of different scales.

[0126] The local feature fusion module is used to stitch together the first feature map, the second feature map, and the third feature map to obtain the features of the local part number region.

[0127] Specifically, this module extracts image features at different scales using multiple convolutional kernels of varying sizes, enabling the network to process both large and small numbered regions simultaneously. In the multi-scale feature extraction module, features are extracted from the original image at different scales. Multiple convolutional layers process feature maps at different scales, with each layer performing convolution operations at a different scale to extract feature information at that scale. Finally, feature concatenation or weighted summation is used to fuse these multi-scale feature information, resulting in more comprehensive features of the numbered regions, particularly those of local part numbered regions.

[0128] The global information modeling unit based on the Transformer module is used to model global features based on the features of local part number regions, obtain global part number region feature vectors, and obtain enhanced region feature vectors by concatenating the global part number region feature vectors with the local part number region feature vectors. Specifically, in this embodiment, through the preceding graph convolution and local feature enhancement, the model has been able to capture relatively rich local features. Next, this embodiment combines local information with the global information obtained through the Transformer module through feature fusion, providing a more accurate feature representation for subsequent number recognition tasks. The global information is modeled by the Transformer module to capture the relationship between the number region and the background and enhance its semantic expression. The feature fusion process is completed through concatenation operations, and the final output is a composite feature representation containing global context information and local number features.

[0129] Specifically, the global information modeling unit based on the Transformer module includes a Transformer encoder module and a global feature fusion module;

[0130] The Transformer encoder module includes a multi-layered stacked Transformer structure;

[0131] The Transformer structure is used to obtain the global attention feature vector of the part number region based on the features of the local part number region using a self-attention mechanism.

[0132] Specifically, the Transformer, through its self-attention mechanism, can capture the global dependencies between different regions in an image. In the part number recognition task, the main role of the Transformer is to model the relationship between the part number region and the background based on the features of the local part number region through the self-attention mechanism, thereby obtaining global features. This modeling of global features enables the model to effectively distinguish between the part number region and the background, and this distinction usually depends on global information, helping to improve recognition accuracy. Therefore, in this embodiment, the Transformer encoder module takes the local features extracted by the Graph Convolutional Network (GCN) in step two as input to model global information. The local features extracted by the Graph Convolutional Network (GCN) are represented in the form of a graph structure, where each feature point represents a node, called a graph node. The input of the Transformer encoder module is the feature representation after GCN processing. These features already contain the local information of the number region. The output of the Transformer encoder is the global feature representation obtained after calculation by the self-attention mechanism. The self-attention mechanism of the Transformer is the core of its powerful function. It can determine how to weight the features by calculating the similarity between the query and key of the input features.

[0133] Furthermore, the formula for obtaining the self-attention mechanism of Transformer, i.e., the global attention feature, is as follows:

[0134] Q p =W Q h p K q =W K h q V q =W V h q

[0135] Where: h p The feature representation of node p; h q W represents the feature representation of node q. Q W K W V Q represents the learnable weight matrix; p K represents the query vector for node p; q V represents the key vector of node q; q Represents the value vector of node q;

[0136] The attention weights between node p and node q are calculated and obtained as follows:

[0137]

[0138] In the formula: αpq The attention weight between nodes p and q is represented by N(p); the set of neighboring nodes of node p is represented by T; and the transpose is represented by T. Through this calculation process, Transformer can automatically adjust the weights between nodes, thereby enhancing the feature representation of the numbered region.

[0139] This embodiment calculates the weight α using a self-attention mechanism. pq This is used to perform a weighted summation of the feature values, resulting in a new feature representation. This new feature representation will contain global information about the numbered regions, helping the model better understand the relationship between the numbers and the background. The formula for obtaining the global attention feature vector is:

[0140]

[0141] Where: h p 'Represents the global attention feature vector of the part number region; in this embodiment, features from different regions can be integrated in this way to improve the semantic understanding of the number region;

[0142] In a specific example, to further enhance feature modeling capabilities, the Transformer encoder module employs a multi-layered stacked structure. In each layer, the Transformer performs self-attention computation based on the input features and outputs an updated feature representation. The output of each layer serves as the input for the next layer. This stacked multi-layered structure effectively improves the depth and complexity of feature representation. Through multiple stacked Transformer encoder layers, the model can capture feature information of the numbered regions at different levels and scales. This section aims to explain the relationship between the structural composition of the Transformer module and the execution of self-attention computation. The self-attention mechanism of each layer is calculated based on the input feature nodes, using the similarity between the query and the key to generate weights. These weights are then used to weighted summation of the values ​​to obtain the updated feature representation. In this way, the model can dynamically adjust the weights between nodes in each layer, thereby enhancing the global information representation of the numbered regions. In each layer of the Transformer encoder, after self-attention computation and feature enhancement, the final output feature vector contains global information about the numbered region. By stacking multiple layers of Transformers, the model can capture richer global contextual information and enhance the representation ability of the numbered region. Finally, the output of the Transformer encoder module will contain detailed information about the numbered region, helping subsequent steps to better locate and identify the numbered region.

[0143] This embodiment further enhances the feature representation of the part numbering region through the Transformer encoder module and performs global information modeling. Although the Graph Convolutional Network (GCN) and adaptive feature extraction module have effectively extracted the local features of the part number, these local features still need to be further enhanced through global information modeling. In industrial part numbering recognition tasks, background interference, lighting changes, and the raised shape of the number may make it difficult for the local features of the number to express its semantics alone; therefore, this embodiment uses the Transformer module to perform global feature modeling of the numbering region, improving the model's ability to recognize part numbers in complex backgrounds.

[0144] The global feature fusion module is used to fuse the global attention feature vectors output by each Transformer structure to obtain the global part number region feature vector, and to obtain the enhanced region feature vector by concatenating the local part number region feature vectors.

[0145] Specifically, in this embodiment, after global information modeling through a multi-layer Transformer encoder, the final feature vector is fused with the extracted local part number region feature vector. Since the Transformer has enhanced the global information of the number region, while the Graph Convolutional Network (GCN) extracts the local features of the number region, feature fusion helps to effectively combine these two types of information and enhance the overall expressive power of the model. The feature fusion process is performed at the feature level (i.e., the feature vector layer), rather than at the feature map level. Specifically, the fusion method can use weighted summation or concatenation operations to ultimately generate a feature vector that integrates local and global information. In this embodiment, by fusing the features extracted by the Transformer encoder module and the Graph Convolutional Network, the final output feature vector will contain rich information about the number region. This information includes both local morphological features and global contextual information. These feature vectors will provide strong support for subsequent number position detection and character recognition, ensuring that the model can accurately identify part numbers in complex backgrounds.

[0146] The part number location and character positioning and recognition module is used to locate the industrial part number area and recognize the industrial part number characters based on the enhanced region feature vector;

[0147] In practical implementation, the part number location and character positioning and recognition module includes a region positioning module and a character recognition module based on a convolutional neural network (CNN).

[0148] S21: The region positioning module locates the industrial part number region based on the enhanced region feature vector, specifically including:

[0149] S211: A set of candidate bounding boxes for the localization region is obtained using the Region Proposal Network (RPN) based on the enhanced region feature vector. Specifically, the RPN generates candidate bounding boxes to accurately locate the part number region. After feature fusion, the final feature vector (i.e., a comprehensive feature vector containing local and global information) is used as input to generate multiple candidate bounding boxes through the RPN. These candidate bounding boxes represent regions that may contain the part number and will be further filtered and refined in subsequent steps. The RPN is a neural network commonly used for object detection. It generates multiple potential target regions from the feature map of the input image. These candidate regions will include regions that may contain the part number. In the RPN, each candidate bounding box has foreground and background scores. These scores are predicted by a binary classification network, representing the probability that the candidate bounding box is foreground (i.e., contains the part number) or background (i.e., does not contain the part number). The probability score is calculated using the following formula:

[0150]

[0151] In the formula: σ represents the Sigmoid activation function; W f W b Let b represent the weight parameters for the foreground and background, respectively, and h represent the candidate bounding box features extracted from the feature map; b f and b b These represent the foreground and background bias terms, respectively. In addition to the probability scores, RPN also generates bounding box regression parameters for each candidate box to further refine the boundaries of the candidate boxes. The output of RPN includes multiple candidate regions, which are filtered based on scores and overlap (using the non-maximum suppression method, NMS) to finally select the region most likely to contain the number.

[0152] S212: Based on bounding box regression and non-maximum suppression (NMS), filter and obtain the optimal localization region candidate box in the set of localization region candidate boxes;

[0153] In this embodiment, bounding box regression is used to fine-tune the position of the numbered regions based on the candidate boxes output by the RPN. The goal of bounding box regression is to optimize the size and position of the candidate boxes through regression parameters, so that the candidate boxes can more accurately surround the numbered regions. Specifically, bounding box regression fine-tunes the position and size of the candidate boxes by predicting four adjustment parameters (i.e., changes in coordinates, width, and height).

[0154] Non-maximum suppression (NMS): After the RPN generates multiple candidate boxes, non-maximum suppression (NMS) is used to filter these candidate boxes. The goal of NMS is to remove boxes with high overlap and low scores, thereby retaining the optimal numbered regions. It filters candidate boxes optimized by bounding box regression, not the original candidate boxes. The formula for calculating non-maximum suppression (NMS) is:

[0155]

[0156] In the formula: IOU represents the intersection-union ratio, which represents the overlap between two candidate boxes; if the overlap between two candidate boxes exceeds a preset threshold (set to 0.5), a candidate box will be selected based on the score, and other overlapping boxes will be removed;

[0157] S22: Using a character recognition module based on a convolutional neural network (CNN), identify and obtain the industrial part number characters in the candidate box of the optimal positioning region.

[0158] After determining the numbered regions, this embodiment requires character recognition of these regions. To this end, this embodiment uses a Convolutional Neural Network (CNN) and a Connectionist Temporal Classification (CTC) loss function for character recognition. The CNN is responsible for extracting local features from the numbered regions, while the CTC loss function helps the model train without requiring character alignment labels, thus enabling it to handle irregularly arranged numbered characters. In traditional character recognition tasks, characters are usually arranged in a fixed order, and each character needs to be aligned with its corresponding label. However, in practical applications, the arrangement of numbered characters may be irregular, and some characters may even be occluded or missing. To address this, "character alignment labels" refer to the explicit alignment of character labels with their positions in the image during standard training (e.g., the first position of the character "1" in the image corresponds to the label "1"). The advantage of the CTC loss function is that it can train the network without explicit alignment labels, i.e., it does not require manual labeling of the specific positions of the characters. Specifically, this includes:

[0159] S221: Extract low-dimensional local character features from the candidate boxes of the optimal localization region using a convolutional neural network (CNN), and convert the low-dimensional local character features into high-dimensional local character features.

[0160] S222: Input the high-dimensional local character features into a pre-set recurrent neural network (RNN) to obtain the probability distribution of the character sequence, and then output the predicted industrial part number characters.

[0161] S3: Train the constructed industrial part number extraction and recognition model based on the preprocessed image data to obtain the optimal industrial part number extraction and recognition model, so as to realize the extraction and recognition of industrial part numbers based on the optimal industrial part number extraction and recognition model.

[0162] In a specific embodiment, the method in S3 for training the constructed industrial part number extraction and recognition model based on preprocessed image data to obtain the optimal industrial part number extraction and recognition model includes the following steps:

[0163] S31: Divide the preprocessed image data into training and testing sets according to a preset ratio;

[0164] S32: Train the constructed industrial part number extraction and recognition model based on the training set to obtain the trained industrial part number extraction and recognition model:

[0165] S33: Use the CTC loss function as the model loss function, and evaluate the trained industrial part number extraction and recognition model based on the test set to determine whether the output of the trained industrial part number extraction and recognition model has converged.

[0166] If so, then the industrial part number extraction and recognition model trained at this time is the optimal industrial part number extraction and recognition model;

[0167] Otherwise, the hyperparameters of the trained industrial part number extraction and recognition model are optimized based on the backpropagation algorithm, and step S31 is repeated.

[0168] The model training process, specifically the training process for the part number location and character positioning and recognition module, includes:

[0169] S100: Input and Feature Extraction: First, the network receives images from numbered regions and uses CNN to extract features. The CNN processes the images through multiple convolutional and pooling layers, gradually extracting local features of the images and converting them into high-dimensional feature representations.

[0170] S200: CTC Loss Function: Next, these features are passed to a recurrent neural network (RNN) or similar sequence model, which outputs a probability distribution of a character. Since the arrangement of numbered characters may be irregular, the CTC loss function calculates the loss based on the probability distribution output by the model and the actual label (the label is not required to be strictly aligned with the image). CTC automatically learns the correct position of the character by decoding the possible character sequences.

[0171] S300: Backpropagation and Optimization: During training, the CTC loss function updates the model parameters through backpropagation, enabling the network to better recognize irregularly arranged characters. Through multiple iterations of training, the model learns to extract the correct character sequences from the numbered regions, even if these characters do not have strict alignment labels. Ultimately, through this training process, the network can accurately recognize and output the characters in the numbered regions, even if the order and position of these characters are irregular.

[0172] Specifically, the CTC loss function is suitable for sequence-to-sequence tasks, especially when the order and length of characters are not fixed. In number recognition, the CTC loss function is optimized by calculating the difference between the probability distribution of the character sequence output by the model and the true label. This embodiment includes image annotation of numbered regions, where each image sample is labeled with a true label. The true label is usually the character sequence corresponding to the numbered region extracted manually from the image. The preprocessed images are annotated and form a dataset containing images and corresponding character labels, which serve as input for subsequent training. The formula for the CTC loss is:

[0173] L CTC = -logP(y|x)

[0174] In the formula: x represents the input image; y represents the corresponding ground truth label; P(y|x) represents the probability of the character sequence predicted by the model. For example... Figures 5 to 7 As shown, after extracting features from the numbered regions using a CNN, these features are extracted from the character regions within the defined candidate boxes. The character recognition network passes these features to a fully connected layer and classifies them using a soft max layer to identify the characters within the numbered regions. The network output is a probability distribution containing the character categories. By maximizing the probability values, the model identifies the specific character with the number. During training, the model optimizes the network's hyperparameters (such as the number of filters in the convolutional layers and the learning rate) using a backpropagation algorithm. During each forward propagation, the network calculates the predicted character sequence probability distribution based on the input image and compares it with the labeled ground truth. The difference between the model output and the ground truth is calculated using the CTC loss function. Then, the error is propagated to each layer of the network through backpropagation to adjust the network's weights and biases, enabling the network to better fit the training data.

[0175] This embodiment also includes a post-processing procedure for character number recognition:

[0176] During character recognition, some misrecognitions may occur due to factors such as changes in lighting and background interference. To further improve recognition accuracy, this embodiment includes a post-processing step for correction. Specifically, the common number "1" and the letter "I" may be misrecognized. This embodiment corrects these errors through context analysis and character replacement, as detailed below:

[0177] S001: Context Analysis: The system will determine whether a character has been misidentified based on the relationship between the characters before and after it in the character sequence (e.g., number sequence, letter sequence, etc.). Suppose the identified character is "1a Z", but the actual character should be "Ia Z". Through the analysis of the character context, the system will automatically replace "1" with "I" to correct this misidentification.

[0178] Furthermore, this embodiment also includes a verification based on a predefined numbering format. The recognition result is first matched against the predefined numbering format to ensure that the output conforms to the expected structure. If the recognition result does not conform to the expected format, the system will automatically trigger format verification and re-correction. The specific operation steps are as follows:

[0179] S002: Number Format Verification: The system verifies the number according to a fixed format (e.g., number-letter-number-letter). If the output character sequence does not conform to this format (e.g., "123-ABC" is misidentified as "123-XYZ"), the system will rearrange the characters according to the format rules; for example, if the character order is incorrect, the system will change the characters "XYZ" to the correct character order "ABC".

[0180] S003: Recalibration: If the number format still does not meet the requirements after verification, the system will correct it using a rule-based replacement algorithm (e.g., replacement rules for common numbers and letters). For misidentification of characters, such as misidentifying "O" as "0", the system will perform character replacement to ensure that the final output conforms to the standard number format. Through these steps, the system can accurately correct misidentifications caused by factors such as changes in lighting and background interference, ensuring that the final identified number meets the actual requirements and conforms to the predefined number format.

[0181] This embodiment also includes a process for output and result evaluation:

[0182] The goal of this embodiment is to output the final result of part number recognition and evaluate the model. The optimal industrial part number extraction and recognition model can extract the number region from the image and recognize the number content. Next, this embodiment will output these recognition results and verify the performance of the model by comparing it with other existing models to ensure its feasibility of application in a real industrial environment.

[0183] A. Model output results:

[0184] The output of the optimal industrial part number extraction and identification model consists of two main parts:

[0185] (1) Location of numbered regions: The coordinates of the numbered regions in the image are determined by the precise locations of the candidate boxes and bounding boxes generated by the Region Proposal Network (RPN) and the bounding boxes. The location of each numbered region can be represented by the coordinates of the top left and bottom right corners, or by the center coordinates and width and height of the rectangle.

[0186] (2) Recognition of numbered content: The specific characters within the numbered region are identified using a character recognition network (CNN and CTC loss function). The recognized character content is output as a string, which typically includes numbers, letters, or combinations thereof.

[0187] For example, the input image may contain multiple part numbers, and the output after model processing is as follows:

[0188] Numbering range 1: Position (50, 80, 150, 120), Number content: "A12345"

[0189] Numbering range 2: Location (200, 300, 260, 340), Number content: "B67890"

[0190] This information will be used in industrial applications such as parts recording, classification, and tracking. To further verify the output results, the model marks the identified numbered regions with rectangles on the original image and labels the number content next to the corresponding region. In this way, users can intuitively see the accuracy of the model's recognition.

[0191] B. Output Result Evaluation:

[0192] To comprehensively evaluate the model's performance, this embodiment uses several standard evaluation metrics, mainly including: (1) Accuracy: This represents the proportion of numbered areas correctly identified by the model out of the total numbered areas. High accuracy indicates that the model can effectively distinguish part numbers from the background.

[0193] (2) Recall: This represents the proportion of the numbered regions identified by the model out of the actual numbered regions. A high recall rate means that the model can identify more real numbered regions.

[0194] (3) F1 score: Taking into account the balance between accuracy and recall, the F1 score is the harmonic average of the two and can measure the model’s performance in terms of accuracy and comprehensiveness.

[0195] (4) Character-level accuracy: This is an indicator for evaluating the accuracy of the model in character recognition. It represents the accuracy of the model in recognizing each character and is particularly suitable for character recognition tasks.

[0196] To verify the effectiveness of the model, this embodiment compares the method described herein with several existing part number recognition methods, focusing on the following models:

[0197] (1) Traditional Convolutional Neural Network (CNN) Model: Traditional CNN models use standard convolution operations to extract features from images. They are suitable for processing simple features, but perform poorly when faced with complex backgrounds and raised numbers.

[0198] (2) Object detection model based on Faster R-CNN: Faster R-CNN combines Region Proposal Network (RPN) and Convolutional Neural Network (CNN) for object localization and classification. It performs well in object detection tasks, but it is not good at capturing details of numbered regions.

[0199] (3) Feature extraction model based on ResNet: ResNet is a deep convolutional neural network model that excels at extracting deep features from complex images and has strong feature representation capabilities. However, relying solely on ResNet may not be sufficient to capture the raised features of part numbers in complex backgrounds.

[0200] This embodiment compares these models on a standard industrial parts image dataset to evaluate their performance in numbered region localization and character recognition. The dataset includes various lighting conditions, background noise, and part numbers of different sizes. The evaluation results of each model are shown in Table 1 based on the collected image data.

[0201] Table 1. Comparison of Model Evaluation Results

[0202] This method 97.6％ 96.4％ 97.0％ 98.2％ Faster R-CNN 94.5％ 91.8％ 93.5％ 92.4％ ResNet 91.2％ 87.7％ 90.4％ 91.2％ Traditional CNN 85.3％ 83.1％ 84.2％ 85.7％

[0203] The results show that the method described in this embodiment outperforms other existing models in terms of accuracy, recall, F1 score, and character-level accuracy. This indicates that the method described in this embodiment has significant advantages in recognizing complex backgrounds, lighting variations, and raised numbering features.

[0204] To further verify the model's practical application effectiveness, this embodiment simulates a real industrial scenario for testing. In this scenario, the images contain various types of interference, including:

[0205] The background is cluttered, the numbering is unclear due to changes in surface texture or lighting, and some part numbers are obscured or distorted.

[0206] The method described in this embodiment successfully and accurately located the part number area and identified the content in these complex scenarios, with an accuracy of 97.6% and a recall rate of 96.4%, which is far superior to traditional models.

[0207] By comparing experimental results and test results in simulated real-world scenarios, this embodiment concludes that the method described herein has significant advantages in part number recognition tasks. Combining the feature extraction methods of Graph Convolutional Networks (GCN) and Transformer modules effectively overcomes challenges such as complex backgrounds, lighting variations, and raised numbering features, providing a highly accurate part number recognition solution. In the industrial field, this method can significantly improve the efficiency of automated part identification and classification, and has broad application prospects.

[0208] The method described in this embodiment has the following beneficial effects:

[0209] (1) Introduction of Adaptive Graph Convolutional Network (AGCN): In order to cope with the raised features of the part number area and the interference of complex background, the introduced adaptive graph convolutional network can dynamically adjust the weight of the convolution kernel based on the local features of the image, efficiently extract the geometric shape and texture features of the part number, and improve the adaptability to complex background and uneven lighting conditions.

[0210] (2) Global information modeling of the Transformer module: Combining the self-attention mechanism of the Transformer model, this invention can effectively process global information in the image, capture the complex relationship between the numbered region and the background, and the Transformer module helps to improve the model's ability to distinguish between the numbered region and the background, strengthens the semantic expression of the numbered features, and can effectively improve the robustness and accuracy of the model.

[0211] (3) Combination of adaptive graph convolution and Transformer: This invention combines adaptive graph convolution network with Transformer module to give full play to the advantages of both in local feature extraction and global information modeling. This combination enables the model to accurately extract and identify part number regions in complex industrial environments, especially in situations with strong background interference and large changes in lighting conditions.

[0212] (4) Multi-level feature extraction and fusion mechanism: This invention designs a multi-level feature extraction and fusion mechanism to process the numbering region features at different scales. Local features are extracted through graph convolution, and combined with the global modeling capability of Transformer, the model can effectively cope with the diverse forms of numbering and improve the recognition accuracy in changing environments. The method described in this embodiment can effectively improve the accuracy and adaptability of part number recognition, and is particularly suitable for industrial scenarios with complex backgrounds, uneven lighting, and raised numbering, and has broad application prospects.

[0213] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An industrial part number extraction and recognition method based on adaptive graph convolution and Transformer, characterized in that, Specifically, the following steps are included: S1: Obtain RGB part image data containing industrial part numbers; The RGB part image data is then subjected to grayscale conversion, normalization, and image enhancement processing in sequence to obtain preprocessed image data. S2: Construct an industrial parts number extraction and recognition model; Furthermore, the industrial part number extraction and recognition model includes a feature extraction unit based on adaptive graph convolution, a global information modeling unit based on the Transformer module, and a positioning and recognition module for part number positions and characters; The feature extraction unit based on adaptive graph convolution is used to extract feature vectors of local part number regions in preprocessed image data. The feature extraction unit based on adaptive graph convolution includes a graph structure acquisition module, an adaptive graph convolution module, a local feature enhancement module, a multi-scale feature extraction module, and a local feature fusion module. The graph structure acquisition module is used to define the numbered region pixels as nodes, and use the pixel value similarity between the pixels in each numbered region as edges, so as to transform the numbered regions in the preprocessed image data into a node graph structure, and obtain the node feature vector based on the node graph structure. And the expression for the node feature vector is: In the formulae: represents a pixel value at a position in the preprocessed image data; represents a feature vector of a node ; The adaptive graph convolution module is used to adjust the convolution kernel parameters of the adaptive graph convolutional network based on the node feature vectors; the adjustment formula for the convolution kernel parameters is as follows: In the formula: represents a node of an adaptive convolution kernel; represents a feature similarity function between a node and its neighborhood nodes ; The node feature vectors are then subjected to graph convolution operations using an adjusted adaptive graph convolution network to obtain an optimized node graph structure. The expression for the graph convolution operation is as follows: In the formula: Indicates the first Level 1 node The output feature vector; Represents a node , neighboring nodes; Represents a node With neighboring nodes The weights of the parameters between them; Indicates the first Layer nodes The output feature vector; Represents a node The bias term; The local feature enhancement module is used to adjust and optimize the attention weights between nodes in the node graph structure based on the self-attention mechanism to obtain feature maps of local numbered regions. And the expression for the attention weights between nodes is: In the formula: Represents a node With nodes Attention weights between them; , This represents the nodes in an optimized node graph structure. With nodes eigenvectors; The multi-scale feature extraction module includes a first-scale convolutional layer, a second-scale convolutional layer, and a third-scale convolutional layer set in parallel. The first-scale convolutional layer is used to perform convolution operations on the feature maps of the local numbered regions to obtain the first feature map; the second-scale convolutional layer is used to perform convolution operations on the feature maps of the local numbered regions to obtain the second feature map; the third-scale convolutional layer is used to perform convolution operations on the feature maps of the local numbered regions to obtain the third feature map. The local feature fusion module is used to stitch together the first feature map, the second feature map, and the third feature map to obtain the features of the local part number region. The global information modeling unit based on the Transformer module is used to perform global feature modeling based on the features of the local part number region, obtain the global part number region feature vector, and obtain the enhanced region feature vector by concatenating the global part number region feature vector with the local part number region feature vector. The part number location and character positioning and recognition module is used to locate the industrial part number area and recognize the industrial part number characters based on the enhanced region feature vector; S3: Train the constructed industrial part number extraction and recognition model based on the preprocessed image data to obtain the optimal industrial part number extraction and recognition model, and realize the extraction and recognition of industrial part numbers based on the optimal industrial part number extraction and recognition model.

2. The method for extracting and recognizing industrial part numbers based on adaptive graph convolution and Transformer according to claim 1, characterized in that, S1 specifically includes the following steps: S11: Obtain RGB part image data containing industrial part numbers, and perform grayscale processing on it to obtain a grayscale image; S12: Normalize the grayscale image to obtain a normalized image; S13: Perform image enhancement processing on the normalized image to obtain preprocessed image data; and the image enhancement processing includes: performing noise reduction processing on the normalized image through a median filter; S14: Based on the Canny edge detection algorithm, edge features are extracted from the denoised normalized image to obtain an edge feature map; and then... The algorithm converts the edge feature map into a binary image; S15: Call preset In The function performs histogram equalization on a binarized image to obtain a histogram equalized image; and performs morphological operations on the histogram equalized image to obtain a morphological image. S16: Call preset In The function performs contour detection and extracts the numbering region of industrial parts in the morphological image; S17: Call preset In The function draws the extracted numbered region onto the RGB part image to obtain the numbered region bounding box and marks the industrial part number characters, and saves the image as a PNG format to obtain preprocessed image data.

3. The method for extracting and recognizing industrial part numbers based on adaptive graph convolution and Transformer according to claim 2, characterized in that, The global information modeling unit based on the Transformer module in S2 includes a Transformer encoder module and a global feature fusion module; The Transformer encoder module includes a multi-layered stacked Transformer structure; The Transformer structure is used to obtain the global attention feature vector of the part number region based on the features of the local part number region using a self-attention mechanism. And the formula for obtaining the global attention feature vector is: In the formula: This represents the global attention feature vector of the part number region; Represents a node and nodes Attention weights between them; Represents a node The set of neighboring nodes; , , Represents the learnable weight matrix; Represents a node The query vector; Represents a node The key vector; Represents a node The value vector; Represents a node Feature representation; Represents a node Feature representation; Indicates transpose; The global feature fusion module is used to fuse the global attention feature vectors output by each Transformer structure to obtain the global part number region feature vector. The enhanced region feature vector is obtained by splicing the feature vectors of the local part number region with the feature vectors of the local part number region.

4. The method for extracting and recognizing industrial part numbers based on adaptive graph convolution and Transformer according to claim 3, characterized in that, The part number position and character location recognition module in S2 includes a region positioning module and a character recognition module based on a convolutional neural network (CNN). S21: The region positioning module locates the industrial part number region based on the enhanced region feature vector, specifically including: S211: Obtain a set of candidate bounding boxes for the localized region based on the enhanced region feature vector through the region proposal network; S212: Based on bounding box regression and non-maximum suppression (NMS), the optimal localization region candidate box is selected and obtained from the set of localization region candidate boxes; S22: Using a character recognition module based on a convolutional neural network (CNN), identify and obtain the industrial part number characters in the candidate bounding box of the optimal positioning region, specifically including: S221: Extract low-dimensional local character features from the candidate boxes of the optimal localization region using a convolutional neural network (CNN), and convert the low-dimensional local character features into high-dimensional local character features. S222: Input the high-dimensional local character features into a pre-set recurrent neural network (RNN) to obtain the probability distribution of the character sequence, and then output the predicted industrial part number characters.

5. The method for extracting and recognizing industrial part numbers based on adaptive graph convolution and Transformer according to claim 4, characterized in that, The method in S3 for training the constructed industrial part number extraction and recognition model based on preprocessed image data to obtain the optimal industrial part number extraction and recognition model includes the following steps: S31: Divide the preprocessed image data into training and testing sets according to a preset ratio; S32: Train the constructed industrial part number extraction and recognition model based on the training set to obtain the trained industrial part number extraction and recognition model: S33: Use the CTC loss function as the model loss function, and evaluate the trained industrial part number extraction and recognition model based on the test set to determine whether the output of the trained industrial part number extraction and recognition model has converged. If so, then the industrial part number extraction and recognition model trained at this time is the optimal industrial part number extraction and recognition model; Otherwise, the hyperparameters of the trained industrial part number extraction and recognition model are optimized based on the backpropagation algorithm, and step S31 is repeated.

Images