Industrial part number extraction and identification method based on adaptive graph convolution and Transform

Abstract

The invention discloses an industrial part number extraction and identification method based on adaptive graph convolution and Transform, and the method comprises the steps: S1, obtaining RGB part image data containing an industrial part number, sequentially carrying out the graying processing, normalization processing and image enhancement processing of the RGB part image data, and obtaining the preprocessed image data; s2, constructing an industrial part number extraction and recognition model; and S3, performing model training on the constructed industrial part number extraction and recognition model according to the preprocessed image data to obtain an optimal industrial part number extraction and recognition model so as to realize industrial part number extraction and recognition according to the optimal industrial part number extraction and recognition model. According to the invention, the problem that the existing number extraction and identification technology cannot effectively cope with the number areas with complex structures and diversity, so that the number extraction and identification precision of the industrial parts is insufficient is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision and artificial intelligence, and particularly relates to an industrial part number extraction and recognition method based on adaptive graph convolution and Transformer. BACKGROUND

[0002] With the rapid development of industrial automation and intelligent manufacturing, the automatic extraction and recognition technology of part numbers has become a key link in industrial production management. Part numbers are used to identify different types of parts and are an important basis for production, material tracking, warehouse management, and quality control. In order to improve production efficiency and automation level, accurate recognition of part numbers has become an important technical demand in modern manufacturing industry. However, the current part number recognition method still has great challenges in the face of complex background, light change, uneven texture on the surface of the part, and convex number features.

[0003] Traditional part number recognition methods mainly rely on image processing techniques such as edge detection, morphological processing, etc. These methods often rely on hand-designed features to extract the number region. However, these traditional methods are often disturbed by environmental changes in practical applications. For example, uneven lighting, complex background, and possible stains or obstructions on the surface of the part make it very difficult to extract the number region. In addition, the font, size, and arrangement of part numbers are diverse, and sometimes there is deformation or occlusion, making it difficult for traditional methods to achieve efficient and accurate recognition.

[0004] In recent years, deep learning, especially convolutional neural networks (CNN), has made significant progress in image recognition. By automatically learning features in data, CNN has achieved very good results in many visual tasks, but there are still certain limitations in dealing with complex backgrounds and non-planar numbers (such as convex numbers). This is because traditional CNN networks often rely on fixed convolution kernels and cannot effectively deal with complex structures and diverse number regions. SUMMARY

[0005] The present application provides an industrial part number extraction and recognition method based on adaptive graph convolution and Transformer to overcome the above technical problems.

[0006] To achieve the above purpose, the technical scheme of the present application is:

[0007] An industrial part number extraction and recognition method based on adaptive graph convolution and Transformer, specifically comprising the following steps:

[0008] S1: Obtain RGB part image data containing industrial part numbers;

[0009] And the RGB part image data is sequentially subjected to grayscale processing, normalization processing and image enhancement processing to obtain preprocessed image data;

[0010] S2: Construct an industrial part number extraction and recognition model;

[0011] And the industrial part number extraction and recognition model comprises a feature extraction unit based on adaptive graph convolution, a global information modeling unit based on a Transformer module, and a part number position and character positioning and recognition module;

[0012] The feature extraction unit based on adaptive graph convolution is used to extract a local part number region feature vector in the preprocessed image data;

[0013] The global information modeling unit based on the Transformer module is used to perform global feature modeling according to the local part number region features, to obtain a global part number region feature vector, and to obtain an enhanced region feature vector by splicing the global part number region feature vector and the local part number region feature vector;

[0014] The part number position and character positioning and recognition module is used to realize positioning of the industrial part number region and recognition of the industrial part number characters according to the enhanced region feature vector;

[0015] S3: Model training is performed on the constructed industrial part number extraction and recognition model according to the preprocessed image data, to obtain an optimal industrial part number extraction and recognition model, so as to realize industrial part number extraction and recognition according to the optimal industrial part number extraction and recognition model.

[0016] Further, the S1 specifically comprises the following steps:

[0017] S11: Obtain RGB part image data containing an industrial part number, and perform grayscale processing on the same to obtain a grayscale image;

[0018] S12: Perform normalization processing on the grayscale image to obtain a normalized image;

[0019] S13: Perform image enhancement processing on the normalized image to obtain preprocessed image data; and the image enhancement processing comprises: performing denoising processing on the normalized image by a median filter;

[0020] S14: Based on a Canny edge detection algorithm, edge feature extraction is performed on the normalized image after denoising processing to obtain an edge feature map; and the edge feature map is converted into a binary image by an Otsu algorithm;

[0021] S15: calling the preset cv2.equalizeHist function in OpenCV to perform histogram equalization on the binary image to obtain a histogram equalization image; and performing morphological operation on the histogram equalization image to obtain a morphological image;

[0022] S16: calling the preset cv2.findContours function in OpenCV to perform contour detection to extract a number region of the industrial part number in the morphological image;

[0023] S17: calling the preset cv2.drawContours function in OpenCV to draw the extracted number region to the RGB part image to obtain a number region frame and mark the industrial part number character, and save the picture as a PNG format to obtain preprocessed image data.

[0024] Further, the S2 adaptive graph convolution-based feature extraction unit includes a graph structure acquisition module, an adaptive graph convolution module, a local feature enhancement module, a multi-scale feature extraction module, and a local feature fusion module.

[0025] The graph structure acquisition module is configured to define the number region pixel points as nodes, and the pixel value similarity between the number region pixel points as edges, so as to convert the number region in the preprocessed image data into a node graph structure, and acquire a node feature vector according to the node graph structure.

[0026] The expression of the node feature vector is

[0027] X i =I(x i ,y i )

[0028] In the formula, I(x i ,y i ) represents the pixel value at position (x i ,y i ) in the preprocessed image data; and X i represents the feature vector of node i.

[0029] The adaptive graph convolution module is configured to adjust the convolution kernel parameter of the adaptive graph convolution network according to the node feature vector; and the adjustment formula of the convolution kernel parameter is

[0030] W i =f(X i ,X j )

[0031] In the formula, W i represents the adaptive convolution kernel of node i; and f(X i ,X jrepresents a feature similarity function between node i and its neighborhood node j;

[0032] and the expression of the graph convolution operation is

[0033]

[0034] wherein: represents the output feature vector of the (l+1)th layer node i; N(i) represents the neighborhood nodes of node i; W ij represents the parameter weight between node i and neighborhood node j; represents the output feature vector of the lth layer node i; b i represents the bias term of node i;

[0035] The local feature enhancement module is configured to adjust the attention weights between nodes in the optimized node graph structure based on a self-attention mechanism to obtain a local numbered region feature map.

[0036] The expression of the attention weights between nodes is

[0037]

[0038] wherein: α ij represents the attention weight between node i and node j; X i ,X j represents the feature vectors of node i and node j in the optimized node graph structure.

[0039] The multi-scale feature extraction module includes a first scale convolution layer, a second scale convolution layer, and a third scale convolution layer arranged in parallel.

[0040] The first scale convolution layer is configured to perform convolution operation on the local numbered region feature map to obtain a first feature map; the second scale convolution layer is configured to perform convolution operation on the local numbered region feature map to obtain a second feature map; and the third scale convolution layer is configured to perform convolution operation on the local numbered region feature map to obtain a third feature map.

[0041] The local feature fusion module is configured to perform splicing operation on the first feature map, the second feature map, and the third feature map to obtain a local part numbered region feature.

[0042] Further, the global information modeling unit based on the Transformer module in S2 includes a Transformer encoder module and a global feature fusion module.

[0043] The Transformer encoder module includes a multi-layer stacked Transformer structure.

[0044] The Transformer structure is used to obtain the global attention feature vector of the part number region based on the local part number region features based on the self-attention mechanism;

[0045] And the formula for obtaining the global attention feature vector is

[0046]

[0047]

[0048] Q p =W Q h p , K q =W K h q , V q =W V h q

[0049] Where: h′ p The global attention feature vector representing the part number region; α pq represents the attention weight between node p and node q; N(p) represents the set of neighboring nodes of node p; W Q ,W K ,W V represents the learnable weight matrix; Q p represents the query vector of node p; K q represents the key vector of node q; V q represents the value vector of node q; h p represents the feature representation of node p; h q represents the feature representation of node q; T represents transpose;

[0050] The global feature fusion module is used to fuse the global attention feature vectors output by each Transformer structure to obtain the global part number region feature vector;

[0051] And by concatenating the local part number region feature vector and the local part number region feature vector, an enhanced region feature vector is obtained.

[0052] Furthermore, the positioning and recognition module for the part number position and characters in S2 includes a region positioning module and a character recognition module based on a convolutional neural network (CNN);

[0053] S21: Using the region positioning module, the industrial part number region is positioned based on the enhanced region feature vector, specifically including:

[0054] S211: obtaining a positioning region candidate box set according to the enhanced region feature vector through a region proposal network;

[0055] S212: screening and obtaining an optimal positioning region candidate box in the positioning region candidate box set based on a boundary box regression and a non-maximum suppression (NMS);

[0056] S22: identifying and obtaining industrial part number characters in the optimal positioning region candidate box through a character recognition module based on a convolutional neural network (CNN), specifically comprising:

[0057] S221: extracting low-latitude local character features in the optimal positioning region candidate box through a convolutional neural network (CNN) and converting the low-latitude local character features into high-latitude local character features;

[0058] S222: inputting the high-latitude local character features into a preset recurrent neural network (RNN) to obtain a probability distribution of a character sequence and then outputting predicted industrial part number characters.

[0059] Further, the method for obtaining the optimal industrial part number extraction and recognition model according to the preprocessed image data for training the constructed industrial part number extraction and recognition model in S3 comprises the following steps:

[0060] S31: dividing the preprocessed image data into a training set and a test set according to a preset proportion;

[0061] S32: training the constructed industrial part number extraction and recognition model according to the training set to obtain a trained industrial part number extraction and recognition model:

[0062] S33: taking a connectionist temporal classification (CTC) loss function as a model loss function and evaluating the trained industrial part number extraction and recognition model according to the test set to determine whether the output of the trained industrial part number extraction and recognition model converges;

[0063] If yes, the trained industrial part number extraction and recognition model is the optimal industrial part number extraction and recognition model at this time;

[0064] Otherwise, the hyperparameters of the trained industrial part number extraction and recognition model are optimized based on a back propagation algorithm, and step S31 is repeatedly executed.

[0065] The present application provides an industrial part number extraction and recognition method based on adaptive graph convolution and Transformer, which has the following beneficial effects:

[0066] (1) The introduction of adaptive graph convolutional network (AGCN): In order to cope with the convex features of the part number area and the complex background interference, the adaptive graph convolutional network introduced can dynamically adjust the convolution kernel weight based on the local features of the image, efficiently extract the geometric shape and texture features of the part number, and improve the adaptability to complex background and uneven illumination.

[0067] (2) Global information modeling of the Transformer module: Combined with the self-attention mechanism of the Transformer model, the application can effectively process the global information in the image and capture the complex relationship between the numbering area and the background. The Transformer module helps to improve the model's ability to distinguish between the numbering area and the background, strengthens the semantic expression of the numbering features, and effectively improves the robustness and accuracy of the model.

[0068] (3) Combination of adaptive graph convolution and Transformer: The application combines adaptive graph convolutional network with Transformer module, fully utilizing the advantages of both in local feature extraction and global information modeling. This combination enables the model to accurately extract and recognize the part number area in complex industrial environments, especially in cases with strong background interference and large changes in lighting conditions.

[0069] (4) Multi-level feature extraction and fusion mechanism: The application designs a multi-level feature extraction and fusion mechanism to process different scale features of the numbering area. By combining graph convolution to extract local features and the global modeling capability of the Transformer, the model can effectively cope with the diversified forms of numbering and improve the recognition accuracy in varying environments. BRIEF DESCRIPTION OF DRAWINGS

[0070] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiment or prior art description. Obviously, the drawings in the following description are some embodiments of the application, and for those skilled in the art, other drawings can be obtained without creative labor based on these drawings.

[0071] Figure 1 Flowchart of the industrial part number extraction and recognition method based on adaptive graph convolution and Transformer of the application;

[0072] Figure 2 Acquisition of the first RGB part image data containing the industrial part number in this embodiment;

[0073] Figure 3 Acquisition of the second RGB part image data containing the industrial part number in this embodiment;

[0074] Figure 4 The third RGB part image data containing the industrial part number is acquired in this embodiment;

[0075] Figure 5 The recognition result map of the first RGB part image data in this embodiment;

[0076] Figure 6 The recognition result map of the second RGB part image data in this embodiment;

[0077] Figure 7 The recognition result map of the third RGB part image data in this embodiment. DETAILED DESCRIPTION

[0078] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described below in conjunction with the accompanying drawings of the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0079] The embodiment provides an industrial part number extraction and recognition method based on adaptive graph convolution and Transformer, as shown in Figure 1 The embodiment specifically comprises the following steps:

[0080] S1: acquiring RGB part image data containing an industrial part number;

[0081] Specifically, image acquisition is performed by using an industrial camera with a resolution of 1024x1024 pixels. Specifically, the Basler acA2440-35um camera is used in the embodiment, so that the image is clear during acquisition, and overexposure or uneven illumination is avoided. Each image ensures that the part number area is included, and is shot under uniform illumination conditions. The acquired original image is as shown in Figures 2 to 4

[0082] The RGB part image data is sequentially subjected to grayscale processing, normalization processing, and image enhancement processing to obtain preprocessed image data, specifically comprising the following steps:

[0083] S11: acquiring RGB part image data containing an industrial part number, and performing grayscale processing to obtain a grayscale image;

[0084] Specifically, the acquired color image is converted into a grayscale image, and the conversion formula is:

[0085] I​gray (x,y) = 0.2989 · R(x,y) + 0.5870 · G(x,y) + 0.1140 · B(x,y)

[0086] I(x,y) = 0.2989 · R(x,y) + 0.5870 · G(x,y) + 0.1140 · B(x,y) gray (x,y) represents the pixel value of the grayscale image, R(x,y), G(x,y), B(x,y) represent the red, green and blue channel pixel values in the RGB image, and the conversion formula is close to the sensitivity of the human eye to color in visual perception;

[0087] S12: normalizing the grayscale image to obtain a normalized image;

[0088] Specifically, the pixel value is adjusted to the range of [0, 1], and the normalization formula is:

[0089]

[0090] I(x,y) = 0.2989 · R(x,y) + 0.5870 · G(x,y) + 0.1140 · B(x,y) norm (x,y) represents the pixel value of the grayscale image, I gray (x,y) represents the pixel value of the original grayscale image, and after normalization, all pixel values are located in the interval [0, 1], which is convenient for subsequent processing;

[0091] S13: performing image enhancement processing on the normalized image to obtain preprocessed image data; and the image enhancement processing includes: performing denoising processing on the normalized image by a median filter;

[0092] Specifically, for the normalized image, a 3x3 size median filter is used to denoise the image, remove noise caused by small particles such as sand particles, and retain edge and texture information;

[0093] S14: based on the Canny edge detection algorithm, edge feature extraction is performed on the denoised normalized image to obtain an edge feature map; and the edge feature map is converted into a binary image by the Otsu algorithm;

[0094] Specifically, the Canny edge detection algorithm is used to extract the edges of the image, and the high and low threshold values are set to 100 and 200 respectively, and the formula for Canny edge detection is:

[0095]

[0096] wherein: ▽I represents the gradient value of the image in the x and y directions; and respectively represent the change rate of the image in the horizontal direction and the vertical direction; here I refers to the normalized image I norm(x, y); the gradient value calculated by this formula, the Canny algorithm can effectively extract the edge information in the image; then the Otsu algorithm is used to automatically determine the best threshold to convert the image into a binary image, so as to further separate the numbering area and the background, wherein the Otsu algorithm is to select the best threshold by maximizing the inter-class variance;

[0097] S15: calling the preset cv2.equalizeHist function in OpenCV to perform histogram equalization on the binary image to obtain a histogram equalization image; and performing morphological operation on the histogram equalization image to obtain a morphological image;

[0098] Specifically, the histogram equalization is performed by using the cv2.equalizeHist function in OpenCV to enhance the contrast of the image, and especially when the image contrast is low, the equalized image helps to improve the recognition of the numbering area; then the morphological operation is performed, the dilation operation is adopted to enhance the convex feature of the numbering area, and the kernel size is set to 5x5 rectangular structural element;

[0099] S16: calling the preset cv2.findContours function in OpenCV to perform contour detection to extract the numbering area of the industrial part number in the morphological image;

[0100] Specifically, the cv2.findContours function in OpenCV is used to perform contour detection to extract the numbering area in the image, that is, the cv2.findContours function is called, and the return value is set to RETR_EXTERNAL mode to detect only the external contour, and the function calling rule is:

[0101] contours,_=cv2.findContours(binary_image,cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE);In this embodiment, the area of each contour detected is calculated, and the contour with larger area is selected as the numbering area, and the possible numbering area is screened out through the area threshold (2000 pixels or more) of the contour;

[0102] S17: calling the preset cv2.drawContours function in OpenCV to draw the extracted numbering area to the RGB part image to obtain a numbering area frame and mark the industrial part number character, and save the picture as PNG format to obtain the preprocessed image data.

[0103] Specifically, the extracted numbered region is drawn onto the image by using the cv2.drawContours function to mark the specific position of the number, and the final image is an image containing a numbered region frame and corresponding numbered characters, which helps the subsequent step to further identify and process the numbered region.

[0104] S2: Construct an industrial part number extraction and recognition model;

[0105] And the industrial part number extraction and recognition model includes a feature extraction unit based on adaptive graph convolution, a global information modeling unit based on a Transformer module, and a part number position and character positioning and recognition module.

[0106] The feature extraction unit based on adaptive graph convolution is used to extract the local part number region feature vector in the preprocessed image data; wherein the feature extraction unit based on adaptive graph convolution includes a graph structure acquisition module, an adaptive graph convolution module, a local feature enhancement module, a multi-scale feature extraction module, and a local feature fusion module.

[0107] The graph structure acquisition module is used to define the numbered region pixel points as nodes, and the pixel value similarity between each numbered region pixel point as an edge, so as to convert the numbered region in the preprocessed image data into a node graph structure, and acquire a node feature vector according to the node graph structure.

[0108] Specifically, converting the numbered region in the image into a graph structure is a key step of the graph convolution network, and the embodiment converts each pixel point in the numbered region in the image into a node in the graph, and the edge in the graph represents the spatial relationship or pixel value similarity between the nodes. In this way, the numbered region (i.e., the region containing the part number) in the image is represented as a part of the structure in the graph, where each pixel point in the region participates in the construction of the graph as an independent node. The construction method of the graph is based on the spatial distance between the nodes. If two pixel points are relatively close in space, an edge connection can be established between them. The weight of the edge is set according to the pixel value difference between the nodes to reflect the relationship between different regions. In the graph convolution network, each node (i.e., a pixel point in the image) needs a feature representation. For each node, the embodiment initializes its feature vector as the pixel value at the position of the node. In a grayscale image, the initial feature vector of the node is the grayscale value of the pixel, with a value range of [0, 1], i.e., the normalized pixel value in the image.

[0109] And the expression of the node feature vector is

[0110] X i =I(x i ,y i )

[0111] Where: I(x i ,y i ) represents the position (x i ,y i ) pixel value at X i Represents the feature vector of node i;

[0112] The adaptive graph convolution module is used to adjust the convolution kernel parameters of the adaptive graph convolution network according to the node feature vector. Specifically, in the traditional graph convolution network, the convolution kernel is fixed and shared among all nodes. However, part numbers usually have irregular shapes and are interfered by background noise. Therefore, the traditional convolution kernel may not be able to effectively capture the features of the part numbers. Therefore, this embodiment introduces an adaptive graph convolution kernel, so that the convolution kernel can be dynamically adjusted according to the local context of each node. The design method of the adaptive convolution kernel is: for each node, the weight of its convolution kernel is adjusted according to the features of the node and the features of its neighboring nodes. Specifically, the feature X i It will be used to update the weight of the adaptive convolution kernel by learning the feature X of node i i and its neighborhood nodes, can dynamically update the weight of the convolution kernel, so that the convolution kernel can adapt to the local context more accurately, and the adjustment formula of the convolution kernel parameters is

[0113] W i =f(X i ,X j )

[0114] Where: W i represents the adaptive convolution kernel of node i; f(X i ,X j ) is a fully connected network, representing the feature similarity function between node i and its neighboring node j;

[0115] The node feature vector is then convolved with the adjusted adaptive graph convolution network to obtain an optimized node graph structure. Specifically, the graph convolution operation is the core part of the graph convolution network. Through the graph convolution operation, the local information of the image can be effectively propagated to each node of the image, thereby enhancing the feature representation of each node. It is worth noting that the graph convolution operation is applied to the graph structure, rather than directly performing convolution on the image obtained in step 1. After the numbered region is converted into a graph structure in step 1, each pixel is regarded as a node in the graph. The graph convolution operation is performed on the graph structure, so that the local information of the image is propagated to each node through the graph structure.

[0116] And the expression of graph convolution operation is

[0117]

[0118] In the formula: represents the output feature vector of the i-th node in the l+1-th layer, i.e. the updated feature representation; N(i) represents the neighborhood nodes of node i, i.e. the node set connected with node i through direct edges; W ij represents the weight matrix between node i and neighborhood node j, which plays an important role in the graph convolution process, and is dynamically adjusted according to the node features; represents the output feature vector of the i-th node in the l-th layer; b i represents the bias term of node i, used to adjust the output of node features; l represents the number of layers of the graph convolution network; σ represents the activation function; W ij and the feature of node j There is a multiplication relationship between them, specifically, the features of node i and its neighborhood node j are multiplied by the weight matrix W ij The weight matrix can be regarded as adaptive (dynamically adjusted according to the local features of the node);

[0119] The local feature enhancement module is used to adjust and optimize the attention weight between nodes in the node graph structure based on the self-attention mechanism to obtain a local number region feature map;

[0120] Specifically, the features extracted by the graph convolution network mainly focus on the structure information of the local region, but due to the diversity of the shape, size and font of the part number, relying only on these local features may not be able to accurately express the semantic information of the number. Therefore, using the self-attention mechanism to enhance the extracted local features can help the model better capture important information of the number region, especially in a complex background; the self-attention mechanism calculates the similarity between nodes to assign different attention weights to each node;

[0121] And the expression of the attention weight between nodes is

[0122]

[0123] In the formula: α ij represents the attention weight between node i and node j; X i ,X jdenotes the feature vector of node i and node j; in this embodiment, in this way, the model can dynamically adjust the weight between nodes, thereby enhancing the feature expression of the numbered area. However, merely assigning attention weights is not enough to directly enhance the features. The role of the attention mechanism is that through modeling and weighting the relationship between nodes, the features of each node can be updated according to the importance thereof in the local and global context, and in this way, the node will pay more attention to important areas when updating the features, especially in the case of a complex background and more interference, the key features of the local numbered area can be effectively strengthened; finally, the features adjusted through the self-attention mechanism will enable the model to more accurately identify and understand the semantic information of the numbered area;

[0124] The multi-scale feature extraction module includes a first scale convolution layer, a second scale convolution layer and a third scale convolution layer arranged in parallel; the first scale convolution layer is used for performing convolution operation on the local numbered area feature map to obtain a first feature map; the second scale convolution layer is used for performing convolution operation on the local numbered area feature map to obtain a second feature map; and the third scale convolution layer is used for performing convolution operation on the local numbered area feature map to obtain a third feature map;

[0125] Specifically, due to the diversity of the scale and font of the industrial part number, the network needs to be able to adapt to numbered areas of different sizes. Therefore, a multi-scale feature extraction module is designed in this embodiment; specifically, in order to process numbered areas of different scales, three commonly used convolution kernel sizes are selected in this embodiment: 3x3, 5x5 and 7x7; these convolution kernels are suitable for numbered areas of different sizes, in small-sized numbered areas, using a 3x3 convolution kernel can effectively capture the detailed features and local texture information of the numbered area; for medium-sized numbered areas, using a 5x5 convolution kernel can extract more extensive local information and enhance the recognition ability of the numbered area form, and in large-sized numbered areas, using a 7x7 convolution kernel can capture large-scale image structures and global background information; through this multi-scale convolution operation, this module can extract rich image features from feature maps of different scales, thereby improving the recognition ability of the model for numbered areas of different sizes; in this embodiment, the input image is processed through parallel convolution operation, in each convolution operation, the convolution kernel is convolved through a sliding window to generate feature maps of different scales, each convolution kernel corresponds to a feature map of a specific scale, and these feature maps are merged in the subsequent feature fusion process in order to combine the feature information of different scales;

[0126] The local feature fusion module is used for performing splicing operation on the first feature map, the second feature map and the third feature map to obtain a local part numbered area feature;

[0127] Specifically, the module extracts image features at different scales through multiple convolution kernels of different sizes, so that the network can process large and small size numbered regions at the same time. Specifically, in the multi-scale feature extraction module, by performing feature extraction at different scales on the original image, multiple convolution layers are used to process feature maps at different scales respectively, each convolution layer performs convolution operation on the image at a different scale to extract feature information at that scale; finally, the multi-scale feature information is fused by using feature splicing or weighted summation, so as to obtain more comprehensive numbered region features, especially the features of local part numbered regions.

[0128] The global information modeling unit based on the Transformer module is used to model global features based on the local part numbered region features, obtain a global part numbered region feature vector, and obtain an enhanced region feature vector by splicing the global part numbered region feature vector and the local part numbered region feature vector; specifically, through the previous graph convolution and local feature enhancement, the model can capture relatively rich local features in the embodiment, and then the embodiment combines local information with global information obtained through the Transformer module by feature fusion, to provide more accurate feature representation for subsequent numbering recognition tasks; the global information is modeled through the Transformer module to capture the relationship between the numbered region and the background and enhance the semantic expression thereof. The feature fusion process is completed through splicing operation, and the final output is a composite feature representation containing global context information and local numbered features;

[0129] Specifically, the global information modeling unit based on the Transformer module includes a Transformer encoder module and a global feature fusion module.

[0130] The Transformer encoder module includes a multi-layer stacked Transformer structure.

[0131] The Transformer structure is used to obtain a global attention feature vector of the part numbered region based on a self-attention mechanism according to the local part numbered region features.

[0132] Specifically, the Transformer can capture the global dependency between various regions in the image through its self-attention mechanism. In the part number recognition task, the main role of the Transformer is to model the relationship between the local part number region and the background according to the features of the local part number region, so as to obtain global features. This modeling of global features enables the model to effectively distinguish the part number region from the background, which usually relies on global information, helping to improve recognition accuracy. Therefore, the embodiment uses the Transformer encoder module to model global information by taking the local features extracted by the step two graph convolutional network (GCN) as input. The local features extracted by the graph convolutional network (GCN) are represented in the form of a graph structure, where each feature point represents a node, referred to as a graph node. The input of the Transformer encoder module is the feature representation processed by the GCN, which already contains local information of the number region. The output of the Transformer encoder is the global feature representation obtained after the self-attention mechanism is calculated. The self-attention mechanism of the Transformer is the core of its powerful function, which can determine how to weight the features by calculating the similarity between the query (Query) and the key (Key) of the input features.

[0133] The self-attention mechanism of the Transformer, i.e., the formula for obtaining global attention features, is as follows:

[0134] Q p = W Q h p , K q = W K h q , V q = W V h q

[0135] where h p represents the feature representation of node p; h q represents the feature representation of node q; W Q , W K , and W V represent learnable weight matrices; Q p represents the query vector of node p; K q represents the key vector of node q; and V q represents the value vector of node q.

[0136] The attention weight between node p and node q is calculated and obtained as follows:

[0137]

[0138] where αpq represents the attention weight between node p and node q; N(p) represents the neighborhood node set of node p; T represents the transpose; through this calculation process, the Transformer can automatically adjust the weight between nodes, thereby strengthening the feature expression of the numbered area;

[0139] The weight α calculated by the self-attention mechanism in this embodiment pq is used for weighted summation of the feature values, thereby generating a new feature representation. The new feature representation will contain global information of the numbered area, helping the model better understand the relationship between the numbering and the background, and the acquisition formula of the global attention feature vector is

[0140]

[0141] In the formula: h p represents the global attention feature vector of the part numbering area; in this way, this embodiment can integrate features from different areas to improve the semantic understanding of the numbering area;

[0142] In specific examples, in order to further improve the feature modeling capability, the Transformer encoder module adopts a multi-layer stacking structure. In each layer, the Transformer will perform self-attention calculation according to the input features and output updated feature representation. The output of each layer will be used as the input of the next layer. The multi-layer stacked structure can effectively improve the depth and complexity of feature expression. Through multiple stacked Transformer encoder layers, the model can capture feature information of the numbered area from different levels and scales. This part aims to explain the execution relationship between the structure composition of the Transformer module and the self-attention calculation. The self-attention mechanism of each layer is calculated according to the input feature nodes, the similarity between the query (Query) and the key (Key) is used to generate the weight, and the value (Value) is weighted and summed through the weight to obtain the updated feature representation. In this way, the model can dynamically adjust the weight between nodes in each layer, thereby enhancing the global information expression of the numbered area. In each layer of the Transformer encoder, after self-attention calculation and feature enhancement, the final output feature vector contains the global information of the numbered area. By stacking multiple layers of Transformers, the model can capture more rich global context information and enhance the representation ability of the numbered area. Finally, the output of the Transformer encoder module will contain detailed information of the numbered area, helping the subsequent steps to better locate and recognize the numbered area.

[0143] The embodiment further strengthens the feature representation of the number region through the Transformer encoder module and performs global information modeling. Although the local features of the part number have been effectively extracted by the graph convolution network (GCN) and the adaptive feature extraction module, these local features still need to be further enhanced through the modeling of global information. In the industrial part number recognition task, background interference, illumination changes and the convex shape of the number may make it difficult for the local features of the number to express their semantics alone. Therefore, the embodiment models the global features of the number region through the Transformer module to improve the ability of the model to recognize part numbers in complex backgrounds.

[0144] The global feature fusion module is configured to perform feature fusion on the global attention feature vectors output by the Transformer structures to obtain a global part number region feature vector, and obtain an enhanced region feature vector by splicing the local part number region feature vector and the local part number region feature vector.

[0145] Specifically, after the global information modeling by the multi-layer Transformer encoder, the final feature vector will be fused with the extracted local part number region feature vector. Since the Transformer has enhanced the global information of the number region and the graph convolution network (GCN) has extracted the local features of the number region, feature fusion helps to effectively combine these two types of information and enhance the overall expression ability of the model. The feature fusion process is performed at the feature level (i.e., the feature vector level) rather than at the feature map level. Specifically, the fusion method can use weighted summation or splicing operation to finally generate a feature vector that integrates local and global information. In the embodiment, the features extracted by the Transformer encoder module and the graph convolution network are fused, and the final output feature vector will contain rich number region information, including both local morphological features and global contextual information. These feature vectors will provide strong support for subsequent number position detection and character recognition, ensuring that the model can accurately recognize part numbers in complex backgrounds.

[0146] The part number position and character positioning and recognition module is configured to realize positioning of the industrial part number region and recognition of the industrial part number characters according to the enhanced region feature vector.

[0147] In specific implementation, the part number position and character positioning and recognition module includes a region positioning module and a character recognition module based on a convolutional neural network (CNN).

[0148] S21: Realize positioning of the industrial part number region according to the enhanced region feature vector through the region positioning module, specifically including:

[0149] S211: Obtain a set of positioning region candidate boxes according to the enhanced region feature vector through a region proposal network; specifically, the region proposal network (RPN): in order to accurately locate the part number region, the embodiment first generates a candidate box by using the region proposal network (RPN). After feature fusion, the final obtained feature vector (i.e., a comprehensive feature vector containing local features and global information) will be input to generate multiple candidate boxes through the RPN. These candidate boxes represent regions that may contain part numbers, and will be further processed, screened and finely positioned in subsequent steps. The RPN is a neural network commonly used for target detection, which generates multiple potential target regions through the feature map of the input image. These candidate regions will include regions that may contain part numbers; in the RPN, each candidate box will have foreground and background scores, which are predicted by a binary classification network, respectively representing the probability of the candidate box being foreground (i.e., containing numbers) and background (i.e., not containing numbers). The calculation formula of the probability score is:

[0150]

[0151] In the formula, σ represents the Sigmoid activation function; W f ,W b respectively represent the foreground and background weight parameters, and h represents the candidate box features extracted from the feature map; b f and b b respectively represent the foreground and background bias terms; in addition to the probability score, the RPN also generates boundary box regression parameters for each candidate box to further accurately locate the boundary of the candidate box; the output of the RPN includes multiple candidate regions, which are screened according to the score and overlap (using the non-maximum suppression method, NMS), and finally the region most likely to contain numbers is selected;

[0152] S212: Based on boundary box regression and non-maximum suppression NMS, screen and obtain the optimal positioning region candidate box in the set of positioning region candidate boxes;

[0153] Boundary box regression of the embodiment: through the candidate box output by the RPN, the model will further use boundary box regression to fine-tune the position of the number region; the goal of boundary box regression is to optimize the size and position of the candidate box through regression parameters, so that the candidate box can more accurately surround the number region. Specifically, boundary box regression fine-tunes the position and size of the candidate box by predicting four adjustment parameters of the candidate box (i.e., coordinate, width and height change amount);

[0154] Non-maximum suppression (NMS): after the RPN generates multiple candidate boxes, non-maximum suppression (NMS) is used to filter these candidate boxes; the goal of NMS is to remove the boxes with higher overlap and lower scores, so as to retain the optimal numbered area; the filtering is performed on the candidate boxes optimized by the bounding box regression, not the original candidate boxes, and the calculation formula of non-maximum suppression NMS is:

[0155]

[0156] In the formula, IOU represents the intersection over union, which represents the overlap between two candidate boxes; if the overlap between two candidate boxes exceeds a preset threshold (set to 0.5), one candidate box is selected according to the score, and the other overlapping boxes are removed;

[0157] S22: identifying and obtaining the industrial part number characters in the optimal positioning area candidate box through a character recognition module based on a convolutional neural network (CNN),

[0158] After determining the numbered areas, the next step is to perform character recognition on these areas. To this end, the present embodiment uses a convolutional neural network (CNN) and a CTC (Connectionist Temporal Classification) loss function for character recognition. The CNN will be responsible for extracting local features from the numbered areas, while the CTC loss function will help the model train without the need for character alignment labels, thereby enabling the processing of irregularly arranged numbered characters. In traditional character recognition tasks, characters are usually arranged in a fixed order, and each character needs to be aligned with its position label. However, in practical applications, the arrangement of numbered characters may be irregular, and even some characters may be obscured or missing. To address this situation, the "character alignment label" means that during the standard training process, the label of a character is explicitly aligned with its position in the image (for example, the first character "1" in the image corresponds to the label "1"). The advantage of the CTC loss function is that it can train the network without explicit alignment labels, i.e., without manually labeling the specific positions of characters. Specifically, the CTC loss function includes:

[0159] S221: extracting low-latitude local character features in the optimal positioning area candidate box through a convolutional neural network (CNN), and converting the low-latitude local character features into high-latitude local character features;

[0160] S222: inputting the high-latitude local character features into a pre-set recurrent neural network (RNN) to obtain the probability distribution of the character sequence, and then outputting the predicted industrial part number characters;

[0161] S3: model training on the constructed industrial part number extraction and recognition model according to the preprocessed image data, obtaining an optimal industrial part number extraction and recognition model, so as to realize industrial part number extraction and recognition according to the optimal industrial part number extraction and recognition model;

[0162] In specific embodiments, the method of S3, model training on the constructed industrial part number extraction and recognition model according to the preprocessed image data, obtaining an optimal industrial part number extraction and recognition model, comprises the following steps:

[0163] S31: dividing the preprocessed image data into a training set and a test set according to a preset ratio;

[0164] S32: model training on the constructed industrial part number extraction and recognition model according to the training set, obtaining a trained industrial part number extraction and recognition model:

[0165] S33: taking the CTC loss function as a model loss function, and model evaluation on the trained industrial part number extraction and recognition model according to the test set, to determine whether the output of the trained industrial part number extraction and recognition model converges;

[0166] If yes, the trained industrial part number extraction and recognition model at this time is the optimal industrial part number extraction and recognition model;

[0167] Otherwise, the hyperparameters of the trained industrial part number extraction and recognition model are optimized based on the backpropagation algorithm, and step S31 is repeatedly executed.

[0168] The model training process, the training process of the part number position and character positioning recognition module, comprises:

[0169] S100: input and feature extraction: first, the network receives an image from the number area and extracts features using CNN, CNN processes the image through multiple convolutional layers and pooling layers, gradually extracts local features of the image, and converts them into high-dimensional feature representations;

[0170] S200: CTC loss function: then, these features are passed to a recurrent neural network (RNN) or similar sequence model, which outputs a probability distribution of a character. Since the arrangement of number characters may be irregular, the CTC loss function calculates the loss according to the probability distribution output by the model and the actual label (which does not require the label to be strictly aligned with the image). CTC automatically learns the correct position of the characters by decoding possible character sequences.

[0171] S300: Backpropagation and optimization: During the training process, the CTC loss function updates the model parameters through backpropagation, enabling the network to better recognize irregularly arranged characters. Through multiple iterations of training, the model learns to extract the correct character sequence from the numbered region, even if these characters do not have strict alignment labels. Ultimately, through this training process, the network is able to accurately recognize and output the characters in the numbered region, even if their arrangement order and position are irregular;

[0172] Specifically, the CTC loss function is suitable for sequence-to-sequence tasks, especially in cases where the order and length of characters are not fixed. In number recognition, the CTC loss function optimizes by calculating the difference between the probability distribution of the character sequence output by the model and the true label. In this embodiment, image annotations including numbered regions are included, where each image sample is annotated with a true label, which is usually a character sequence corresponding to the numbered region manually extracted from the image. The preprocessed images will be labeled and composed into a dataset containing images and corresponding character labels as input for subsequent training, and the formula for the CTC loss is:

[0173] L CTC = -logP(y|x)

[0174] where x represents the input image, y represents the corresponding true label, and P(y|x) represents the probability of the character sequence predicted by the model. As shown in Figures 5 to 7 After extracting the features of the numbered region through CNN, these features are extracted from the character region within the determined candidate box. The character recognition network passes these features to the fully connected layer and classifies them through the soft max layer to identify the characters in the numbered region. The output of the network will be a probability distribution containing character categories. By maximizing the probability value, the model will identify the specific characters of the number. During the training process, the model will optimize the network's hyperparameters (such as the number of filter layers in the convolution layer, learning rate, etc.) through the backpropagation algorithm. Each time the network is forward propagated, it will calculate the predicted character sequence probability distribution based on the input image and compare it with the labeled true label. The difference between the model output and the true label is calculated through the CTC loss function, and then the error is propagated to each layer of the network through backpropagation to adjust the network's weights and biases, enabling the network to better fit the training data.

[0175] The post-processing process for character number recognition is also included in this embodiment:

[0176] In the character recognition process, due to factors such as light changes, background interference, etc., some misrecognition may occur. In order to further improve the recognition accuracy, the embodiment designs a post-processing step for correction. Specifically, the common number "1" and letter "I" may be misrecognized. The embodiment corrects them through context analysis and character replacement. The specific operation is as follows:

[0177] S001: Context analysis: The system will determine whether the character is misrecognized according to the relationship between the characters in the character sequence (for example, number sequence, letter sequence, etc.). Assuming that the recognized character is "1a Z", and the actual character should be "Ia Z", through the analysis of the context of the character, the system will automatically replace "1" with "I" to correct the misrecognition.

[0178] In addition, the embodiment also designs a verification based on the predefined numbering format. The recognition result will first be matched with the predefined numbering format to ensure that the output conforms to the expected structure. If the recognition result does not conform to the expected format, the system will automatically trigger format verification and re-correction. The specific operation steps are as follows:

[0179] S002: Number format verification: The system verifies according to the fixed format of the number (for example: number-letter-number-letter). If the output character sequence does not conform to this format (such as "123-ABC" is misrecognized as "123-XYZ"), the system will rearrange the characters according to the format rules; for example, if the character order is incorrect, the system will change the character "XYZ" to the correct character order "ABC".

[0180] S003: Re-correction: If the numbering format still does not meet the requirements after verification, the system will correct it through a rule-based replacement algorithm (for example, the replacement rules of common numbers and letters). For character misrecognition, such as misrecognizing "O" as "0", the system will replace the characters to ensure that the final output conforms to the standard numbering format. Through these steps, the system can accurately correct the misrecognition caused by factors such as light changes and background interference, ensuring that the final recognized number meets the actual requirements and conforms to the predefined numbering format.

[0181] The embodiment also includes the process of output and result evaluation:

[0182] The goal of this embodiment is to output the final result of part number recognition and evaluate the model. The optimal industrial part number extraction and recognition model has been able to extract the number area from the image and recognize the number content. Next, the embodiment will output these recognition results and verify the performance of the model by comparing it with other existing models to ensure its feasibility in practical industrial environments.

[0183] A. Model output result:

[0184] The output of the optimal industrial part number extraction and recognition model includes two main parts:

[0185] (1) Number region position: The precise location of the number region in the image is determined by the bounding box regression after the candidate boxes generated by the region proposal network (RPN). The position of each number region can be represented by the coordinates of the top-left corner and the bottom-right corner, or by the center coordinates and the width and height of the rectangular box.

[0186] (2) Number content recognition: The specific characters within the number region are recognized by the character recognition network (CNN and CTC loss function). The recognized character content is output as a string, usually including numbers, letters, or their combinations.

[0187] For example, the input image may contain multiple part numbers, and after processing by the model, the output is as follows:

[0188] Number region 1: position (50, 80, 150, 120), number content: "A12345"

[0189] Number region 2: position (200, 300, 260, 340), number content: "B67890"

[0190] These information will be used for industrial application scenarios such as part recording, classification, tracking, etc. To further verify the output results, the model will mark the recognized number region with a rectangular box on the original image and mark the number content next to the corresponding region. In this way, users can intuitively see the accuracy of the model's recognition.

[0191] B. Output result evaluation:

[0192] To comprehensively evaluate the performance of the model, this embodiment uses multiple standard evaluation indicators, mainly including: (1) Accuracy: represents the proportion of the number regions correctly recognized by the model to the total number regions. High accuracy indicates that the model can effectively distinguish part numbers from the background.

[0193] (2) Recall: represents the proportion of the number regions recognized by the model to the actual number regions. High recall indicates that the model can recognize more true number regions.

[0194] (3) F1 score: considers the balance between accuracy and recall, and the F1 score is the harmonic mean of the two, which can measure the performance of the model in accuracy and comprehensiveness.

[0195] (4) Character-level accuracy: this is an indicator to evaluate the accuracy of the model's character recognition, representing the correct rate of the model in recognizing each character, especially suitable for character recognition tasks.

[0196] To verify the effectiveness of the model, the method described in this embodiment is compared with several existing part number recognition methods, focusing on comparing the following models:

[0197] (1) Traditional Convolutional Neural Network (CNN) model: The traditional CNN model uses standard convolution operations to extract features from images, which is suitable for processing simple features, but performs poorly in the face of complex backgrounds and raised numbers.

[0198] (2) Object detection model based on Faster R-CNN: Faster R-CNN combines Region Proposal Network (RPN) and Convolutional Neural Network (CNN) for object localization and classification, which performs well in object detection tasks, but lacks detailed capture of numbered areas.

[0199] (3) Feature extraction model based on ResNet: ResNet is a deep convolutional neural network model that is good at extracting deep features from complex images and has strong feature expression ability. However, relying solely on ResNet may not be able to fully capture the raised features of the part number in a complex background.

[0200] This embodiment compares these models on a standard industrial part image dataset to evaluate their performance in number area localization and character recognition. The dataset includes various lighting conditions, background noise, and different sizes of part numbers. According to the collected image data, the evaluation results of each model are as shown in Table 1.

[0201] Table 1. Comparison of evaluation results of models

[0202] Model Accuracy Recall F1 Character-level accuracy The method 97.6％ 96.4％ 97.0％ 98.2％ Faster R-CNN 94.5％ 91.8％ 93.5％ 92.4％ ResNet 91.2％ 87.7％ 90.4％ 91.2％ Traditional CNN 85.3％ 83.1％ 84.2％ 85.7％

[0203] From the results, it can be seen that the method described in this embodiment is superior to other existing models in terms of accuracy, recall rate, F1 score, and character-level accuracy, indicating that the method described in this embodiment has significant advantages in recognizing complex backgrounds, lighting changes, and raised number features.

[0204] To further verify the actual application effect of the model, this embodiment simulates a real industrial scene for testing, in which the image contains various types of interference, including:

[0205] chaotic background, unclear numbers due to surface texture or lighting changes, and partially obscured or deformed part numbers;

[0206] The method described in this embodiment successfully locates the part number area in these complex scenarios and identifies the content with an accuracy of 97.6% and a recall rate of 96.4%, far superior to traditional models.

[0207] Through comparative experiments and simulation of real scene test results, it can be concluded that the method described in this embodiment has significant advantages in part number identification tasks. The feature extraction method combining Graph Convolutional Network (GCN) and Transformer module can effectively overcome challenges such as complex background, light changes and convex number features, providing a high-accuracy part number identification solution. In the industrial field, this method can greatly improve the efficiency of automatic identification and classification of parts, and has wide application prospects.

[0208] The method described in this embodiment has the following beneficial effects:

[0209] (1) Introduction of adaptive graph convolutional network (AGCN): In order to deal with the convex features of part number area and complex background interference, the adaptive graph convolutional network introduced can dynamically adjust the convolution kernel weight based on the local features of the image, efficiently extract the geometric shape and texture features of the part number, and improve the adaptability to complex background and uneven light conditions.

[0210] (2) Global information modeling of Transformer module: Combined with the self-attention mechanism of the Transformer model, the invention can effectively process the global information in the image and capture the complex relationship between the number area and the background. The Transformer module helps to improve the model's ability to distinguish between the number area and the background, strengthens the semantic expression of the number features, and effectively improves the robustness and accuracy of the model.

[0211] (3) Combination of adaptive graph convolution and Transformer: This invention combines adaptive graph convolutional network and Transformer module, fully utilizing the advantages of both in local feature extraction and global information modeling. This combination enables the model to accurately extract and identify part number areas in complex industrial environments, especially in cases with strong background interference and large light condition changes.

[0212] (4) Multi-level feature extraction and fusion mechanism: the application designs a multi-level feature extraction and fusion mechanism, respectively processes the features of the numbered regions of different scales, extracts local features through graph convolution, and combines the global modeling ability of the Transformer, so that the model can effectively cope with the diversified forms of numbering and improve the recognition accuracy in variable environments. The method described in the embodiment can effectively improve the accuracy and adaptability of part number recognition, and is particularly suitable for complex background, uneven illumination and numbered industrial scenes, and has a wide application prospect.

[0213] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. An industrial part number extraction and recognition method based on adaptive graph convolution and Transformer, characterized in that, Specifically comprising the following steps: S1: Obtain RGB part image data containing industrial part number; And sequentially perform grayscale processing, normalization processing and image enhancement processing on the RGB part image data to obtain preprocessed image data; S2: Construct an industrial part number extraction and recognition model; And the industrial part number extraction and recognition model includes a feature extraction unit based on adaptive graph convolution, a global information modeling unit based on a Transformer module, and a part number position and character positioning and recognition module; The feature extraction unit based on adaptive graph convolution is used to extract local part number region feature vectors in the preprocessed image data; The global information modeling unit based on the Transformer module is used to perform global feature modeling according to the local part number region features, obtain global part number region feature vectors, and obtain enhanced region feature vectors by splicing the global part number region feature vectors and the local part number region feature vectors; The part number position and character positioning and recognition module is used to realize positioning of the industrial part number region and recognition of the industrial part number characters according to the enhanced region feature vectors; S3: Model training is performed on the constructed industrial part number extraction and recognition model according to the preprocessed image data to obtain an optimal industrial part number extraction and recognition model, so as to realize industrial part number extraction and recognition according to the optimal industrial part number extraction and recognition model.

2. The industrial part number extraction and recognition method based on adaptive graph convolution and Transformer according to claim 1, characterized in that, The S1 specifically comprises the following steps: S11: Obtain RGB part image data containing industrial part number, and perform grayscale processing to obtain a grayscale image; S12: Perform normalization processing on the grayscale image to obtain a normalized image; S13: Perform image enhancement processing on the normalized image to obtain preprocessed image data; and the image enhancement processing includes: performing denoising processing on the normalized image by a median filter; S14: Based on the Canny edge detection algorithm, edge feature extraction is performed on the denoised normalized image to obtain an edge feature map; and the edge feature map is converted into a binary image by the Otsu algorithm; S15: The cv2.equalizeHist function in the preset OpenCV is called to perform histogram equalization on the binary image to obtain a histogram equalization image; and morphological operation is performed on the histogram equalization image to obtain a morphological image; S16: The cv2.findContours function in the preset OpenCV is called for contour detection to extract the number region of the industrial part number in the morphological image; S17: The cv2.drawContours function in the preset OpenCV is called to draw the extracted number region to the RGB part image to obtain a number region frame and mark the industrial part number characters, and save the image as a PNG format to obtain the preprocessed image data.

3. The industrial part number extraction and recognition method based on adaptive graph convolution and Transformer according to claim 2, characterized in that, The feature extraction unit based on adaptive graph convolution in the S2 comprises a graph structure acquisition module, an adaptive graph convolution module, a local feature enhancement module, a multi-scale feature extraction module and a local feature fusion module; The graph structure acquisition module is configured to define the pixel points in the numbered area as nodes and the pixel value similarity between the pixel points in the numbered area as edges, so as to convert the numbered area in the preprocessed image data into a node graph structure and acquire a node feature vector according to the node graph structure. and the expression of the node feature vector is X i = I(x i ,y i ) where: I(x i ,y i ) represents the pixel value at position (x j ,y i ) in the pre-processed image data; X i represents the feature vector of node i; An adaptive graph convolution module is configured to adjust a convolution kernel parameter of an adaptive graph convolution network according to a node feature vector; an adjustment formula of the convolution kernel parameter is W i =f(X i ,X j ) where: W i represents the adaptive convolution kernel of node i; f(X i , X j ) represents the feature similarity function between node i and its neighboring node j; The graph convolution operation is performed on the node feature vector by the adjusted adaptive graph convolution network to acquire an optimized node graph structure, and the expression of the graph convolution operation is In the formula: represents the output feature vector of the i-th node in the l+1 layer; N (i) denotes the neighborhood nodes of node i; W ij denotes the parameter weight between node i and neighborhood node j; denotes the output feature vector of the l-th layer node i; b i bias term for node i; The local feature enhancement module is configured to adjust the attention weight between the nodes in the optimized node graph structure based on a self-attention mechanism to acquire a local numbered area feature map. The expression of the attention weight between the nodes is Where: α ij represents the attention weight between node i and node j; X i ,X j Represents the feature vectors of nodes i and j in the optimized node graph structure; The multi-scale feature extraction module includes a first scale convolution layer, a second scale convolution layer and a third scale convolution layer arranged in parallel. The first scale convolution layer is configured to perform convolution operation on the local numbered area feature map to acquire a first feature map. The local feature fusion module is configured to perform splicing operation on the first feature map, the second feature map and the third feature map to acquire a local part numbered area feature.

4. The industrial part number extraction and recognition method based on adaptive graph convolution and Transformer according to claim 3, characterized in that, The S2 global information modeling unit based on the Transformer module includes a Transformer encoder module and a global feature fusion module. The Transformer encoder module includes a plurality of stacked Transformer structures. The Transformer structure is configured to acquire a global attention feature vector of the part numbered area based on a self-attention mechanism according to the local part numbered area feature. The global attention feature vector is acquired by the formula Q p = W Q h p , K q = W K h q , V q = W V h q wherein: h' p global attention feature vector representing the part number region; alpha pq denotes the attention weight between node p and node q; N(p) denotes the set of neighborhood nodes of node p; W Q denotes the set of weights; W K denotes the set of weights; W V denotes the set of learnable weight matrices; Q p denotes the query vector of node p; K q denotes the key vector of node q; V q denotes the value vector of node q; h p denotes the feature representation of node p; h q denotes the feature representation of node q; T denotes the transpose; The global feature fusion module is configured to perform feature fusion on the global attention feature vectors output by the Transformer structures to acquire a global part numbered area feature vector. The enhanced region feature vector is acquired by splicing the local part numbered area feature vector and the local part numbered area feature vector.

5. The industrial part number extraction and recognition method based on adaptive graph convolution and Transformer according to claim 4, characterized in that, The S2 part numbered position and character positioning and recognition module includes a region positioning module and a character recognition module based on a convolutional neural network CNN. S21: The region positioning module is used to position the industrial part numbered area according to the enhanced region feature vector, specifically including: S211: The region proposal network is used to acquire a set of positioning region candidate boxes according to the enhanced region feature vector; S212: The optimal positioning region candidate box in the set of positioning region candidate boxes is screened and acquired based on a boundary box regression and a non-maximum suppression NMS; S22: The character recognition module based on the convolutional neural network CNN is used to recognize and acquire the industrial part numbered character in the optimal positioning region candidate box, specifically including: S221: The convolutional neural network CNN is used to extract low-latitude local character features in the optimal positioning region candidate box and convert the low-latitude local character features into high-latitude local character features. S222: input the high latitude local character feature into the preset recurrent neural network (RNN) to obtain the probability distribution of the character sequence, and then output the predicted industrial part number character.

6. The industrial part number extraction and recognition method based on adaptive graph convolution and Transformer according to claim 5, characterized in that, The method for training the constructed industrial part number extraction and recognition model according to the preprocessed image data in S3, obtaining the optimal industrial part number extraction and recognition model, comprises the following steps: S31: divide the preprocessed image data into a training set and a test set according to a preset proportion; S32: train the constructed industrial part number extraction and recognition model according to the training set to obtain a trained industrial part number extraction and recognition model: S33: use the CTC loss function as the model loss function, and evaluate the trained industrial part number extraction and recognition model according to the test set to determine whether the output of the trained industrial part number extraction and recognition model converges; If yes, the trained industrial part number extraction and recognition model at this time is the optimal industrial part number extraction and recognition model; Otherwise, optimize the hyperparameters of the trained industrial part number extraction and recognition model based on the back propagation algorithm, and repeat step S31.

Images