Recognition method and device for steel seal characters and storage medium

Abstract

The invention discloses a steel seal character recognition method and device and a storage medium. The method comprises the steps that a steel seal character image of a to-be-recognized steel seal is acquired; extracting a first feature map of the steel seal character image based on a preset text detection algorithm; determining the character direction of the steel seal character in the first feature map; based on a preset text recognition algorithm, decomposing the first feature map into a plurality of character components according to the character direction, and adding a feature mark to each character component; performing multi-scale feature extraction on the plurality of character components carrying the feature marks to extract local stroke features and global inter-character dependency features of the first feature map; and determining character information in the to-be-recognized steel seal based on the local stroke features and the global inter-character dependency features through a preset text recognition algorithm. According to the scheme, the labor cost can be reduced, and the accuracy and efficiency of steel seal character recognition are improved.

Description

Technical Field

[0001] The present application relates to the technical field of text recognition, and in particular to a method, device and storage medium for recognizing steel-printed characters. Background Art

[0002] The content of the steel stamp characters carries important information about the product. With the development of informatization in the manufacturing industry, automatic recognition of steel stamp characters has become an inevitable requirement for product informatization management. Most of the characters we usually see have the characteristics of clear texture, obvious outline, and high contrast. Character recognition can be achieved using traditional template matching methods or simple deep learning methods. However, in industrial sites, the environment in which steel stamp characters are located is complex and there are many types and numbers. Therefore, steel stamp characters will have problems such as blurred characters and unclear character edges. If the characters are recorded manually, it will lead to low efficiency, high labor costs, and difficulty in connecting with information systems such as QMS, which cannot meet the requirements of intelligent and automated industrial production. Summary of the invention

[0003] The purpose of the embodiments of the present application is to provide a method, device and storage medium for recognizing steel-printed characters, so as to solve the problems of low efficiency, high cost and easy error in human transcription and recognition of characters in the prior art.

[0004] In order to achieve the above-mentioned purpose, the first aspect of the present application provides a method for recognizing steel-stamped characters, the recognition method comprising:

[0005] Obtaining a steel stamp character image of a steel stamp to be identified;

[0006] Extracting a first feature map of the stamped character image based on a preset text detection algorithm;

[0007] Determine the character direction of the stamped character in the first feature image;

[0008] Based on a preset text recognition algorithm, decomposing the first feature map into a plurality of character components according to character directions, and adding a feature mark to each character component;

[0009] Performing multi-scale feature extraction on the multiple character components carrying the feature markers to extract local stroke features and global inter-character dependency features of the first feature map;

[0010] The character information in the steel seal to be identified is determined based on local stroke features and global character dependency features through a preset text recognition algorithm.

[0011] In an embodiment of the present application, performing multi-scale feature extraction on multiple character components carrying feature markers to extract local stroke features and global inter-character dependency features of a first feature map includes: inputting multiple character components carrying feature markers into a multi-stage convolution module for multi-scale feature extraction, so as to output a second feature map corresponding to the first feature map through the multi-stage convolution module;

[0012] The second feature map is input into the global mixing module to extract local stroke features and global inter-character dependency features through the global mixing module.

[0013] In an embodiment of the present application, the multi-stage convolution module is a deep separable convolutional network of the PP-LCNet algorithm.

[0014] In an embodiment of the present application, extracting a first feature map of a stamped character image based on a preset text detection algorithm includes: extracting image features of multiple scales of the stamped character image based on a residual network of the preset text detection algorithm; fusing image features of multiple scales through a feature pyramid network of the preset text detection algorithm to output corresponding target image features through the feature pyramid network, wherein the feature pyramid network is a convolutional layer of a residual structure and a channel attention structure; binarizing the target image features through a differentiable binarization module of the preset text detection algorithm to output the first feature map of the stamped character image.

[0015] In an embodiment of the present application, the method also includes: lightweight processing of the preset text detection algorithm through a model cropping module to obtain an updated preset text detection algorithm; inputting the preset text detection algorithm and the updated preset text detection algorithm into a knowledge distillation module to optimize the updated preset text detection algorithm through the knowledge distillation module to obtain an optimized preset text detection algorithm.

[0016] In an embodiment of the present application, the method also includes: before extracting the first feature map of the stamped character image based on a preset text detection algorithm, dividing the stamped character image into multiple sub-image areas; for each sub-image area, performing histogram equalization processing on the sub-image area to obtain a processed sub-image area; and obtaining a processed stamped character image based on all the processed sub-image areas.

[0017] In an embodiment of the present application, the method also includes: uploading the character information to the quality management system so that the quality management system calls the production schedule of the production execution system; matching the character information with the production schedule and historical character information respectively to obtain corresponding matching results; and recording the matching results in the system log of the quality management system.

[0018] In an embodiment of the present application, character information is matched with the production schedule and historical character information respectively to obtain corresponding matching results including: when the character information is the same as the historical character information, determining that the matching result is a duplicate detection; when there is no character information in the production schedule, determining that the matching result is an error; when there is character information in the production schedule, determining that the matching result is correct.

[0019] A second aspect of the present application provides a device for recognizing steel-stamped characters, comprising:

[0020] a memory configured to store instructions;

[0021] The processor is configured to call the instructions from the memory and implement the above-mentioned method for recognizing steel-printed characters when executing the instructions.

[0022] A third aspect of the present application is a machine-readable storage medium, on which instructions are stored, and the instructions are used to enable a machine to execute the above-mentioned method for recognizing steel-printed characters.

[0023] Through the above technical solution, by obtaining the steel stamp character image of the steel stamp to be identified; extracting the first feature map of the steel stamp character image based on the preset text detection algorithm; determining the character direction of the steel stamp character in the first feature map; based on the preset text recognition algorithm, decomposing the first feature map into multiple character components according to the character direction, and adding feature tags to each character component; performing multi-scale feature extraction on the multiple character components carrying the feature tags to extract the local stroke features and global inter-character dependency features of the first feature map; determining the character information in the steel stamp to be identified based on the local stroke features and global inter-character dependency features through the preset text recognition algorithm. The above solution can reduce labor costs and improve the accuracy and efficiency of steel stamp character recognition.

[0024] Other features and advantages of the embodiments of the present application will be described in detail in the subsequent specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the following specific implementations, they are used to explain the embodiments of the present application, but do not constitute a limitation on the embodiments of the present application. In the accompanying drawings:

[0026] Figure 1 A schematic diagram of a process flow of a method for recognizing steel-printed characters according to an embodiment of the present application is shown schematically;

[0027] Figure 2 A schematic diagram schematically shows a network structure of an optimized text detection algorithm according to an embodiment of the present application;

[0028] Figure 3 A schematic diagram schematically shows the network structure of RSE-FPN and RSEConv according to an embodiment of the present application;

[0029] Figure 4 A schematic diagram schematically shows a network structure of DML distillation according to an embodiment of the present application;

[0030] Figure 5 A schematic diagram of a convolutional network using a residual structure according to an embodiment of the present application is schematically shown;

[0031] Figure 6 A schematic diagram of a three-stage height progressively decreasing network structure according to an embodiment of the present application is schematically shown;

[0032] Figure 7 A schematic diagram schematically shows a network structure of a preset text recognition algorithm according to an embodiment of the present application;

[0033] Figure 8 A schematic diagram schematically shows a preset text recognition algorithm training process according to an embodiment of the present application;

[0034] Fig. 9 A schematic diagram of a human-machine interface for steel stamp character recognition according to an embodiment of the present application is schematically shown;

[0035] Fig.10 The schematic diagram shows a structure of a computer device according to an embodiment of the present application. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the specific implementation methods described herein are only used to illustrate and explain the embodiments of the present application, and are not used to limit the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0037] It should be noted that if the embodiments of the present application involve directional indications (such as up, down, left, right, front, back, etc.), the directional indications are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0038] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present application, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or suggesting their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0039] Figure 1 The following is a schematic diagram of a flow chart of a method for recognizing steel-printed characters according to an embodiment of the present application. Figure 1 As shown, an embodiment of the present application provides a method for recognizing steel-printed characters, which may include the following steps.

[0040] S102, obtaining a steel stamp character image of the steel stamp to be identified.

[0041] It is understandable that in the manufacturing process of large-scale construction machinery, it is necessary to engrave steel stamp characters on the surface of the products and their metal parts, and realize the quantity statistics, vehicle assembly, quality supervision and tracking management of the products through the one-to-one code method. The steel stamp character image refers to an image containing steel stamp characters, that is, the production number of the product. However, for computers, it is impossible to directly obtain the steel stamp characters in the image, and a series of image processing is required to identify the steel stamp characters in the image.

[0042] S104, extracting a first feature map of the stamped character image based on a preset text detection algorithm.

[0043] It can be understood that the text detection algorithm refers to a method for identifying the position of text in an image or video. The preset text detection algorithm refers to a text detection algorithm designed by a technician that can identify the position of the stamped characters in the stamped character image. After the stamped character image is detected by the preset text detection algorithm, the corresponding first feature map can be obtained. The first feature map refers to the area image where the stamped characters are located in the stamped character image.

[0044] In an embodiment of the present application, extracting a first feature map of a stamped character image based on a preset text detection algorithm includes: extracting image features of multiple scales of the stamped character image based on a residual network of the preset text detection algorithm; fusing image features of multiple scales through a feature pyramid network of the preset text detection algorithm to output corresponding target image features through the feature pyramid network, wherein the feature pyramid network is a convolutional layer of a residual structure and a channel attention structure; binarizing the target image features through a differentiable binarization module of the preset text detection algorithm to output the first feature map of the stamped character image.

[0045] It can be understood that the backbone network of the preset text detection algorithm is a residual network, which is a deep convolutional neural network. The use of residual connections and other mechanisms can effectively solve the gradient vanishing and representation bottleneck problems in deep neural networks, thereby deeply learning and extracting the features of the stamped character image. Furthermore, the feature fusion layer of the preset text detection algorithm includes a feature pyramid network. Through the multi-scale feature fusion method, the features of different levels are fused, so that the feature pyramid network can simultaneously capture text information of different scales, that is, output the corresponding target image features. Furthermore, the differentiable binarization module integrates the binarization process into the network training, so that the network can adaptively select a more appropriate binarization threshold, thereby outputting a probability map and a threshold map, and then comparing the probability map and the threshold map pixel by pixel, and outputting the first feature map of the stamped character image. Among them, the probability map represents the probability that each pixel belongs to the text area, and the threshold map represents the binarization threshold of each pixel.

[0046] In one embodiment, the text detection algorithm may be a DBNet algorithm (Differentiable Binarization Module, DBModule), and the preset text detection algorithm is an improved text detection algorithm for the DBNet algorithm. Specifically, the network structure is improved mainly in the feature fusion layer. First, a PAN with a large receptive field is used to change the convolution kernel in the path augmentation of the PAN structure from 3*3 to 7*7. By increasing the convolution kernel, the receptive field covered by each position of the feature map is improved, making it easier to detect large font text and text with extreme aspect ratios. The overall parameters are adjusted by fusing the BN structure. The conversion method is as follows: Figure 2As shown. Furthermore, the residual structure and channel attention structure RSE-FPN are introduced in the feature fusion layer, and the convolution layer in FPN is replaced with the RSEConv layer of the channel attention structure to further improve the representation ability of the feature map. Considering that the number of FPN channels in the detection model is very small, only 96, if SEblock is directly used to replace the convolution in FPN, the features of some channels will be suppressed and the accuracy will decrease. The introduction of residual structure in RSEConv will alleviate the above problems and improve the text detection effect. The network structure of RSE-FPN and RSEConv is as follows Figure 3 After optimizing the network structure, the preset text detection algorithm uses the public data 30w+ text detection data set for training. Under the premise of ensuring the number of model parameters and inference speed, the main evaluation index Hmean of text detection is improved by 5%.

[0047] In an embodiment of the present application, the method also includes: lightweight processing of the preset text detection algorithm through a model cropping module to obtain an updated preset text detection algorithm; inputting the preset text detection algorithm and the updated preset text detection algorithm into a knowledge distillation module to optimize the updated preset text detection algorithm through the knowledge distillation module to obtain an optimized preset text detection algorithm.

[0048] Specifically, after the network structure optimization of the preset text detection algorithm is completed, DML distillation is used to improve the model performance, and mutual distillation is performed in the same or different non-pretrained networks to improve the performance of the deep neural network. Specifically, the preset text detection algorithm can be lightweighted by the model pruning module to obtain an updated preset text detection algorithm. The preset text detection algorithm and the updated preset text detection algorithm are input into the knowledge distillation module. In this way, a compact network with better network performance than the static teacher extracted from the strong network can be obtained. Compared with the teacher-guided distillation model, the mutual learning strategy has the following advantages: 1. As the number of student networks increases, its efficiency is also improved; 2. It can be applied to a variety of networks, including networks of different sizes; 3. Even if a very large network adopts the mutual learning strategy, its performance can be improved. The teacher model mutual learning strategy can effectively improve the accuracy of the text detection model by learning from each other by two models with the same structure.

[0049] refer to Figure 4, the preset text detection algorithm Net1 and the updated preset text detection algorithm Net2, the same input, passes through the two networks Net1 and Net2 respectively to obtain two logits recorded as z1 and z2. After softmax, the predicted soft distributions p1 and p2 are obtained. The KL divergence is sent to calculate the similarity of the two distributions as the mimetic loss, and compared with the label to calculate the label loss. Specifically, the mimetic loss is calculated according to the following formula (1):

[0050]

[0051] Among them, D KL (p2||p1) refers to the divergence, N refers to the total number of samples, M refers to the total number of categories, i and m are count values, respectively, x i Refers to the data of the i-th sample.

[0052] The total loss of network Net1 is calculated according to the following formula (2):

[0053]

[0054] Among them, D KL (p2||p1) refers to the divergence, Refers to the overall loss of Net1, Refers to the classification loss of Net1.

[0055] The total loss of network Net2 is calculated according to the following formula (3):

[0056]

[0057] in, refers to the overall loss of Net2, D KL (p2||p1) refers to the divergence, Refers to the classification loss of Net2.

[0058] Assuming there are K student networks, their loss objective function is calculated according to the following formula (4):

[0059]

[0060] in, refers to the overall loss of network K, refers to the network K classification loss, K is the total number of student networks, l is the student network number, D KL (p2||p1) refers to the divergence.

[0061] In an embodiment of the present application, the method also includes: before extracting the first feature map of the stamped character image based on a preset text detection algorithm, dividing the stamped character image into multiple sub-image areas; for each sub-image area, performing histogram equalization processing on the sub-image area to obtain a processed sub-image area; and obtaining a processed stamped character image based on all the processed sub-image areas.

[0062] S106, determining the character direction of the stamped character in the first feature image.

[0063] It is understandable that due to the production environment, it is not convenient for workers to take photos of the vehicle code and structural part code in the forward direction, so some of the steel stamp codes are inverted. In order to solve this problem, it is proposed to intercept the area where the steel stamp characters are located after the text detection is completed, and use ResNet18 to identify the direction of the intercepted image. In order to solve the problem of gradient disappearance or gradient explosion, it is proposed to solve it through data preprocessing and using BN (Batch Normalization) layer in the network. In order to solve the degradation problem in deep networks, a residual structure is proposed to alleviate the degradation problem. Reference Figure 5 , Figure 5 It is a convolutional network using a residual structure.

[0064] S108, based on a preset text recognition algorithm, decompose the first feature map into multiple character components according to character directions, and add a feature mark to each character component.

[0065] It can be understood that the text recognition algorithm refers to a subtask of OCR (Optical Character Recongnition), which is used to recognize the text content in the image. The preset text recognition algorithm refers to a text recognition algorithm designed by technicians that can recognize the content of the stamped characters in the stamped character image. The preset text recognition algorithm can decompose the first feature map into multiple character components according to the character direction. The character component refers to the pixel-level image component that contains the information of the stamped character part in the first feature map, which can be understood as a small 2D patch. By adding a feature marker to each character component, each character component can be accurately located, and other character components adjacent to a character component can be searched through the feature marker.

[0066] S110, performing multi-scale feature extraction on the multiple character components carrying the feature markers to extract local stroke features and global inter-character dependency features of the first feature map.

[0067] Specifically, multi-scale feature extraction can adopt a multi-stage convolution module, and each stage of convolution can have different features of different scales. Through multi-stage convolution, the features of input data are gradually extracted and abstracted, which can improve the recognition accuracy and generalization ability of the model. The multi-scale feature extraction is performed on multiple character components carrying feature tags, so as to extract local stroke features and global inter-character dependency features of the first feature map. Among them, the local stroke features mainly refer to the dependency between an area in the image and its surrounding areas. These local information usually refer to pixel values ​​or features, which are related to pixels or features in a certain area. Local information helps to capture details and textures in the image. For example, in image segmentation, the color and brightness differences between adjacent pixels can help determine the boundaries of objects. In text recognition, local stroke features are very important for extracting stroke-level details, which are crucial for accurate character recognition. Global inter-character dependency features refer to the long-distance dependency between different areas in the image, usually across a large spatial range. This dependency involves a wider range of contextual information, including the relationship between objects, the overall image structure, etc. Global information is very important for more advanced tasks such as image classification, object detection, and semantic segmentation because it provides more semantic information. In text recognition, global inter-character dependency features help understand the relationship and context between characters, which is crucial for correctly recognizing and transcribing text. The extraction and utilization of these two features are of great significance for improving the accuracy and efficiency of text recognition. Local stroke features help capture the details and structure of characters, while global inter-character dependency features provide association and context information between characters. The combination of the two can significantly improve the accuracy and reliability of text recognition.

[0068] In an embodiment of the present application, multi-scale feature extraction is performed on multiple character components carrying feature markers to extract local stroke features and global character dependency features of a first feature map, including: inputting multiple character components carrying feature markers into a multi-stage convolution module for multi-scale feature extraction, so as to output a second feature map corresponding to the first feature map through the multi-stage convolution module; and inputting the second feature map into a global mixing module to extract local stroke features and global character dependency features through the global mixing module.

[0069] It can be understood that the global hybrid module can evaluate the dependencies between all character components. Because text and non-text are the two main elements in an image, the purpose of the global hybrid module is to establish long-distance dependencies between components of different characters. In addition, it is also able to weaken the influence of non-text components while enhancing the importance of text components.

[0070] Specifically, the preset text recognition algorithm can optimize the CRNN algorithm. First, the network structure is improved. The backbone network layer integrates the Transformer network, the lightweight CNN network and the PP-LCNet into a lightweight text recognition network. The model can successfully capture the local patterns within the characters and the long-term dependencies between characters. The former encodes the stroke features that describe the fine-grained features of the characters and is the key source for distinguishing characters. The latter records the language-like knowledge that describes the text from a complementary perspective. SVTR decomposes the image text into small 2D patches called character components, and uses patch-based image tokenization and self-attention to capture the recognition clues between character components. Through a three-level highly step-down hybrid module (multi-stage convolution module), specifically, the local component-level affinity representing the character stroke features and the long-term dependencies between different characters are obtained to form multi-granular feature perception. Finally, recognition can be achieved through simple linear prediction.

[0071] Specifically, refer to Figure 6 , Figure 6 The three-stage highly progressively decreasing network structure is illustrated. The input is an image of size H×W×3. First, the original image is converted into a patch of size H / 4×W / 4 through progressive overlapping patch embedding, and its channel dimension is D0. We call it character components, and each character component is associated with a part of the text characters in the image. Then there is a three-stage highly progressively decreasing process, each stage consists of a series of mixing blocks and merging operations or combining operations to extract multi-scale features. The local and global modules are designed to extract local stroke features and global inter-character dependency features. Using this backbone network, the intra-character stroke features, inter-character context dependency features and multi-scale features can be fused to obtain the final feature expression C, whose dimension is 1×W / 4×D3.

[0072] In the embodiment of the present application, the multi-stage convolution module is a deep separable convolutional network of the PP-LCNet algorithm. Figure 7 , the preset text recognition algorithm can be an optimized algorithm that replaces the first half of the SVTR network with the deep separable convolutional network of the PP-LCNet algorithm, that is, the first three stages of the PP-LCNet algorithm, retaining 4 GlobalMixing Blocks, with an accuracy of 76% and an acceleration of 69%

[0073] S112, determining the character information in the steel seal to be recognized based on local stroke features and global inter-character dependency features through a preset text recognition algorithm.

[0074] Through the above technical solution, by obtaining the steel stamp character image of the steel stamp to be identified; extracting the first feature map of the steel stamp character image based on the preset text detection algorithm; determining the character direction of the steel stamp character in the first feature map; based on the preset text recognition algorithm, decomposing the first feature map into multiple character components according to the character direction, and adding feature tags to each character component; performing multi-scale feature extraction on the multiple character components carrying the feature tags to extract the local stroke features and global inter-character dependency features of the first feature map; determining the character information in the steel stamp to be identified based on the local stroke features and global inter-character dependency features through the preset text recognition algorithm. The above solution can reduce labor costs and improve the accuracy and efficiency of steel stamp character recognition.

[0075] In one embodiment, reference Figure 8 In terms of training strategy, the preset text recognition algorithm uses Attention to guide CTC training, integrates the expression of multiple text features, and improves the text recognition strategy. Using this strategy, the Attention module is completely removed during prediction, and no time is added in the reasoning stage. The accuracy of the recognition model is further improved to 75.8% (+1.82%).

[0076] In an embodiment of the present application, the method also includes: uploading the character information to the quality management system so that the quality management system calls the production schedule of the production execution system; matching the character information with the production schedule and historical character information respectively to obtain corresponding matching results; and recording the matching results in the system log of the quality management system.

[0077] It can be understood that historical character information refers to character information obtained by detecting other steel stamp character images at historical times. The matching result includes the result of matching the production schedule with the character information and the result of matching the character information with the historical character information.

[0078] In an embodiment of the present application, character information is matched with the production schedule and historical character information respectively to obtain corresponding matching results including: when the character information is the same as the historical character information, determining that the matching result is a duplicate detection; when there is no character information in the production schedule, determining that the matching result is an error; when there is character information in the production schedule, determining that the matching result is correct.

[0079] Specifically, after the text recognition is completed, the results need to be uploaded to the QMS system. The QMS system calls the production schedule of the MES system, matches the recognition results with the system production schedule and the historical character information detected at the historical time, and returns a duplicate if it is repeated with the historical character information. If it matches the production schedule result, it returns a correct answer, and other results return an error. If the final result is a manually modified result, it is recorded and returned to the front end. At the same time, the results of each photo shooting need to be recorded in the QMS system log to facilitate later quality inspections.

[0080] In one embodiment, with the rapid development of machine learning and deep learning technologies, industrial workshops are gradually transforming towards intelligence and automation. Among them, after taking a photo with a mobile phone, the deep learning OCR recognition model is used to perform high-precision recognition of the stamped characters of the excavator vehicle code, and then combined with information systems such as QMS and MES, it is an important step to realize the automatic generation of excavator vehicle licenses. It can greatly reduce labor costs, reduce employee workload, and improve overall accuracy and efficiency, which is of great significance to the production of large equipment vehicles. Specifically, the human-machine interface is such as Fig. 9 As shown, click on the photo recognition on the mobile phone interface to take a photo of the steel stamp in the steel stamp constraint frame. After the recognition is completed, check whether the steel stamp recognition result is correct. If it is not correct, analyze whether the picture is unclear. If it is not clear, retake it. When collecting mobile phone images, adjust the frame selection area to cover the entire steel stamp code, click the camera button to take a photo, and after the photo is taken, the background will capture the steel stamp code sub-image in the frame, compress it to 48×320 pixels, and upload it to the QMS system, which will improve the network transmission speed and reduce the preprocessing time before model prediction without affecting the OCR recognition accuracy.

[0081] Furthermore, in order to solve the problem of uneven local illumination caused by the reflection of the steel plate, the preprocessing part firstly adopts adaptive histogram equalization, which is a variant of histogram equalization and comprehensively considers the local differences of different regions in the image. Unlike histogram equalization, adaptive histogram equalization divides the image into multiple small blocks and applies histogram equalization independently in each small block. In this way, adaptive histogram equalization can better preserve the details of the image and avoid over-enhancing noise. Adaptive histogram equalization can automatically adjust the grayscale level of each small block according to the local characteristics of the image to achieve more refined image enhancement.

[0082] Furthermore, due to the low overall output of large equipment, the number of vehicle codes is less than 100 per day, the overall data is small, and due to the existence of coding restrictions, the character distribution gap is large, and the model is prone to overfitting. To solve this problem, on the one hand, we use flipping, rotation, cropping, deformation and scaling of existing data to increase the sample size, and on the other hand, we use text_render to generate a large number of simulated samples. The generation methods mainly include: generating samples based on the coding rules of vehicle codes and structural parts codes and generating samples without rules. This method solves the problem of model overfitting and improves the overall accuracy of the model by 3%.

[0083] Furthermore, if the characters are caused by handwriting itself or stains, you need to click the modification button, manually check and modify and confirm. The recognition results will be displayed in the system matching line and matched with the corresponding fields of the system. Click the result upload to upload the steel stamp code to the quality management system (QMS) system for recording. After taking the photo, after the OCR recognition service receives the picture uploaded by the mobile phone, it first divides the specific text based on the preset text detection algorithm, and then uses ResNet18 to classify and identify whether the text is inverted. If it is an inverted text picture, it needs to be flipped through Opencv. Finally, after processing the picture through image enhancement and other methods, the preset text recognition algorithm is used to identify the final steel stamp code. After the steel stamp code is recognized, it will be sent to the QMS system. QMS calls the MES system production list, matches the recognition results with the system production list and historical character information, and returns a duplicate if it is repeated with the historical character information. If it matches the production list result, it returns correct, and other results return an error. If the final result is a manual modification result, it is recorded and returned to the front end.

[0084] Figure 1 FIG. 1 is a flow chart of a method for recognizing steel-printed characters in one embodiment. It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0085] The present application embodiment provides a device for recognizing steel-stamped characters, which may include:

[0086] a memory configured to store instructions; and

[0087] The processor is configured to call instructions from the memory and implement the above-mentioned method for recognizing steel-printed characters when executing the instructions.

[0088] Specifically, in the embodiment of the present application, the processor may be configured to:

[0089] Obtaining a steel stamp character image of a steel stamp to be identified;

[0090] Extracting a first feature map of the stamped character image based on a preset text detection algorithm;

[0091] Determine the character direction of the stamped character in the first feature image;

[0092] Based on a preset text recognition algorithm, decomposing the first feature map into a plurality of character components according to character directions, and adding a feature mark to each character component;

[0093] Performing multi-scale feature extraction on the multiple character components carrying the feature markers to extract local stroke features and global inter-character dependency features of the first feature map;

[0094] The character information in the steel seal to be identified is determined based on local stroke features and global character dependency features through a preset text recognition algorithm.

[0095] In the embodiment of the present application, the processor may be configured to:

[0096] Multiple character components carrying feature markers are input into a multi-stage convolution module for multi-scale feature extraction, so as to output a second feature map corresponding to the first feature map through the multi-stage convolution module; the second feature map is input into a global mixing module, so as to extract local stroke features and global inter-character dependency features through the global mixing module.

[0097] In an embodiment of the present application, the multi-stage convolution module is a deep separable convolutional network of the PP-LCNet algorithm.

[0098] In the embodiment of the present application, the processor may be configured to:

[0099] A residual network based on a preset text detection algorithm extracts image features of multiple scales of the stamped character image; a feature pyramid network of the preset text detection algorithm is used to fuse image features of multiple scales, so as to output corresponding target image features through the feature pyramid network, wherein the feature pyramid network is a convolutional layer of a residual structure and a channel attention structure; the target image features are binarized through a differentiable binarization module of the preset text detection algorithm, so as to output a first feature map of the stamped character image.

[0100] In the embodiment of the present application, the processor may be configured to:

[0101] The preset text detection algorithm is lightweighted through the model cropping module to obtain an updated preset text detection algorithm; the preset text detection algorithm and the updated preset text detection algorithm are input into the knowledge distillation module to optimize the updated preset text detection algorithm through the knowledge distillation module to obtain an optimized preset text detection algorithm.

[0102] In the embodiment of the present application, the processor may be configured to:

[0103] Before extracting the first feature map of the stamped character image based on a preset text detection algorithm, the stamped character image is divided into a plurality of sub-image areas; for each sub-image area, the sub-image area is subjected to histogram equalization processing to obtain a processed sub-image area; and a processed stamped character image is obtained based on all the processed sub-image areas.

[0104] In the embodiment of the present application, the processor may be configured to:

[0105] Upload the character information to the quality management system so that the quality management system can call the production schedule of the production execution system; match the character information with the production schedule and historical character information respectively to obtain corresponding matching results; and record the matching results in the system log of the quality management system.

[0106] In the embodiment of the present application, the processor may be configured to:

[0107] When the character information is identical to the historical character information, the matching result is determined to be a duplicate detection; when the character information does not exist in the production schedule, the matching result is determined to be an error; when the character information exists in the production schedule, the matching result is determined to be correct.

[0108] An embodiment of the present application also provides a machine-readable storage medium, on which instructions are stored, and the instructions are used to enable a machine to execute the above-mentioned method for recognizing steel-printed characters.

[0109] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Fig.10As shown. The computer device includes a processor A01, a network interface A02, a memory (not shown in the figure) and a database (not shown in the figure) connected via a system bus. Among them, the processor A01 of the computer device is used to provide computing and control capabilities. The memory of the computer device includes an internal memory A03 and a non-volatile storage medium A04. The non-volatile storage medium A04 stores an operating system B01, a computer program B02 and a database (not shown in the figure). The internal memory A03 provides an environment for the operation of the operating system B01 and the computer program B02 in the non-volatile storage medium A04. The database of the computer device is used to store data for a method for recognizing steel-printed characters. The network interface A02 of the computer device is used to communicate with an external terminal through a network connection. When the computer program B02 is executed by the processor A01, a method for recognizing steel-printed characters is implemented.

[0110] Those skilled in the art will understand that Fig.10 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0111] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0112] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0113] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0114] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0115] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0116] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0117] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0118] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0119] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A method for recognizing steel-printed characters, characterized in that: The identification method comprises: Obtaining a steel stamp character image of a steel stamp to be identified; Extracting a first feature map of the stamped character image based on a preset text detection algorithm; Determine the character direction of the stamped character in the first feature image; Based on a preset text recognition algorithm, decomposing the first feature map into a plurality of character components according to the character directions, and adding a feature mark to each character component; Performing multi-scale feature extraction on the multiple character components carrying the feature markers to extract local stroke features and global inter-character dependency features of the first feature map; The character information in the steel seal to be recognized is determined based on the local stroke features and the global inter-character dependency features through the preset text recognition algorithm.

2. The method for recognizing steel-printed characters according to claim 1, characterized in that: The step of performing multi-scale feature extraction on the plurality of character components carrying feature markers to extract local stroke features and global inter-character dependency features of the first feature map includes: Inputting a plurality of character components carrying feature markers into a multi-stage convolution module for multi-scale feature extraction, so as to output a second feature map corresponding to the first feature map through the multi-stage convolution module; The second feature map is input into a global mixing module, so as to extract the local stroke features and the global inter-character dependency features through the global mixing module.

3. The method for recognizing steel-stamped characters according to claim 2, characterized in that: The multi-stage convolution module is a deep separable convolutional network of the PP-LCNet algorithm.

4. The method for recognizing steel-stamped characters according to claim 1, characterized in that: The first feature map of the steel stamp character image extracted based on a preset text detection algorithm comprises: Extracting image features of multiple scales of the stamped character image using a residual network based on a preset text detection algorithm; The image features of the multiple scales are fused through a feature pyramid network of the preset text detection algorithm to output corresponding target image features through the feature pyramid network, wherein the feature pyramid network is a convolution layer of a residual structure and a channel attention structure; The target image features are binarized by the differentiable binarization module of the preset text detection algorithm to output a first feature map of the stamped character image.

5. The method for recognizing steel-printed characters according to claim 4, characterized in that: The method further comprises: The preset text detection algorithm is lightweight processed by a model clipping module to obtain an updated preset text detection algorithm; The preset text detection algorithm and the updated preset text detection algorithm are input into a knowledge distillation module, so as to optimize the updated preset text detection algorithm through the knowledge distillation module to obtain an optimized preset text detection algorithm.

6. The method for recognizing steel-stamped characters according to claim 1, characterized in that: The method further comprises: Before extracting the first feature map of the stamped character image based on a preset text detection algorithm, the stamped character image is divided into a plurality of sub-image areas; For each sub-image region, performing a histogram equalization process on the sub-image region to obtain a processed sub-image region; A processed steel stamp character image is obtained based on all processed sub-image areas.

7. The method for recognizing steel-printed characters according to claim 1, characterized in that: The method further comprises: Uploading the character information to a quality management system so that the quality management system calls a production schedule of a production execution system; Matching the character information with the production schedule list and the historical character information respectively to obtain corresponding matching results; The matching result is recorded in the system log of the quality management system.

8. The method for recognizing steel-stamped characters according to claim 7, characterized in that: The character information is matched with the production schedule list and the historical character information respectively to obtain corresponding matching results, including: In the case where the character information is identical to the historical character information, determining that the matching result is a duplicate detection; In the case where the character information does not exist in the production schedule, determining that the matching result is an error; In the case where the character information exists in the production schedule, it is determined that the matching result is correct.

9. A device for recognizing steel-printed characters, characterized in that: include: a memory configured to store instructions; A processor is configured to call the instructions from the memory and implement the method for recognizing steel-printed characters according to any one of claims 1 to 8 when executing the instructions.

10. A machine-readable storage medium, characterized in that: The machine-readable storage medium stores instructions for causing a machine to execute the method for recognizing steel-printed characters according to any one of claims 1 to 8.

Images