A steel plate code spraying information recognition method and system

Abstract

The application relates to the technical field of steel plate code spraying. A steel plate code spraying information identification method comprises the following steps: respectively marking sample pictures and virtual pictures, and correspondingly obtaining training data and test data; inputting the training data into a steel plate code spraying information pre-training model to obtain a steel plate code spraying information training model; then, the test data is identified twice to obtain text information of the test data; next, text region comparison and text information comparison are carried out to obtain evaluation indexes; the evaluation indexes are used to correct the steel plate code spraying information training model, which is then converted into a steel plate code spraying information inference model; the steel plate code spraying information inference model is used to continuously identify to-be-tested steel plates to obtain to-be-fused code spraying information of continuous multiple frames of to-be-tested steel plates; a steel plate code spraying information fusion algorithm is used to fuse the to-be-fused code spraying information of the continuous multiple frames of to-be-tested steel plates to obtain fused code spraying information of the to-be-tested steel plates, which is compared with stock information. The steel plate code spraying information fusion algorithm improves the stock management efficiency of a steel stockhouse.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of steel plate code spraying, and in particular to a steel plate code spraying information recognition method and system. BACKGROUND

[0002] In recent years, character recognition technology has become a research hotspot in the field of machine vision and artificial intelligence, and has developed rapidly. However, there are still certain limitations in recognizing steel plate code spraying information. For example, in complex environments such as warehouse management and truck loading and delivery, due to factors such as light, steel plate specifications, steel plate stacking positions, and steel plate code spraying positions, the recognition rate of side code spraying information of multiple steel plates with non-fixed positions and disordered arrangement is poor, resulting in a large amount of manual work and increased cost of warehouse management for enterprises.

[0003] Therefore, the development of warehouse steel plate code spraying information recognition is imminent. SUMMARY

[0004] The present application provides a steel plate code spraying information recognition method and system to solve the problem of poor recognition effect of existing steel plates in the warehouse due to complex environment.

[0005] The first aspect of the present application provides a steel plate code spraying information recognition method, comprising:

[0006] Obtain a sample picture and a virtual picture, label the sample picture and the virtual picture respectively, and obtain training data and test data correspondingly, the training data includes first code spraying information on existing steel plates, and second code spraying information on virtual steel plates is constructed according to preset code spraying rules, and the test data includes at least part of the first code spraying information on the existing steel plates;

[0007] Input the training data into a steel plate code spraying information pre-training model to obtain a steel plate code spraying information detection training model and a steel plate code spraying information recognition training model, and the steel plate code spraying information detection training model and the steel plate code spraying information recognition training model constitute a steel plate code spraying information training model;

[0008] Use the steel plate code spraying information detection training model to perform first recognition on the test data to obtain a recognized text area of the test data;

[0009] Use the steel plate code spraying information recognition training model to perform second recognition on the text area of the test data to obtain recognized text information of the test data;

[0010] Compare the recognized text area of the test data with the corresponding text area of the existing steel plate, and compare the recognized text information of the test data with the corresponding text information of the existing steel plate, and obtain evaluation indicators;

[0011] The evaluation index is used to correct the steel plate code spraying information training model to obtain an optimal steel plate code spraying information training model, which is converted into a steel plate code spraying information inference model;

[0012] The steel plate code spraying information inference model is used for continuous identification of a to-be-tested steel plate to obtain to-be-fused code spraying information of the to-be-tested steel plate in multiple continuous frames;

[0013] A steel plate code spraying information fusion algorithm is used to fuse the to-be-fused code spraying information of the to-be-tested steel plate in multiple continuous frames to obtain fused code spraying information of the to-be-tested steel plate;

[0014] The fused code spraying information is compared with inventory information or an out-of-stock preset rule, and information different from the inventory information and information not meeting the out-of-stock preset rule is specially marked and warned, the inventory information refers to steel plate code spraying information in an inventory system, and the out-of-stock preset rule refers to a value range of steel grade and size preset in an out-of-stock link.

[0015] In an implementable manner, the steps of obtaining sample pictures and virtual pictures, respectively labeling the sample pictures and the virtual pictures, and obtaining training data and test data correspondingly, include:

[0016] The sample pictures and the virtual pictures in the training data are labeled to obtain a training data set and a test data set with labels, and the labels represent marking of text regions and text information in the sample pictures and the virtual pictures;

[0017] The first code spraying information on a plurality of existing steel plates is obtained to obtain the test data and part of the training data, wherein the first code spraying information includes images of existing steel plates, text region label coordinates, and text information;

[0018] According to a preset code spraying rule, images, text region label coordinates, and text information of a plurality of virtual steel plates are constructed to obtain a plurality of second code spraying information;

[0019] Part of the training data and the plurality of second code spraying information are combined to obtain training data.

[0020] In an implementable manner, the steps of inputting the training data into a steel plate code spraying information pre-training model to obtain a steel plate code spraying information detection training model and a steel plate code spraying information recognition training model include:

[0021] A segmented network model is used to obtain a pre-detection training model of steel plate code spraying information;

[0022] A text recognition network model is used to obtain a pre-recognition training model of steel plate code spraying information;

[0023] The pre-detection training model of the steel plate code information and the pre-recognition training model of the steel plate code information are combined to obtain a steel plate code information pre-training model.

[0024] The pre-detection training model of the steel plate code information is trained to obtain a steel plate code information detection training model.

[0025] The pre-recognition training model of the steel plate code information is trained to obtain a steel plate code information recognition training model.

[0026] In an implementable manner, the step of inputting the training data into the steel plate code information pre-training model to obtain a steel plate code information detection training model and a steel plate code information recognition training model comprises:

[0027] The training data set with labels is input into the pre-detection training model of the steel plate code information, the parameters of the pre-detection training model of the steel plate code information are updated using a back propagation algorithm, and the steel plate code information detection training model with a loss function reaching a preset convergence condition is obtained.

[0028] The training data set with labels is input into the pre-recognition training model of the steel plate code information, the parameters of the pre-recognition training model of the steel plate code information are updated using a back propagation algorithm, and the steel plate code information recognition training model with a loss function reaching a preset convergence condition is obtained.

[0029] The steel plate code information detection training model and the steel plate code information recognition training model are combined to obtain a steel plate code information training model.

[0030] In an implementable manner, the step of training the pre-detection training model of the steel plate code information to obtain a steel plate code information detection training model comprises:

[0031] In the pre-detection training model of the steel plate code information, the pixel values of the code label area of each picture in the training data are extracted using a convolutional neural network to obtain a two-dimensional feature map of each picture.

[0032] The two-dimensional feature maps are connected to obtain a fused feature map of each picture.

[0033] The fused feature map is processed using an FCN network structure in the pre-detection model of the steel plate code information to obtain a probability map and a threshold map of each picture, wherein the probability map represents the probability that a feature point is a target, and the threshold map represents the boundary between the labeled code information and the background.

[0034] The probability map and the threshold map of each picture are processed by using differentiable binarization to obtain an approximate binary map of each picture.

[0035] The probability map, the threshold map and the approximate binary map of each picture are used for supervised learning, the parameters are updated using a back propagation algorithm, a loss function is calculated, and the loss function is made to reach a preset convergence condition to obtain the steel plate code information detection training model.

[0036] In an implementable manner, the step of training the pre-identification training model of the steel plate code information to obtain the steel plate code information identification training model comprises:

[0037] The features of the text region of the training data are extracted by using the convolutional neural network in the text recognition network model to obtain the image features of the steel plate code;

[0038] The sequence features of the steel plate code are learned by using the bidirectional long short-term memory network in the text recognition network model to obtain long semantic information related in front and back;

[0039] The long semantic information related in front and back is mapped to obtain the probability distribution of specific characters;

[0040] The probability distribution of specific characters is sorted by using a regularization method to obtain the text information of the training data;

[0041] The image features and the text labels in the training data set are used for supervised learning, the text recognition network model parameters are updated using a back propagation algorithm, a loss function is calculated, and the loss function is made to reach a preset convergence condition to obtain the steel plate code information identification training model.

[0042] In an implementable manner, the step of comparing the identified text region of the test data with the corresponding text region of the existing steel plate and comparing the identified text information of the test data with the corresponding text information of the existing steel plate to obtain an evaluation index comprises:

[0043] The error of the identified text region of the test data and the corresponding text region of the existing steel plate is calculated, and the error of the identified text information of the test data and the corresponding text information of the existing steel plate is calculated to obtain the evaluation index, wherein the evaluation index at least includes prediction accuracy, recall rate and F value.

[0044] In an implementable manner, the step of correcting the steel plate code information training model by using the evaluation index to obtain a steel plate code information inference model comprises:

[0045] The steel plate code information training model is evaluated multiple times using the prediction accuracy, the recall rate and the F value respectively to obtain a judgment result;

[0046] The judgment result is compared with a preset threshold value,

[0047] If the judgment result is less than the preset threshold value, the steel plate code information training model is optimized and adjusted, iteratively trained, and the steps of the first identification and the second identification are cyclically executed to obtain an optimal steel plate code information training model;

[0048] If the judgment result is greater than or equal to the preset threshold value, an optimal steel plate code information training model is obtained;

[0049] The optimal steel plate code information training model is converted into a steel plate code information inference model and is quantized to obtain a required deployed steel plate code information inference model, wherein the quantization represents that the weights of the model precision parameters are converted without loss of accuracy to adjust the calculation speed.

[0050] In an implementable manner, the step of fusing the code information to be fused of the continuous multiple frames of the steel plate to be tested by using a steel plate code information fusion algorithm to obtain the fused code information of the steel plate to be tested comprises:

[0051] Obtaining continuous first, second and third frames of images, the first, second and third frames of images being code information to be fused;

[0052] Using the steel plate batch number of the steel plate as an identifier, the number of times of occurrence of code information in the first, second and third frames of images is counted to obtain a statistical result;

[0053] The statistical result is compared with a preset threshold value;

[0054] If the statistical result is less than the preset threshold value, the code information of the first frame of images, the code information of the second frame of images and the code information of the third frame of images are excluded;

[0055] If the statistical result is greater than or equal to the preset threshold value, the code information of the first frame of images, the code information of the second frame of images and the code information of the third frame of images are retained;

[0056] Using a fusion algorithm, the code information of the first, second and third frames of images is respectively split into parts to obtain the information of each part of the first, second and third frames of images, respectively, the parts including the data of steel grade, size, steel plate batch number and team;

[0057] respectively, and the edit distance between the steel grade information of the first frame image, the second frame image and the third frame image and the actual inventory steel grade list is calculated, the most similar steel grade is selected, if the edit distances of the steel grade data of the first frame image, the steel grade data of the second frame image and the steel grade data of the third frame image are all less than a threshold value, the most similar steel grade is assigned, and whether all the most similar steel grades point to the same steel grade is determined; if yes, the current steel grade is fused into the same steel grade pointed to by all the most similar steel grades, and if no, the correction of the current steel grade is ended;

[0058] According to the size and team data of the first frame image, the second frame image and the third frame image, the same data number greater than or equal to a preset threshold value and the size and team within a preset numerical range are counted, and fusion is performed.

[0059] The fused code information of each part is combined to form complete steel plate code information, and the fusion code information of the to-be-tested steel plate is obtained.

[0060] The second aspect of the application provides a steel plate code information recognition system, which is applied to the steel plate code information recognition method described above, and the system comprises:

[0061] An application end is configured to obtain the code information of the to-be-tested steel plate.

[0062] A server end is connected to the application end, and the server end is configured to:

[0063] Obtain sample pictures and virtual pictures, label the sample pictures and the virtual pictures respectively, and obtain training data and test data correspondingly, the training data comprises first code information on existing steel plates, and second code information on virtual steel plates is constructed according to a preset code rule, and the test data comprises at least part of the first code information on the existing steel plates.

[0064] The training data is input into a steel plate code information pre-training model to obtain a steel plate code information detection training model and a steel plate code information recognition training model, and the steel plate code information detection training model and the steel plate code information recognition training model form a steel plate code information training model.

[0065] The test data is identified for the first time by using the steel plate code information detection training model to obtain the recognized text area of the test data.

[0066] The text area of the test data is identified for the second time by using the steel plate code information recognition training model to obtain the recognized text information of the test data.

[0067] Comparing the recognized text area of the test data with the corresponding text area of the existing steel plate and comparing the recognized text information of the test data with the corresponding text information of the existing steel plate, an evaluation index is obtained;

[0068] Using the evaluation index, the steel plate code spraying information training model is corrected to obtain an optimal steel plate code spraying information training model, which is converted into a steel plate code spraying information inference model;

[0069] Using the steel plate code spraying information inference model, the to-be-tested steel plate is continuously identified to obtain continuous multiple frames of to-be-fused code spraying information of the to-be-tested steel plate;

[0070] Using a steel plate code spraying information fusion algorithm, the continuous multiple frames of to-be-fused code spraying information of the to-be-tested steel plate are fused to obtain fused code spraying information of the to-be-tested steel plate;

[0071] The fused code spraying information is compared with inventory information or an out-of-stock preset rule, and information different from the inventory information and information not meeting the out-of-stock preset rule are specially marked and warned, the inventory information refers to steel plate code spraying information in a stock system, and the out-of-stock preset rule refers to a value range of a steel type and a size preset in an out-of-stock link.

[0072] The third aspect of the application provides a computer storage medium having a computer program stored thereon, and the computer program is executed by a processor to implement the steps of the steel plate code spraying information recognition method.

[0073] Advantages:

[0074] The application provides a steel plate code spraying information recognition method and system. First, sample pictures and virtual pictures are obtained, and the sample pictures and the virtual pictures are labeled respectively to obtain training data and test data. Then, the training data is input into a steel plate code spraying information pre-training model to obtain a steel plate code spraying information training model. Next, the test data is identified for the first time by using the steel plate code spraying information training model to obtain an identified text area of the test data. Then, the text area of the test data is identified for the second time by using the steel plate code spraying information training model to obtain identified text information of the test data. Finally, the identified text area of the test data is compared with the corresponding text area of the existing steel plate, and the identified text information of the test data is compared with the corresponding text information of the existing steel plate to obtain evaluation indexes. The evaluation indexes are used to modify the steel plate code spraying information training model to obtain an optimal steel plate code spraying information training model, which is converted into a steel plate code spraying information inference model. The steel plate code spraying information inference model is used to continuously identify a to-be-tested steel plate to obtain to-be-fused code spraying information of the to-be-tested steel plate. The to-be-fused code spraying information of the to-be-tested steel plate is fused by using a steel plate code spraying information fusion algorithm to obtain fused code spraying information of the to-be-tested steel plate. The fused code spraying information of the to-be-tested steel plate is compared with inventory information or a preset out-of-warehouse rule. If there is a difference between the fused code spraying information and the inventory information or the fused code spraying information does not conform to the preset out-of-warehouse rule, the information is given a special mark for warning. The above method can use the steel plate code spraying information inference model and the fusion algorithm to collect and identify the side code spraying information of the steel plates in the warehouse and on the vehicle in a complex scene, reduce human intervention, and improve the inventory management efficiency of the steel warehouse. BRIEF DESCRIPTION OF DRAWINGS

[0075] In order to more clearly illustrate the specific embodiments of the present application or the technical solutions in the prior art, the drawings required to be used in the specific embodiments or the prior art description will be briefly introduced below. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0076] Figure 1 A flowchart of a steel plate code spraying information recognition method of the present application;

[0077] Figure 2 A flowchart of obtaining sample pictures and virtual pictures, labeling the sample pictures and the virtual pictures, and obtaining training data and test data in a steel plate code spraying information recognition method of the present application;

[0078] Figure 3 A flowchart of obtaining a steel plate code spraying information detection training model in a steel plate code spraying information recognition method of the present application;

[0079] Figure 4 A flowchart for obtaining a steel plate code spraying information recognition training model of a steel plate code spraying information recognition method of the present application;

[0080] Figure 5 A flowchart for obtaining a steel plate code spraying information inference model of a steel plate code spraying information recognition method of the present application;

[0081] Figure 6 A flowchart for obtaining continuous multiple frames of to-be-fused code spraying information of to-be-tested steel plates of a steel plate code spraying information recognition method of the present application;

[0082] Figure 7 A flowchart for sub-analysis of a steel plate code spraying information recognition method of the present application;

[0083] Figure 8 A flowchart for data fusion of a steel plate code spraying information recognition method of the present application;

[0084] Figure 9 A flowchart for a steel plate code spraying information recognition method of the present application;

[0085] Figure 10 A system architecture diagram of a steel plate code spraying information recognition system of the present application. DETAILED DESCRIPTION

[0086] The technical solutions of the present application will be described in detail below in conjunction with embodiments. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0087] In order to facilitate the technical solutions of the application, the following first explains some concepts related to the present application.

[0088] The segmented network model refers to the segmented network model DBNet, DB, Differentable Binarization, which is a commonly used method in text detection in natural scenes based on segmentation. It can be binarized in a segmentation network, and the segmentation network can adaptively set the threshold of binarization, not only simplifying post-processing, but also improving the effect of text detection, and finding a relatively ideal balance point between precision and speed. It belongs to a conventional segmentation method.

[0089] The text recognition grid model refers to a CRNN (Convolutional Recurrent Neural Network) network model, which is used for end-to-end recognition of text sequences of indefinite length without cutting individual characters first. Instead, the text recognition is converted into time-dependent sequence learning. The text recognition grid model includes three parts, which are called convolutional layer, recurrent layer, and transcription layer. The convolutional layer is composed of CNN, which is used to extract features from the input image. The extracted feature map will be input to the next recurrent layer, which is composed of RNN, which will output the prediction of each frame of the feature sequence. Finally, the transcription layer converts the obtained prediction probability distribution into a label sequence to obtain the final recognition result. It is actually the loss function in the model. By minimizing the loss function, the network composed of CNN and RNN is trained.

[0090] The backpropagation algorithm refers to a commonly used method for training artificial neural networks. It calculates the error between the network output and the expected output, and adjusts the network weights according to the error to gradually optimize the performance of the network.

[0091] Softmax is a mathematical function that is usually used to convert a set of arbitrary real numbers into real numbers representing a probability distribution. It is essentially a normalization function that can convert a set of arbitrary real number values into probability values between [0, 1]. Because softmax converts them to values between 0 and 1, they can be interpreted as probabilities. If one of the inputs is small or negative, softmax will turn it into a small probability, and if the input is large, it will turn it into a large probability, but it will always remain between 0 and 1.

[0092] The evaluation index refers to the index used to gradually optimize the model. The prediction accuracy, also known as precision, refers to the proportion of correctly classified samples in the total number of samples in a classification task. The error rate refers to the proportion of incorrectly classified samples in the total number of samples. Recall, also known as recall, refers to the proportion of correctly classified positive samples to the total number of positive samples. Intuitively, recall refers to the ability of the classifier to find all positive samples. F value refers to F1 value, which takes into account both precision and recall (because there is a contradiction between precision and recall). The prediction accuracy, recall, and F value included in the evaluation index are all conventional indicators with conventional calculation formulas.

[0093] Pruning quantization refers to precision conversion through TensorRT, which is a C++ inference framework that can run on various NVIDIA GPU hardware platforms. TensorRT can decompose and fuse trained models, and the fused models have high degree of integration. For example, after convolutional layers and activation layers are fused, the calculation speed can be improved.

[0094] Multi-frame data fusion method refers to integrating and fusing information from multiple time series or multiple data sources to improve the accuracy and reliability of data analysis and decision-making. For example, by calculating the similarity or distance between multiple time series, data with high similarity is fused. Common similarity calculation methods include Euclidean distance, cosine similarity, and Pearson correlation coefficient.

[0095] As shown in Figure 1 and Figure 9 The present application relates to a method for identifying the code information of a steel plate, comprising:

[0096] S100: Obtain sample pictures and virtual pictures, and label the sample pictures and virtual pictures respectively to obtain training data and test data.

[0097] The training data includes first code information on existing steel plates, and second code information on virtual steel plates constructed according to a preset code rule. The test data includes at least part of the first code information on existing steel plates.

[0098] The sample pictures are obtained by photographing the existing steel plates. If multiple steel plates are stored in a warehouse, each steel plate can form at least one sample picture. It should be noted that each steel plate can also form multiple sample pictures. The first code information on the existing steel plates can be collected manually by workers using handheld devices on site (such as a warehouse), or collected by a guide rail robot camera.

[0099] The virtual pictures are virtual steel plate pictures constructed in a computer using a preset code rule. The virtual steel plate pictures have the same code font and foreground and background as the existing steel plates, and the purpose is to increase the number of training data and improve accuracy. That is, the second code information refers to the side code that may appear on the steel plate in the future actual scene.

[0100] It should be noted that, due to the continuous entry and exit of steel plates in the warehouse, the side spray code of the steel plate changes continuously, and the application needs to have the ability to identify the future side spray code of the steel plate. For this reason, a virtual picture is established, which is established according to the preset spray code rule. That is, since the image acquisition can only collect the existing side spray code of the steel plate in the on-site warehouse, it is impossible to collect the side spray code information such as the steel plate number, steel type and specification that may be produced in the future. However, the side spray code of the steel plate is generally coded in sequence, so the side spray code of the steel plate in the future can be estimated according to the spray code rule. For example, the production year of the steel plate number in the current steel plate side spray code is only up to 23 years, and the spray code data of 24 and 25 years and even later needs to be synthesized, therefore, according to the preset spray code rule, the second spray code information on the virtual steel plate is constructed, which can improve the recognition rate in the steel plate spray code information model training.

[0101] It should also be noted that the existing sample pictures on the steel plate are used as test data for recognition rate testing. The sample pictures and virtual pictures included in the training data can increase the training amount.

[0102] As shown in Figure 2 , specifically, obtaining sample pictures and virtual pictures as a data preparation stage can include two processes of data acquisition and data labeling. The sample pictures and virtual pictures are obtained, and the sample pictures and virtual pictures are labeled to obtain the specific steps of training data and test data, including S101 to S104.

[0103] S101: Labeling the sample pictures and virtual pictures in the training data to obtain the labeled training data set and test data set.

[0104] Wherein, labeling means marking the text region and text information in the sample picture and virtual picture.

[0105] Wherein, the sample pictures and virtual pictures in the training data are labeled to obtain the labeled pictures, that is, through this way, the steel plate spray code information pre-training model is informed that the labeled position is the region that needs to be identified, so that the steel plate spray code information pre-training model can be identified. After labeling the sample pictures and virtual pictures, the training data set is formed.

[0106] The labeling set mainly adopts a manual labeling method to label all pictures.

[0107] S102: Obtain the first spray code information on the existing steel plate to obtain the test data and part of the training data.

[0108] Wherein, the first spray code information includes the image of the existing steel plate, the text region labeling coordinates and the text information.

[0109] Specifically, after obtaining the sample picture and the virtual picture in the foregoing steps, the sample picture and the virtual picture are respectively data-labeled in the computer. For example, after opening the picture, the text information of the code in the picture is framed to form a framed area, and the labeling coordinates of the text area and the text labeling are established. In this way, the image of the picture and the text area are distinguished, and the steel plate code information pre-training model is informed that the framed area is the text area, which is the area that the steel plate code information pre-training model needs to mainly detect and identify. For the steel plate code information pre-training model, after the text area and the text content are determined, the data recognition training of the text area and the non-text area can be performed. In this way, after the steel plate code information pre-training model is trained for multiple times, when the steel plate code information pre-training model is input with a picture without data labeling, the steel plate code information model can still identify the code information of the picture without data labeling according to the parameters obtained through the previous training.

[0110] S103: According to the preset code spraying rule, the image of a plurality of virtual steel plates, the text area labeling coordinates and the text information are constructed to obtain a plurality of second code spraying information.

[0111] Specifically, the preset code spraying rule can be artificially set, for example, the code spraying rule is formed according to the year, the batch and the steel plate model. Based on this code spraying rule, the text information that can appear on the future steel plate is constructed, that is, the virtual steel plate picture is constructed in the computer by using the preset code spraying rule. In addition, in order to increase the data samples for training, the background and the text of the virtual steel plate can be established when the virtual steel plate is constructed. The data samples, that is, the virtual pictures, are synthesized by using the background, the font and the preset code spraying rule. By adjusting the background and the code spraying information of the constructed virtual picture, the training data under different conditions is obtained. By using these training data, the amount of training data of the steel plate code information pre-training model can be increased, and the training effect of the steel plate code information model can be improved.

[0112] S104: The part of the training data and the plurality of second code spraying information are combined to obtain the training data.

[0113] It can be understood that the training data is composed of two parts, one of which is composed of the existing steel plate first code spraying information and the second code spraying information, so as to obtain the complete training data.

[0114] Need to explain, after obtaining the sample picture and the virtual picture, the sample picture and the virtual picture are labeled, and whether the data is cleaned according to the need, the action can also be called data preprocessing, through the filtering mode, the picture data that does not meet the requirements or unqualified in the obtained sample picture and the virtual picture is removed, the meaningless labeling work is reduced, and the labeling efficiency is improved. The common operation of data cleaning includes: cleaning fuzzy, similar data, cropping, rotating, mirroring, picture brightness adjustment, picture contrast adjustment, picture sharpening and the like. The way of data cleaning is not limited in the application, which can reduce the removal of data that does not meet the requirements.

[0115] S200: input the training data into the steel plate code information pre-training model to obtain a steel plate code information detection training model and a steel plate code information recognition training model.

[0116] The steel plate code information detection training model and the steel plate code information recognition training model constitute a steel plate code information training model.

[0117] The pre-detection training model of the steel plate code information is obtained by using the segmentation network model. The pre-recognition training model of the steel plate code information is obtained by using the text recognition network model. The steel plate code information pre-training model is obtained by combining the pre-detection training model of the steel plate code information and the pre-recognition training model of the steel plate code information. The training data is input into the steel plate code information pre-training model to train the steel plate code information pre-training model. The training is performed by using the segmentation network model and the text recognition network model respectively. It should be noted that the segmentation network model and the text recognition network model correspond to the detection and recognition steps required in the steel plate code information recognition process respectively. Specifically, the detection model of the steel plate code information is to locate the text area position of the steel plate and distinguish the foreground and the background. The recognition model of the steel plate code information is to recognize the text content of the text area. It should be further noted that on the basis of balancing the accuracy and the speed, a two-stage steel plate code information model is adopted, that is, the detection model of the steel plate code information and the recognition model of the steel plate code information are connected in series, which has the characteristics of short training period, high accuracy and strong pertinence.

[0118] Specifically, the training data set with labels is input into the pre-detection training model of the steel plate code information, the parameters of the pre-detection training model of the steel plate code information are updated by using the back propagation algorithm, and the steel plate code information detection training model with the loss function reaching the preset convergence condition is obtained.

[0119] The training data set with labels is input into the pre-recognition training model of the steel plate code information, the parameters of the pre-recognition training model of the steel plate code information are updated by using the back propagation algorithm, and the steel plate code information recognition training model with the loss function reaching the preset convergence condition is obtained.

[0120] The steel plate code information detection training model and the steel plate code information recognition training model are combined to obtain a steel plate code information training model.

[0121] As shown in Figure 3 The training data is input into the steel plate code information pre-detection training model to obtain the steel plate code information detection training model, and the specific steps include steps S201 to S205.

[0122] S201: In the steel plate code information pre-detection training model, the pixel values of the code marking area of each picture in the training data are extracted by using a convolutional neural network to obtain a two-dimensional feature map of each picture.

[0123] S202: The two-dimensional feature maps are connected to obtain a fused feature map of each picture.

[0124] S203: The fused feature map is processed by using the FCN network structure in the steel plate code information detection model to obtain a probability map and a threshold map of each picture.

[0125] The probability map represents the probability that the feature point is a target, and the threshold map represents the boundary between the marked code information and the background.

[0126] S204: The probability map and the threshold map of each picture are processed by using a differentiable binarization to obtain an approximate binary map of each picture.

[0127] S205: Supervised learning is performed by using the probability map, the threshold map and the approximate binary map of each picture, the parameters are updated by using a back propagation algorithm, a loss function is calculated, and the preset convergence condition is reached to obtain the steel plate code information detection training model.

[0128] In this embodiment, the pre-detection training model of the steel plate code information is preferably a segmentation-based network model DBNet, which performs pixel-level classification of foreground and background in the image. The specific implementation is that first, the convolutional neural network of the segmentation network model DBNet is used to extract each picture in the labeled training data respectively, the pixel value of the code labeling area of each picture is extracted as a feature point, after extracting a plurality of feature points, a feature map can be formed by connecting the feature points, and a fused feature map is obtained by connecting the two-dimensional feature map. The fused feature map generates a probability map P and a threshold map T through the FCN network structure of the segmentation network model, the probability map P represents the probability that the feature point is a target, and the threshold map T represents the boundary between the target area (text area) and the background; the probability map P and the threshold map T are obtained by differentiable binarization to obtain an approximate binary map, and in the training process, the three maps are supervised learning, using the back propagation algorithm, the parameters of each module in the steel plate code information recognition training model are iteratively updated until the loss function reaches the preset convergence condition. The inference process directly uses the probability map, and then uses a fixed threshold to obtain the result. The differentiable binarization is a key point for improving the inference speed of the entire network.

[0129] As shown in Figure 4 the process of inputting the training data into the steel plate code information pre-recognition model to obtain the steel plate code information recognition training model further includes steps S206 to S210.

[0130] S206: The features of the text area of the training data are extracted using the convolutional neural network in the text recognition network model to obtain the image features of the steel plate code.

[0131] S207: The sequence features of the steel plate code are learned using the bidirectional long short-term memory network in the text recognition network model to obtain the long semantic information related to the front and back.

[0132] S208: The long semantic information related to the front and back is mapped to obtain the probability distribution of the specific characters.

[0133] S209: The probability distribution of the specific characters is sorted using a regularization method to obtain the text information of the training data.

[0134] S210: The image features and text labels in the training data set are used for supervised learning, the text recognition network model parameters are updated using the back propagation algorithm, the loss function is calculated, and the preset convergence condition is reached. Among them, the predicted text sequence represents the text information in the text area.

[0135] In this embodiment, the steel plate code information recognition model uses a CRNN network model to recognize the text information in the text area. The convolutional neural network (CNN) of the CRNN network model is used to extract the features of the input image data. Then, the bidirectional long short-term memory network (BLSTM) in the CRNN network model is used to learn the sequence features of the steel plate code, and the long semantic information related to the front and back is obtained. Next, the CTCLoss transcription layer decoding of the CRNN network model is used to map the fixed-length output of the BLSTM network to the probability distribution of the specific character, and the softmax function is used to regularize the probability distribution, and finally the predicted text sequence (text information) is obtained. The loss function is calculated, and the parameters are updated using the back propagation algorithm until the loss function reaches the preset convergence condition, and the steel plate code information recognition training model is obtained.

[0136] It should be noted that the detection model and the recognition model are iteratively trained multiple times to reach the loss function convergence condition, increase the recognition accuracy of the steel plate code information detection model and the steel plate code information recognition model, and make the detection model and the recognition model meet the requirements of long text detection and recognition of the steel plate side code in complex environments. Through the above-mentioned manner, the trained detection model and recognition model are obtained. Next, the test data is input into the detection model and the recognition model through the foregoing steps, and the final prediction result is obtained by using the calculation and output of the detection model and the recognition model. The prediction result is the text area and the text information.

[0137] Among them, the recognized text area is obtained through the first recognition, and the recognized text information of the test data is obtained through the second recognition using the text recognition network model. The recognized text information represents the recognized text content of the steel plate side code. It should be noted that the recognized text area and the recognized text information may have errors compared with the actual steel plate side code information, so the subsequent steps are needed to compare the recognized text area and the recognized text information with the actual code area and text content on the steel plate. During the comparison process, the steel plate code information detection and recognition training model is continuously optimized, and after multiple processes, the steel plate code information model connected in series by the detection and recognition models is formed.

[0138] S300: The steel plate code information detection training model is used to perform first recognition on the test data to obtain the recognized text area of the test data.

[0139] S400: The steel plate code information recognition model is used to perform second recognition on the text area of the test data to obtain the recognized text information of the test data.

[0140] S500: comparing the recognized text region of the test data with the text region of the corresponding existing steel plate, and comparing the recognized text information of the test data with the text information of the corresponding existing steel plate, to obtain an evaluation index.

[0141] The text region of the existing steel plate represents the code information that has actually existed on the existing steel plate.

[0142] Specifically, the error of the recognized text region of the test data and the text region of the corresponding existing steel plate is calculated, and the error of the recognized text information of the test data and the text information of the corresponding existing steel plate is calculated, both of which obtain the evaluation index, wherein the evaluation index at least includes prediction accuracy, recall rate and F value. That is, the prediction accuracy, recall rate and F value included in the evaluation index obtained by the error are used to evaluate the steel plate code information training model, and the steel plate code information training model is corrected according to the evaluation result. That is, in the subsequent step, the evaluation index can be used to judge the pros and cons of the steel plate code information model, and according to the judgment result, it is further selected whether to optimize.

[0143] S600: correcting the steel plate code information training model by using the evaluation index to obtain an optimal steel plate code information training model, and converting it into a steel plate code information inference model.

[0144] As shown in Figure 5 , wherein obtaining the steel plate code information inference model specifically includes steps S601 to S603.

[0145] S601: using a plurality of prediction accuracy, recall rate and F value to evaluate the steel plate code information training model multiple times to obtain a judgment result.

[0146] Each evaluation obtains a judgment result, that is, the judgment result obtained by each evaluation can be understood as an evaluation of the steel plate code information training model.

[0147] S602: comparing the judgment result with a preset threshold.

[0148] If the judgment result is less than the preset threshold, the steel plate code information training model is optimized and adjusted, iteratively trained, and the steps of the first recognition and the second recognition are cyclically executed to obtain an optimal steel plate code information training model.

[0149] The first recognition represents step S300, and the second recognition represents step S400.

[0150] If the judgment result is greater than or equal to the preset threshold, an optimal steel plate code information training model is obtained.

[0151] S603: Convert the optimal steel plate code information training model into a steel plate code information inference model, perform pruning quantization, and obtain the required deployed steel plate code information inference model.

[0152] The pruning quantization means that the weights of the model precision parameters are converted and the calculation speed is adjusted without losing accuracy.

[0153] In this embodiment, according to the evaluation result (judgment result) of the steel plate code information training model, the steel plate code information training model is adjusted or optimized, that is, the steel plate code information model structure and parameters are adjusted (such as adding an intermediate layer; changing the activation function type, learning rate, etc.), or the training data is increased, to further improve the accuracy and generalization ability of the steel plate code information training model, so as to obtain the optimal steel plate code information training model. In this way, a closed-loop process from the labeling of the training data to the steel plate code information training model, and then to the evaluation and adjustment of the steel plate code information training model is formed. When the steel plate code information training model is adjusted to the optimal steel plate code information training model, it means that the steel plate code information training model has reached the best model at present, and then the optimal steel plate code information training model needs to be converted into a steel plate code information inference model. In this conversion process, unnecessary structures are removed, and only network structures and parameters are retained. Then the steel plate code information inference model is deployed, and the steel plate code information inference model is mainly deployed in two deployment modes: application end deployment and server end deployment. Due to the limited computing power of handheld devices, in order to improve the computing performance, the steel plate code information inference model needs to be pruned and quantized before deployment. Pruning quantization is to convert the FP32 precision model parameters (weights) into Int8 (8-bit integer) precision without losing the accuracy of the inference model, reduce the size of the steel plate code information inference model parameters, and speed up the calculation. The quantized steel plate code information inference model has speed advantage when deployed on mobile devices. That is, the application end applies the pruned and quantized steel plate code information inference model, and the server end can apply the steel plate code information inference model without pruning and quantization.

[0154] S700: Use the steel plate code information inference model to continuously identify the steel plate to be tested, and obtain the to-be-fused code information of the continuous multiple frames of steel plate to be tested.

[0155] As shown in Figure 6 , specifically, obtaining the to-be-fused code information of the continuous multiple frames of steel plate to be tested specifically includes steps S701 to S707.

[0156] S701: Obtain a continuous first frame image, a second frame image, and a third frame image, the first frame image, the second frame image, and the third frame image being to-be-fused code information.

[0157] The code information is represented as text information.

[0158] S702: The number of occurrences of the code information in the first frame image, the second frame image and the third frame image is counted using the steel plate batch number of the steel plate as an identifier, and a statistical result is obtained.

[0159] S703: The statistical result is compared with a preset threshold value.

[0160] If the statistical result is less than the preset threshold value, the code information of the first frame image, the code information of the second frame image and the code information of the third frame image are excluded.

[0161] If the statistical result is greater than or equal to the preset threshold value, the code information of the first frame image, the code information of the second frame image and the code information of the third frame image are retained.

[0162] S704: The code information of the first frame image, the second frame image and the third frame image is respectively split using a fusion algorithm, and each part of information of the first frame image, the second frame image and the third frame image is obtained.

[0163] The part of information includes data information of steel type, size, steel plate batch number and team. That is, the text information at least includes data information of steel type, size, steel plate batch number and team.

[0164] S705: The edit distance between the steel type information of the first frame image, the second frame image and the third frame image and the actual inventory steel type list is calculated, the most similar steel type is selected, if the edit distance of the steel type data of the first frame image, the steel type data of the second frame image and the steel type data of the third frame image is less than the threshold value, the most similar steel type is assigned, and it is judged whether all the most similar steel types point to the same steel type; if yes, the current steel type is fused into the same steel type pointed to by all the most similar steel types, and if no, the correction of the current steel type is ended.

[0165] S706: According to the size and team data of the first frame image, the second frame image and the third frame image, the same data is counted, which is greater than or equal to a preset threshold value, and the size and team are within a preset numerical range, and fusion is performed.

[0166] S707: The fused each part of code information is composed into complete steel plate code information, and the fusion code information of the to-be-tested steel plate is obtained.

[0167] S800: The fusion code information is compared with the inventory information or the preset rule of the warehouse, and information different from the inventory information and information not meeting the preset rule of the warehouse is given special marking warning.

[0168] The inventory information refers to the steel plate code spraying information in the inventory system, and the outbound preset rule refers to the preset value range of the steel type and size at the outbound link.

[0169] In this embodiment, the obtained picture is prone to motion blur due to focusing problems of the shooting device, resulting in unreliable single-frame data recognition results, so data post-processing is required for the recognized results, that is, fusion code spraying information of the to-be-tested steel plate is obtained through data post-processing to improve the accuracy of the fusion code spraying information of the to-be-tested steel plate. Data post-processing mainly includes analysis of each part of the recognition result, design of a data fusion algorithm, and the like. The specific sub-analysis process is shown in FIG. 6. Figure 7 , Figure 7 The process in FIG. 6 first splits the recognition result according to spaces, and stores the split content in an array; then, from back to front, the first index equal to the length of the steel plate batch number and meeting the format is found, and the first 13 bits (the length of the batch number is 13 bits) are taken; the size position is found according to the index position, and the thickness, width, and length are divided according to X; the furnace number is further searched according to the size or furnace number index; and the steel type is found according to the size or furnace number index. The steel plate batch number index position is backward to the team or empty. Exemplarily, first, the side code spraying rule is counted according to the preset code spraying rule or the collected sample picture, such as steel plate code spraying rule 1: identifier + steel type (mixed English and numbers) + space + thickness (numbers or decimal point) + X + width (numbers or decimal point) + X + length (numbers or decimal point) + space + steel plate batch number (mixed English and numbers) + space + (team number). After filtering out the recognition results with a length that is too short, the recognition result is sub-analyzed according to the code spraying rule. First, the steel plate batch number is obtained by splitting according to spaces, and the team, size, and steel type are extracted according to the steel plate batch number position. Each item of the size is split into length, width, and thickness information by X. Taking DH36||18X2500X12000||23S-025369-06||(D11) as an example, (|| is used instead of spaces for demonstration convenience), the steel plate code is split into four items by ||, the steel plate batch number 23S-025369-06 is determined according to the 13-bit steel plate batch number, the text between () is the team D11, which is searched backward from the steel plate batch number position, the size 18X2500X12000 is searched forward until the next ||, the size is split by X, and the length, width, and thickness are 12000, 2500, and 18 in turn; and the steel type DH36 is searched forward until the next ||, and thus the analysis of the steel plate code is completed.

[0170] As Figure 8 and Figure 9As shown, specifically, the data fusion adopts a multi-frame data fusion method, fuses the current image recognition result with the last two frames of data, and uses the last two frames of data to correct and fill the current recognition result. For example, in consecutive first, second and third frames of images, the first and second frames of images are used as the last two frames of images of the third frame of image, and the first and second frames of images are used to correct and fill the third frame of image recognition result.

[0171] First, the content in the image is screened, the steel plate batch number is selected as the unique identifier, and the number of occurrences of the batch number in the continuous three frames of recognition results is counted. If the number of occurrences is less than a threshold value, it is screened out.

[0172] Secondly, each part of the screened text information is fused. A list with empty parts is initialized to store the fusion result.

[0173] Steel grade fusion: The edit distance between the recognized steel grades and the actual inventory steel grade list is calculated, the most similar steel grade is selected, and if it points to the same and the maximum edit distance is less than a threshold value, the most similar steel grade is assigned. Size, team fusion: two frames of data are the same in three frames of data (assuming a preset threshold value of 2), otherwise discard. The results of each fusion are combined to form a complete steel plate code, and it is determined whether it has appeared in the confirmation list. If it has not appeared, it is inserted into the confirmation list. During the fusion of each part, the recognition results that do not meet the rules (such as exceeding the length, width and thickness value range, not in the team list, etc.) are filtered out. When connected with the warehouse management system, the inventory information is compared and corrected with the fusion result based on the steel plate batch number as the identifier, so as to obtain the optimal result.

[0174] It should be noted that the method for obtaining the steel plate code recognition information is to use deep learning and fusion algorithm to achieve the purpose of efficiently recognizing the steel plate side code.

[0175] The steel plate code information recognition method provided by the application can realize multi-mode image acquisition and steel plate side code recognition in a complex environment in the warehouse management of a steel warehouse, and can be applied to warehouse entry, warehouse inventory, and warehouse exit.

[0176] Corresponding to the embodiments of the foregoing steel plate code information recognition method, the second aspect of the application also provides an embodiment of a steel plate code information recognition system.

[0177] As Figure 10 shown, the second aspect of the application provides a steel plate code information recognition system, which includes an application end and a server end.

[0178] The application end is used to obtain the code information of the steel plate to be tested.

[0179] The server end is connected with the application end, and the server end is configured to:

[0180] Obtaining a sample picture and a virtual picture, respectively labeling the sample picture and the virtual picture, and correspondingly obtaining training data and test data, the training data including first code information on an existing steel plate, and second code information on a virtual steel plate constructed according to a preset code rule, and the test data including at least part of the first code information on the existing steel plate;

[0181] Inputting the training data into a steel plate code information pre-training model to obtain a steel plate code information detection training model and a steel plate code information recognition training model, the steel plate code information detection training model and the steel plate code information recognition training model forming a steel plate code information training model;

[0182] Performing first recognition on the test data by using the steel plate code information detection training model to obtain a recognized text region of the test data;

[0183] Performing second recognition on the text region of the test data by using the steel plate code information recognition training model to obtain recognized text information of the test data;

[0184] Comparing the recognized text region of the test data with a corresponding text region of the existing steel plate, and comparing the recognized text information of the test data with corresponding text information of the existing steel plate, to obtain an evaluation index;

[0185] Using the evaluation index to correct the steel plate code information training model to obtain an optimal steel plate code information training model, and converting the optimal steel plate code information training model into a steel plate code information inference model;

[0186] Using the steel plate code information inference model to perform continuous recognition on a to-be-tested steel plate to obtain to-be-fused code information of the continuous multiple frames of the to-be-tested steel plate;

[0187] Using a steel plate code information fusion algorithm to fuse the to-be-fused code information of the continuous multiple frames of the to-be-tested steel plate to obtain fused code information of the to-be-tested steel plate.

[0188] Comparing the fused code information with inventory information or out-of-warehouse preset rules, and giving special marks and warnings to information that is different from the inventory information and information that does not conform to the out-of-warehouse preset rules, the inventory information referring to steel plate code information in a warehouse system, and the out-of-warehouse preset rules referring to a value range of steel grade and size preset at an out-of-warehouse link.

[0189] As Figure 10As shown, in the image acquisition and recognition under the business scenario, through multi-mode image acquisition, the steel plate code spraying image is input and collected, image preprocessing is carried out, including grayscale, binarization and other operations, and is transmitted to a text detection model, the text area is extracted from the image as the input of the recognition model, and the initial recognition result is obtained through the recognition model. The initial recognition result is processed through the fusion algorithm, and the steel plate information is corrected and prompted through the warehouse system information comparison, and finally the complete recognition result is output.

[0190] The third aspect of the present application provides a computer storage medium, which stores a computer program, and the computer program is executed by a processor to realize the steps of the above-mentioned steel plate code information recognition method.

[0191] In the description of the embodiments of the present application, those skilled in the art should know that the embodiments of the present application can be implemented as methods, devices, electronic devices and computer readable storage media. Therefore, the embodiments of the present application can be specifically implemented in the following forms: complete hardware, complete software (including firmware, resident software, microcode, etc.), hardware and software combined form. In addition, in some embodiments, the embodiments of the present application can also be implemented in the form of computer program product in one or more computer readable storage media, which contains computer program code.

[0192] The above computer readable storage medium can adopt any combination of one or more computer readable storage media. The computer readable storage medium includes: electrical, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or devices, or any combination thereof. More specific examples of computer readable storage medium include: portable computer diskette, hard disk, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM), flash memory (Flash Memory), optical fiber, compact disk read only memory (CD-ROM), optical storage device, magnetic storage device or any combination thereof. In the embodiments of the present application, the computer readable storage medium can be any tangible medium containing or storing programs, which can be used or combined with instruction execution system, device or device.

[0193] The computer program code contained in the above computer readable storage medium can be transmitted by any appropriate medium, including: wireless, wire, optical cable, radio frequency (Radio Frequency, RF) or any appropriate combination thereof.

[0194] Computer program code for carrying out operations of embodiments of the present application can be written in an assembly language, an instruction-set-architecture (ISA) language, machine language, machine dependent language, microcode, firmware, state-setting data, integrated circuit configuration data, or in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer program code can execute entirely on a user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0195] Embodiments of the present application are described herein with reference to the accompanying drawings.

[0196] It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer readable program instructions. These computer readable program instructions can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0197] These computer readable program instructions can also be stored in a computer readable storage medium that can be a magnetic, optical, or other storage device. The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0198] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0199] The terms "first" and "second" and the like in the description and claims of the present application are used for distinguishing between similar elements and not necessarily for describing a specific sequential or chronological order. For example, the first target object and the second target object are used for distinguishing between two structurally similar objects but not for describing specific sequential or chronological order.

[0200] In the present application, the words "exemplary" and "for example" are used to help clarify the description of embodiments of the present application. Any embodiment or design scheme described as "exemplary" or "for example" in the present application should not be interpreted as being more preferred or advantageous than other embodiments or design schemes. Rather, the use of the words "exemplary" and "for example" is intended to present concepts in a particular manner.

[0201] In the description of the present application, the meaning of "a plurality of" is two or more unless otherwise specified. For example, a plurality of processing units means two or more processing units; a plurality of systems means two or more systems.

[0202] The above embodiments only express the specific implementation of the present application, which is described in a more specific and detailed manner. However, it should not be understood as a limitation on the scope of the patent of the present application. It should be noted that for those skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are all within the scope of protection of the present application.

Claims

1. A method of identifying code information of a steel sheet, characterized by, The method comprises: acquiring sample pictures and virtual pictures, respectively labeling the sample pictures and the virtual pictures, and correspondingly obtaining training data and test data, the training data comprising first code information on existing steel plates, and second code information on virtual steel plates constructed according to preset code rules, and the test data comprising at least part of the first code information on the existing steel plates; inputting the training data into a steel plate code information pre-training model to obtain a steel plate code information detection training model and a steel plate code information recognition training model, the steel plate code information detection training model and the steel plate code information recognition training model forming a steel plate code information training model; performing first recognition on the test data by using the steel plate code information detection training model to obtain recognized text regions of the test data; performing second recognition on the text regions of the test data by using the steel plate code information recognition training model to obtain recognized text information of the test data; comparing the recognized text regions of the test data with corresponding text regions of the existing steel plates, and comparing the recognized text information of the test data with corresponding text information of the existing steel plates, and obtaining evaluation indexes; correcting the steel plate code information training model by using the evaluation indexes to obtain an optimal steel plate code information training model, and converting the optimal steel plate code information training model into a steel plate code information inference model; performing continuous recognition on a to-be-tested steel plate by using the steel plate code information inference model to obtain to-be-fused code information of the to-be-tested steel plate in multiple frames in succession; fusing the to-be-fused code information of the to-be-tested steel plate in multiple frames in succession by using a steel plate code information fusion algorithm to obtain fused code information of the to-be-tested steel plate; comparing the fused code information with inventory information or out-of-warehouse preset rules, giving special marks and warnings to information different from the inventory information and information not meeting the out-of-warehouse preset rules, the inventory information referring to steel plate code information in a warehouse inventory system, and the out-of-warehouse preset rules referring to a value range of steel type and size preset in an out-of-warehouse link.

2. The method of identifying the inkjet-printed information of a steel sheet according to claim 1, characterized by, The step of acquiring sample pictures and virtual pictures, respectively labeling the sample pictures and the virtual pictures, and correspondingly obtaining training data and test data comprises: labeling the sample pictures and the virtual pictures in the training data to obtain labeled training data sets and test data sets, the labeling indicating that text regions and text information are labeled in the sample pictures and the virtual pictures, respectively; acquiring the first code information on a plurality of existing steel plates to obtain the test data and part of the training data, wherein the first code information comprises images of existing steel plates, text region label coordinates, and text information; constructing images of a plurality of virtual steel plates, text region label coordinates, and text information according to preset code rules to obtain a plurality of second code information; combining part of the training data and the plurality of second code information to obtain training data.

3. The method of identifying the inkjet-printed information of a steel sheet according to claim 2, characterized by, The step of inputting the training data into the steel plate code information pre-training model to obtain a steel plate code information detection training model and a steel plate code information recognition training model comprises: A pre-detection training model of the steel plate code information is obtained by using the segmented network model; A pre-recognition training model of the steel plate code information is obtained by using the text recognition network model; The pre-detection training model of the steel plate code information and the pre-recognition training model of the steel plate code information are combined to obtain the steel plate code information pre-training model; The pre-detection training model of the steel plate code information is trained to obtain the steel plate code information detection training model; The pre-recognition training model of the steel plate code information is trained to obtain the steel plate code information recognition training model.

4. The method of identifying the inkjet-printed information of a steel sheet according to claim 3, characterized by, The step of inputting the training data into the steel plate code information pre-training model to obtain a steel plate code information detection training model and a steel plate code information recognition training model comprises: The pre-detection training model of the steel plate code information is obtained by using the segmented network model; A pre-recognition training model of the steel plate code information is obtained by using the text recognition network model; The pre-detection training model of the steel plate code information and the pre-recognition training model of the steel plate code information are combined to obtain the steel plate code information pre-training model; 5. The method of identifying the inkjet-printed information of a steel sheet according to claim 4, characterized in that, The pre-detection training model of the steel plate code information is trained to obtain the steel plate code information detection training model; The pre-recognition training model of the steel plate code information is trained to obtain the steel plate code information recognition training model. The step of inputting the training data into the steel plate code information pre-training model to obtain a steel plate code information detection training model and a steel plate code information recognition training model comprises: The pre-detection training model of the steel plate code information is obtained by using the segmented network model; A pre-recognition training model of the steel plate code information is obtained by using the text recognition network model; The pre-detection training model of the steel plate code information and the pre-recognition training model of the steel plate code information are combined to obtain the steel plate code information pre-training model; 6. The method of identifying the inkjet-printed information of a steel sheet according to claim 5, characterized in that, The pre-detection training model of the steel plate code information is trained to obtain the steel plate code information detection training model; The pre-recognition training model of the steel plate code information is trained to obtain the steel plate code information recognition training model. The step of inputting the training data into the steel plate code information pre-training model to obtain a steel plate code information detection training model and a steel plate code information recognition training model comprises: The pre-detection training model of the steel plate code information is obtained by using the segmented network model; A pre-recognition training model of the steel plate code information is obtained by using the text recognition network model; The pre-detection training model of the steel plate code information and the pre-recognition training model of the steel plate code information are combined to obtain the steel plate code information pre-training model; The pre-detection training model of the steel plate code information is trained to obtain the steel plate code information detection training model; The pre-recognition training model of the steel plate code information is trained to obtain the steel plate code information recognition training model. The step of inputting the training data into the steel plate code information pre-training model to obtain a steel plate code information detection training model and a steel plate code information recognition training model comprises: The pre-detection training model of the steel plate code information is obtained by using the segmented network model; A pre-recognition training model of the steel plate code information is obtained by using the text recognition network model; The pre-detection training model of the steel plate code information and the pre-recognition training model of the steel plate code information are combined to obtain the steel plate code information pre-training model; The pre-detection training model of the steel plate code information is trained to obtain the steel plate code information detection training model; The pre-recognition training model of the steel plate code information is trained to obtain the steel plate code information recognition training model. The step of inputting the training data into the steel plate code information pre-training model to obtain a steel plate code information detection training model and a steel plate code information recognition training model comprises: The pre-detection training model of the steel plate code information is obtained by using the segmented network model; A pre-recognition training model of the steel plate code information is obtained by using the text recognition network model; The pre-detection training model of the steel plate code information and the pre-recognition training model of the steel plate code information are combined to obtain the steel plate code information pre-training model; The pre-detection training model of the steel plate code information is trained to obtain the steel plate code information detection training model; The pre-recognition training model of the steel plate code information is trained to obtain the steel plate code information recognition training model. The step of inputting the training data into the steel plate code information pre-training model to obtain a steel plate code information detection training model and a steel plate code information recognition training model comprises: The pre-detection training model of the steel plate code information is obtained by using the segmented network model; A pre-recognition training model of the steel plate code information is obtained by using the text recognition network model; The pre-detection training model of the steel plate code information and the pre-recognition training model of the steel plate code information are combined to obtain the steel plate code information pre-training model; The pre-detection training model of the steel plate code information is trained to obtain the steel plate code information detection training model; The pre-recognition training model of the steel plate code information is trained to obtain the steel plate code information recognition training model. The sequence features of the steel plate code spraying are learned by using a bidirectional long short-term memory network in the text recognition network model, and long semantic information related in front and back is obtained. The long semantic information related in front and back is mapped to obtain a probability distribution of specific characters. The probability distribution of specific characters is sorted by using a regularization method to obtain text information of the training data. The image features and text labels in the training data set are used for supervised learning, the text recognition network model parameters are updated using a back propagation algorithm, a loss function is calculated, and the preset convergence condition is reached to obtain the steel plate code information recognition training model.

7. The method of identifying the inkjet-printed information of a steel sheet according to claim 1, characterized by, The steps of comparing the recognized text area of the test data with the corresponding text area of the existing steel plate and comparing the recognized text information of the test data with the corresponding text information of the existing steel plate to obtain evaluation indexes include: The errors of the recognized text area of the test data and the corresponding text area of the existing steel plate and the errors of the recognized text information of the test data and the corresponding text information of the existing steel plate are calculated to obtain the evaluation indexes, wherein the evaluation indexes at least include a prediction accuracy, a recall rate and an F value.

8. The method of identifying the inkjet-printed information of a steel sheet according to claim 7, characterized in that, The steps of correcting the steel plate code information training model by using the evaluation indexes to obtain a steel plate code information inference model include: The steel plate code information training model is evaluated multiple times by using the prediction accuracy, the recall rate and the F value to obtain a judgment result; The judgment result is compared with a preset threshold value, If the judgment result is less than the preset threshold value, the steel plate code information training model is optimized and adjusted, iteratively trained, and the steps of first recognition and second recognition are cyclically executed to obtain an optimal steel plate code information training model; If the judgment result is greater than or equal to the preset threshold value, an optimal steel plate code information training model is obtained; The optimal steel plate code information training model is converted into the steel plate code information inference model and is quantized by clipping to obtain a required deployed steel plate code information inference model, wherein the clipping quantization means that the weights of model precision parameters are converted and the calculation speed is adjusted without loss of accuracy.

9. The method of identifying the inkjet-printed information of a steel sheet according to claim 8, characterized in that, The steps of fusing the to-be-fused code information of the continuous multiple frames of the to-be-tested steel plate by using a steel plate code information fusion algorithm to obtain the fused code information of the to-be-tested steel plate include: Continuous first, second and third images are obtained, and the first, second and third images are to-be-fused code information; The number of times of code information appearing in the first, second and third images is counted by using the steel plate batch number of the steel plate as an identifier to obtain a counting result; The counting result is compared with a preset threshold value; If the counting result is less than the preset threshold value, the code information of the first image, the code information of the second image and the code information of the third image are excluded. If the statistical result is greater than or equal to the preset threshold, the inkjet code information of the first frame image, the inkjet code information of the second frame image and the inkjet code information of the third frame image are retained; The inkjet code information of the first frame image, the second frame image and the third frame image is respectively split by using a fusion algorithm, and the information of each part of the first frame image, the second frame image and the third frame image is obtained, wherein the each part includes the data of steel grade, size, steel plate batch number and team. The edit distances between the steel grade information of the first frame image, the second frame image and the third frame image and the actual inventory steel grade list are calculated respectively, the most similar steel grade is selected, if the edit distances of the steel grade data of the first frame image, the second frame image and the third frame image are all less than a threshold, the most similar steel grade is assigned, and it is determined whether all the most similar steel grades point to the same steel grade; if yes, the current steel grade is fused into the same steel grade pointed by all the most similar steel grades, and if no, the correction of the current steel grade is ended. According to the size and team data of the first frame image, the second frame image and the third frame image, if the number of the same data is greater than or equal to a preset threshold and the size and team are within a preset numerical range, the fusion is performed. The complete steel plate inkjet code information is formed by the fused inkjet code information of each part, and the fusion inkjet code information of the to-be-tested steel plate is obtained.

10. A steel plate inkjet printing information recognition system, characterized in that, The system is applied to the steel plate inkjet code information recognition method in any one of claims 1-9, and the system comprises: An application end is configured to acquire the inkjet code information of the to-be-tested steel plate. A server end is connected with the application end, and the server end is configured to: Acquire sample pictures and virtual pictures, and label the sample pictures and the virtual pictures respectively to obtain training data and test data, wherein the training data comprises first inkjet code information on existing steel plates, and second inkjet code information on virtual steel plates is constructed according to a preset inkjet code rule, and the test data comprises at least part of the first inkjet code information on the existing steel plates; Input the training data into a steel plate inkjet code information pre-training model to obtain a steel plate inkjet code information detection training model and a steel plate inkjet code information recognition training model, wherein the steel plate inkjet code information detection training model and the steel plate inkjet code information recognition training model constitute a steel plate inkjet code information training model; Use the steel plate inkjet code information detection training model to perform first-time identification on the test data to obtain an identified text area of the test data; Use the steel plate inkjet code information recognition training model to perform second-time identification on the text area of the test data to obtain identified text information of the test data; Compare the identified text area of the test data with the corresponding text area of the existing steel plate, and compare the identified text information of the test data with the corresponding text information of the existing steel plate to obtain an evaluation index; Use the evaluation index to correct the steel plate inkjet code information training model to obtain an optimal steel plate inkjet code information training model, and convert the optimal steel plate inkjet code information training model into a steel plate inkjet code information inference model; and The application end is configured to acquire the inkjet code information of the to-be-tested steel plate. The server end is connected with the application end, and the server end is configured to: Acquire sample pictures and virtual pictures, and label the sample pictures and the virtual pictures respectively to obtain training data and test data, wherein the training data comprises first inkjet code information on existing steel plates, and second inkjet code information on virtual steel plates is constructed according to a preset inkjet code rule, and the test data comprises at least part of the first inkjet code information on the existing steel plates; Input the training data into a steel plate inkjet code information pre-training model to obtain a steel plate inkjet code information detection training model and a steel plate inkjet code information recognition training model, wherein the steel plate inkjet code information detection training model and the steel plate inkjet code information recognition training model constitute a steel plate inkjet code information training model; Use the steel plate inkjet code information detection training model to perform first-time identification on the test data to obtain an identified text area of the test data; Use the steel plate inkjet code information recognition training model to perform second-time identification on the text area of the test data to obtain identified text information of the test data; Compare the identified text area of the test data with the corresponding text area of the existing steel plate, and compare the identified text information of the test data with the corresponding text information of the existing steel plate to obtain an evaluation index; Use the evaluation index to correct the steel plate inkjet code information training model to obtain an optimal steel plate inkjet code information training model, and convert the optimal steel plate inkjet code information training model into a steel plate inkjet code information inference model; and The steel plate code information inference model is used for continuous identification of the to-be-tested steel plate, and continuous multiple frames of to-be-fused code information of the to-be-tested steel plate are obtained. A steel plate code information fusion algorithm is used to fuse the to-be-fused code information of the continuous multiple frames of to-be-tested steel plate, and the fusion code information of the to-be-tested steel plate is obtained. The fusion code information is compared with inventory information or an out-of-warehouse preset rule, information different from the inventory information and information not meeting the out-of-warehouse preset rule are given special marks for warning, the inventory information refers to steel plate code information in an inventory system, and the out-of-warehouse preset rule refers to a value range of steel grade and size preset in an out-of-warehouse link.

Images