Recognition method and recognition system for steel plate identification characters

Abstract

The invention belongs to the technical field of steel plate picture recognition, and discloses a steel plate identification character recognition method and system.According to the scheme, after identification characters on the surface of a steel plate are recognized, identity codes contained in the identification characters are converted into identity code vectors, the identity code vectors are matched with a vector database, and multiple pieces of code vector data with the highest similarity are obtained; and a plurality of identity codes corresponding to the plurality of pieces of code vector data are obtained based on the mapping relation, and the data returned to the mobile terminal comprises the identified identification characters and an identity code recommendation list. Even if the identified identification character is wrong, the worker can quickly find the correct identity code based on the returned identity code recommendation list, so that the operation efficiency can be improved. In addition, a model data set is constructed by intercepting a single character of a steel plate picture in a real scene for drawing, randomly synthesizing the character and a background, repeating the real scene picture and the like, so that the recognition accuracy of a verification set and a real test environment model is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of steel plate picture recognition, in particular to a steel plate identification character recognition method and system. BACKGROUND

[0002] In the sorting and distribution work of steel processing workshops, setting identification characters on the surface of steel plates is a commonly used material tracking and product life cycle management tool. The identification characters are formed on the surface of the steel plate by stamping or code spraying, and by identifying the identification characters in the steel plate picture, the steel plate category, serial number and other information of the steel plate can be obtained.

[0003] In the traditional technology, the identification characters on the surface of the steel plate are recorded by manual on-site recording, and the recorded information is returned to the clerk to be supplemented to the table or information system. This method usually leads to inaccurate and real-time identification characters, affecting the formulation and execution of the production plan.

[0004] In recent years, with the rise of deep neural networks, Ali, Baidu, Tencent and other enterprises have successively launched OCR (Optical Character Recognition) service platforms. By inputting the on-site collected steel plate pictures into the OCR service platform, the identification characters on the surface of the steel plate can be identified, but the accuracy is low. Especially in the case of scratches, debris or dust obstruction, natural environment water immersion and wind erosion, combined with the fixed angle and position of the on-site shooting steel plate picture, the angle of the text line of the steel plate identification characters is completely random, and the overall accuracy of the OCR service platform recognition result will further decrease. SUMMARY

[0005] In order to solve the above problems, the present application provides a steel plate identification character recognition method and system.

[0006] According to the embodiments of the present application, a steel plate identification character recognition method is disclosed, which comprises the following steps:

[0007] Obtaining a steel plate picture collected by a mobile terminal, the steel plate picture containing identification characters formed on the surface of the steel plate, the identification characters containing at least an identity code;

[0008] Identifying the identification characters based on the steel plate picture;

[0009] Converting the identity code contained in the identified identification characters into an identity code vector, and matching the identity code vector with the code vector data in the vector database to obtain a plurality of code vector data with the highest similarity, the vector database storing a mapping relationship between the code vector data and the identity code;

[0010] obtain a plurality of identity codes corresponding to the plurality of pieces of encoding vector data based on the plurality of pieces of encoding vector data and the mapping relationship, and form an identity code recommendation list;

[0011] return the recognized identification character and the identity code recommendation list to the mobile terminal.

[0012] In some embodiments, the matching of the identity code vector with the encoding vector data in the vector database to obtain a plurality of pieces of encoding vector data with the highest similarity includes: matching the identity code vector with the encoding vector data in the vector database to obtain a plurality of pieces of encoding vector data with the highest similarity and a similarity score corresponding to each of the plurality of pieces of encoding vector data, the similarity score being positively correlated with the similarity between the encoding vector data and the identity code vector. The obtaining of a plurality of identity codes corresponding to the plurality of pieces of encoding vector data based on the plurality of pieces of encoding vector data and the mapping relationship, and the forming of an identity code recommendation list includes: obtaining a plurality of identity codes corresponding to the plurality of pieces of encoding vector data based on the plurality of pieces of encoding vector data and the mapping relationship; and forming an identity code recommendation list based on the plurality of identity codes corresponding to the plurality of pieces of encoding vector data and the similarity score corresponding to each of the plurality of pieces of encoding vector data.

[0013] In some embodiments, the forming of an identity code recommendation list based on the plurality of identity codes corresponding to the plurality of pieces of encoding vector data and the similarity score corresponding to each of the plurality of pieces of encoding vector data includes: taking the plurality of identity codes corresponding to the plurality of pieces of encoding vector data as a data column, the similarity score corresponding to each of the plurality of pieces of encoding vector data as another data column, and forming an identity code recommendation list according to the similarity score from high to low.

[0014] In some embodiments, after obtaining the plurality of pieces of encoding vector data with the highest similarity and the similarity score corresponding to each of the plurality of pieces of encoding vector data, the identification method further includes: comparing a maximum value of the similarity score corresponding to each of the plurality of pieces of encoding vector data with a score threshold value; and if the maximum value of the similarity score is below the score threshold value, determining that the identification of the steel plate identification character is incorrect, storing the steel plate picture and the identification result of the identification character as model training data.

[0015] In some embodiments, the identification character includes first character data, second character data, and the identity code, the first character data is used to represent at least one of the flow direction, processing type and item code of the steel plate, and the second character data is used to represent at least one of the material and thickness of the steel plate. After identifying the identification character based on the steel plate picture, the identification method further includes: distinguishing the first character data, the second character data, and the identity code based on a character judgment rule.

[0016] In some embodiments, the identification character based on the steel plate picture includes: inputting the steel plate picture into a trained identification model, and identifying the identification character through the trained identification model.

[0017] In some embodiments, the identification model is PP-OCR.

[0018] In some embodiments, before inputting the steel plate picture into the trained identification model, the identification method further includes constructing a model data set to train the identification model by the following steps: cutting out all single character pictures from the collected multiple steel plate pictures, each of the identification characters includes multiple single characters, and the single character picture contains one single character; drawing a character outline path in the single character picture, filling the character outline path, and setting the background of the single character picture to a transparent color to obtain a first drawn character picture; cutting out multiple different picture backgrounds from the collected multiple steel plate pictures to obtain multiple background pictures; randomly synthesizing based on the first drawn character picture and the multiple background pictures to obtain multiple random background random synthesis pictures; and based on the random synthesis pictures and the multiple steel plate pictures, constructing a training data set and a verification data set of the identification model to train the identification model.

[0019] In some embodiments, based on the random synthesis picture and the multiple steel plate pictures, the training data set and the verification data set of the identification model are constructed, including: rotating a part of the random synthesis picture by a preset angle to obtain a synthesis rotation picture; copying a part of the multiple steel plate pictures a preset number of times to obtain a target number of steel plate pictures; mixing the target number of steel plate pictures with another part of the synthesis rotation picture and the random synthesis picture to obtain a training data set of the identification model, and taking another part of the multiple steel plate pictures as a verification data set of the identification model.

[0020] According to the embodiment of the present application, a steel plate identification character recognition system is also disclosed, which comprises a first server, a second server and a third server. The first server acquires a steel plate picture collected by a mobile terminal, and sends the steel plate picture to the second server. The steel plate picture contains identification characters formed on the surface of the steel plate, and the identification characters contain at least an identity code. The second server receives the steel plate picture, identifies the identification characters based on the steel plate picture, and returns the identified identification character information to the first server. The first server receives the identification character information, converts the identity code contained in the identified identification characters into an identity code vector, and sends the identity code vector to the third server. The third server matches the identity code vector with the code vector data in a vector database, obtains a plurality of code vector data with the highest similarity, and based on the plurality of code vector data and a mapping relationship, obtains a plurality of identity codes corresponding to the plurality of code vector data, forms an identity code recommendation list, and sends the identity code recommendation list to the first server. The vector database stores the mapping relationship between the code vector data and the identity code. The first server returns the identified identification characters and the identity code recommendation list to the mobile terminal.

[0021] The technical scheme provided by the embodiment of the present application has at least the following beneficial effects:

[0022] After the identification characters on the surface of the steel plate are identified based on the steel plate picture collected by the mobile terminal, the identity code contained in the identified identification characters is converted into an identity code vector, the identity code vector is matched with the code vector data in the vector database, a plurality of code vector data with the highest similarity is obtained, the plurality of code vector data and the mapping relationship between the code vector data and the identity code are used to obtain a plurality of identity codes corresponding to the plurality of code vector data, an identity code recommendation list is formed, and then the identified identification characters and the identity code recommendation list are returned to the mobile terminal. Even if the identified identification characters are incorrect, the staff can quickly find the correct identity code based on the returned identity code recommendation list, and the work efficiency can be improved.

[0023] It should be understood that the foregoing general description and the following detailed description are only exemplary and cannot limit the present application. BRIEF DESCRIPTION OF DRAWINGS

[0024] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0025] Figure 1 A flowchart of a steel plate identification character recognition method according to an embodiment of the present application is shown;

[0026] Figure 2 A flow chart of a method for recognizing a steel plate identification character according to another embodiment of the present application is shown;

[0027] Figure 3 A flow chart of constructing a model data set according to an embodiment of the present application is shown;

[0028] Figure 4 A flow chart of Figure 3 A detailed flow chart of step S350 is shown;

[0029] Figure 5 A flow chart of constructing a model data set according to another embodiment of the present application is shown;

[0030] Figure 6 A flow chart of constructing a model data set according to still another embodiment of the present application is shown;

[0031] Figure 7 A hardware architecture diagram of a steel plate identification character recognition system according to an embodiment of the present application is shown;

[0032] Figure 8 An interaction logic diagram of a steel plate identification character recognition system according to an embodiment of the present application is shown.

[0033] The following is a description of the reference signs:

[0034] 700, recognition system; 701, first server; 702, second server; 703, third server; 704, mobile terminal. DETAILED DESCRIPTION

[0035] Example implementations will now be described more fully with reference to the accompanying drawings. Example implementations may, however, be implemented in many different forms and should not be construed as limited to the examples set forth herein; rather, these example implementations are provided so that this disclosure will be thorough and complete, and will fully convey the scope of example implementations to those skilled in the art. Like reference numerals refer to like elements throughout the description.

[0036] Moreover, the terms "first", "second", etc. are used herein only to describe different instances, and do not imply or suggest relative importance or a number of the indicated technical features. Thus, the features defined with "first", "second", etc. can explicitly or implicitly include one or more features.

[0037] Moreover, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the application. One skilled in the relevant art will recognize, however, that the technology can be practiced without one or more of the specific details, or with other methods, components, materials, and so forth. In other instances, well-known structures, devices, implementations, or operations are not shown or described in detail to avoid obscuring aspects of the application.

[0038] The flowcharts shown in the drawings are only illustrative, and do not necessarily include all contents and operations / steps, nor are they necessarily executed in the order described. For example, some operations / steps can be further decomposed, and some operations / steps can be combined or partially combined, so the actual execution order can be changed according to actual conditions.

[0039] Setting identification characters on the surface of a steel plate is a commonly used material tracking and product lifecycle management tool. By identifying the identification characters in the steel plate picture, the steel plate category, serial number, and other information of the steel plate can be obtained. For example, the shipbuilding industry sprays identification characters on the steel plate. The identification characters are generally divided into multiple lines, and each line represents different meanings. For example, the identification characters include the following three lines of characters:

[0040] M12V1-XG-H614: flow direction- processing type- project code

[0041] AH36-24.00: material- plate thickness

[0042] R16P-HG46A-K1: identity code (segment number- group number- steel plate category + serial number)

[0043] In the related art, the identification character recognition method in the steel plate picture has the following problems: 1. The identification characters obtained by manually recording the identification characters on the surface of the steel plate at the steel plate site are not accurate and real-time; 2. The steel plate picture collected on site is input into an OCR service platform, and the identification characters on the surface of the steel plate are recognized by the OCR service platform, which has a low accuracy.

[0044] To solve the above problems, the applicant collected 2182 real scene steel plate pictures, labeled them using the semi-automatic graphic labeling tool PPOCRLabel, and fine-tuned the pre-trained model based on the PP-OCRv4 recognition module. After testing, the accuracy of the fine-tuned recognition module in the validation set reached about 96%. However, in actual use, it was found that in the real scene steel plate pictures collected on site, the segmented part of the project code and the identity code was relatively concentrated in one project code and three segment numbers. In the actual production process, there are dozens of project codes, and the segment number of a single project reaches several hundred, causing the two parts to be identified as the project code and segment number in the sample, although other project codes and segment numbers are also identified.

[0045] To solve the above problems, the applicant collected 2182 real scene steel plate pictures, labeled them using the semi-automatic graphic labeling tool PPOCRLabel, and fine-tuned the pre-trained model based on the PP-OCRv4 recognition module. After testing, the accuracy of the fine-tuned recognition module in the validation set reached about 96%. However, in actual use, it was found that in the real scene steel plate pictures collected on site, the segmented part of the project code and the identity code was relatively concentrated in one project code and three segment numbers. In the actual production process, there are dozens of project codes, and the segment number of a single project reaches several hundred, causing the two parts to be identified as the project code and segment number in the sample, although other project codes and segment numbers are also identified.

[0046] After improvement, the above technical solutions still have the following problems: 1. The identification characters of the steel plate are affected by factors such as light and shadow, obstruction, blur, scratches, and random angles, which makes the accuracy of the recognition model unable to reach 100%. When the identification characters are recognized incorrectly, the user can only manually input to solve the problem, which reduces the work efficiency; 2. The amount of real scene steel plate pictures is small, the data set text content is repetitive, the existing font library cannot find the same font as the identification characters of the real scene, and the amount of synthesized pictures is much larger than the amount of real steel plate pictures. The background of the synthesized pictures is too simple, which easily causes the recognition model to overfit, resulting in a decrease in recognition accuracy.

[0047] To solve the above technical problem 1, the application provides a steel plate identification character recognition method and a recognition system with higher recognition accuracy and capable of helping users quickly find correct identification characters when identification character recognition errors occur. After the identification characters on the surface of a steel plate are recognized based on a steel plate picture collected by a mobile terminal, the identity code contained in the recognized identification characters is converted into an identity code vector, the identity code vector is matched with code vector data in a vector database, a plurality of code vector data with the highest similarity is obtained, the plurality of code vector data and the mapping relationship between the code vector data and the identity code are used to obtain a plurality of identity codes corresponding to the plurality of code vector data, and an identity code recommendation list is formed, after which the recognized identification characters and the identity code recommendation list are returned to the mobile terminal.

[0048] The steel plate identification character recognition method provided by the application will be described in detail below in combination with a specific embodiment.

[0049] Figure 1 A flowchart of the steel plate identification character recognition method according to an embodiment of the application is shown in FIG. 1. Figure 1 As shown in FIG. 1, the steel plate identification character recognition method includes at least a steel plate picture acquisition step, an identification character recognition step, a vector matching step, a recommendation list generation step, and a result feedback step, which correspond to steps S110, S120, S130, S140, and S150 respectively, and will be described in detail as follows.

[0050] In step S110, a steel plate picture collected by a mobile terminal is acquired. Then, step S120 is performed.

[0051] The steel plate picture contains identification characters formed on the surface of the steel plate. The identification characters can be formed on the surface of the steel plate by means of jet printing, embossing, or the like.

[0052] The identification characters contain at least an identity code.

[0053] In some embodiments, the identification characters can contain only the identity code.

[0054] In some embodiments, the identification characters contain not only the identity code but also other characters. Exemplarily, the identification characters include first character data, second character data, and third character data, the first character data is used to represent at least one of the flow direction, the processing type, and the project code of the steel plate, the second character data is used to represent at least one of the material and the thickness of the steel plate, and the third character data is the identity code. In these embodiments, the identification characters contain not only the identity code but also more other character data, which can carry more information of the steel plate for easy calling during production.

[0055] Exemplarily, the first character data, the second character data, and the identity code are located in different text lines respectively, the first character data includes characters for representing the flow direction, the processing type, and the item code of the steel plate, the second character data includes characters for representing the material and the thickness of the steel plate, and the identity code includes characters for representing the segment number, the group number, the steel plate category, and the serial number. In an embodiment, the identification characters include:

[0056] M12V1-XG-H614: flow direction-processing type-item code;

[0057] AH36-24.00: material-plate thickness

[0058] R16P-HG46A-K1: identity code (segment number-group number-steel plate category+serial number)

[0059] In step S120, the identification characters are recognized based on the steel plate picture. Then, step S130 is performed.

[0060] In some embodiments, the identification characters are recognized based on the steel plate picture, specifically, the steel plate picture is input into a trained recognition model, and the identification characters are recognized by the trained recognition model.

[0061] In some embodiments, the recognition model is PP-OCRv4. PP-OCR adopts a lightweight design and can recognize more than 3 pictures per second without using a GPU server. When training, the accuracy rate of the validation set reaches more than 97% after traversing the data set once. In actual use, the identification character recognition accuracy rate can also reach more than 95%, and the overall recognition accuracy rate is relatively high.

[0062] In some embodiments in which the identification characters include not only the identity code but also other characters, after the identification characters are recognized based on the steel plate picture, a character distinguishing step needs to be performed first to recognize the identity code included in the identification characters, and then a vector matching step is further performed, which is step S130 described below.

[0063] Specifically, the character distinguishing step includes: distinguishing the first character data, the second character data, and the identity code based on a character judgment rule.

[0064] In an example embodiment, in combination with the recognition result in the real scene, the text accuracy rate of the "material-plate thickness" meaning is high, close to 100%, and ends with.00, which is determined as the second character data; the project code accuracy rate in the text of the "flow direction-processing type-project code" meaning is high, and whether the character after the last "-" is the project code is determined, and if so, it is determined as the first character data; the segment number accuracy rate in the text of the "identity code" meaning is the highest, and whether the character before the first character "-" is a segment number is determined, and if so, it is determined as the identity code, or if it is not determined as the first character data and the second character data, it is also determined as the identity code. Exemplarily, the character determination rule is shown in the following table:

[0065]

[0066] In step S130, the identity code contained in the recognized identification character is converted into an identity code vector, and the identity code vector is matched with the code vector data in the vector database to obtain multiple pieces of code vector data with the highest similarity. Then, step S140 is performed.

[0067] The vector database stores the mapping relationship between the code vector data and the identity code. The mapping relationship stored in the vector database can be obtained by storing the newly generated identity code and the converted identity code vector into the vector database at a certain time.

[0068] The conversion of the character into the vector and the similarity matching of the vector can be realized by the existing technology, and therefore will not be described here.

[0069] In some embodiments, obtaining multiple pieces of code vector data with the highest similarity is to obtain 10 pieces of code vector data with the highest similarity. Of course, the number of code vector data with the highest similarity obtained is not limited to 10, and in other embodiments, the number of code vector data with the highest similarity obtained can be, for example, 7, 8, 9, etc.

[0070] In step S140, based on the multiple pieces of code vector data and the mapping relationship, multiple pieces of identity code corresponding to the multiple pieces of code vector data are obtained, and an identity code recommendation list is formed. Then, step S150 is performed.

[0071] In some embodiments, based on the multiple pieces of code vector data and the mapping relationship, the multiple pieces of identity code corresponding to the multiple pieces of code vector data are obtained by searching the mapping relationship between the multiple pieces of code vector data and the identity code to obtain the multiple pieces of identity code corresponding to the multiple pieces of code vector data.

[0072] In step S150, the recognized identification character and the identity code recommendation list are returned to the mobile terminal.

[0073] At this time, the mobile terminal can display the recognized identification character and the identity code recommendation list. The user can further confirm whether the identity code in the recognized identification character is correct, and if correct, click to confirm; if incorrect, can select the correct identity code from the identity code recommendation list for confirmation. Even if there is no correct identity code in the identity code recommendation list, the user can also choose to manually input the correct identity code and then click to confirm. After receiving the correct identity code, the background can further execute subsequent instructions, such as updating the position information of the steel plate, so as to achieve the final sorting and distribution purposes.

[0074] In some embodiments, in the result feedback step, in addition to returning the recognized identification character and the identity code recommendation list to the mobile terminal, the steel plate picture obtained in the steel plate picture obtaining step is also returned to the mobile terminal, so as to facilitate the user to confirm whether the identity code in the recognized identification character is correct and manually input the correct identity code according to the steel plate picture when there is no correct identity code in the identity code recommendation list.

[0075] Figure 2 The flow chart of the steel plate identification character recognition method of another embodiment of the present application is shown, and the steel plate identification character recognition method comprises at least the following steps S210-S250, which are described in detail as follows: Figure 2 The steel plate identification character recognition method comprises at least the following steps S210-S250, which are described in detail as follows:

[0076] In step S210, the steel plate picture collected by the mobile terminal is obtained. Then, step S220 is executed.

[0077] In step S220, the steel plate picture is input into the trained recognition model, and the identification character is recognized by the trained recognition model. Then, step S230 is executed.

[0078] In step S230, the identity code contained in the recognized identification character is converted into an identity code vector, and the identity code vector is matched with the code vector data in the vector database to obtain the multiple pieces of code vector data with the highest similarity and the respective similarity scores of the multiple pieces of code vector data. Then, step S240 is executed.

[0079] The similarity score is positively correlated with the similarity of the code vector data and the identity code vector.

[0080] In some embodiments, the similarity score is represented by a value between 0 and 1, and the higher the similarity of the code vector data and the identity code vector, the closer the similarity score is to 1.

[0081] At step S240, based on the plurality of pieces of encoding vector data and the mapping relationship, a plurality of pieces of identity codes corresponding to the plurality of pieces of encoding vector data are obtained, and based on the plurality of pieces of identity codes corresponding to the plurality of pieces of encoding vector data and the similarity scores corresponding to the plurality of pieces of encoding vector data respectively, an identity code recommendation list is formed. Then, step S250 is executed.

[0082] In some embodiments, forming the identity code recommendation list based on the plurality of pieces of identity codes corresponding to the plurality of pieces of encoding vector data and the similarity scores corresponding to the plurality of pieces of encoding vector data respectively includes: taking the plurality of pieces of identity codes corresponding to the plurality of pieces of encoding vector data as a data column and the similarity scores corresponding to the plurality of pieces of encoding vector data as another data column, and forming the identity code recommendation list according to the similarity scores from high to low.

[0083] Taking the plurality of pieces of identity codes corresponding to the plurality of pieces of encoding vector data as a data column and the similarity scores corresponding to the plurality of pieces of encoding vector data as another data column, and forming the identity code recommendation list according to the similarity scores from high to low, facilitates the user to select the correct identity code from the identity code recommendation list when the identity code in the recognized identification character is incorrect.

[0084] At step S250, the recognized identification character and the identity code recommendation list are returned to the mobile terminal.

[0085] In Figure 2 In the embodiment shown, the identity code recommendation list contains the similarity scores of the encoding vector data corresponding to each recommended identity code and the identity code vector converted from the identity code contained in the recognized identification character, which facilitates the user to quickly find the correct identity code from the identity code recommendation list when the identity code in the recognized identification character is incorrect.

[0086] In some embodiments, after obtaining the plurality of pieces of encoding vector data with the highest similarity and the similarity scores corresponding to the plurality of pieces of encoding vector data respectively, the recognition method further includes a score comparison step, which includes: comparing the maximum value of the similarity scores corresponding to the plurality of pieces of encoding vector data with a score threshold value, and if the maximum value is below the score threshold value, determining that the steel plate identification character recognition is incorrect, and further executing a data storage step. The data storage step includes: storing the steel plate picture and the identification character recognition result as model training data.

[0087] The score threshold value can be a value set flexibly according to experience, for example, 0.99, etc.

[0088] The vector database stores all encoding vector data. When the identified identification character is accurate, the vector database is searched based on the identified identification character, and a high similarity encoding vector data can be found. If the identified identification character is incorrect, the vector database does not store corresponding data, and the vector database cannot be searched based on the identified identification character to find a high similarity encoding vector data, and the similarity score is low. By comparing the maximum value of the similarity scores corresponding to the plurality of encoding vector data with the score threshold, it can be determined whether the identified identification character is accurate. When it is determined that the identification character of the steel plate is incorrect, the steel plate picture and the identification character recognition result are further stored, which can be used for retraining of the identification model to optimize the identification model, thereby improving the recognition accuracy.

[0089] To solve the above technical problem 2, the application also provides a method for constructing an identification model data set.

[0090] Before the steel plate picture is input into the trained identification model, the identification model needs to be trained to obtain the trained identification model. Training the identification model requires first constructing a model data set. In some embodiments, as shown in Figure 3 The model data set is constructed by at least the following steps S310-S350, which are described in detail as follows:

[0091] In step S310, all single character pictures are cut from the collected plurality of steel plate pictures. Each identification character includes a plurality of single characters, and a single character picture contains one single character. Then, step S320 is performed.

[0092] In step S320, the character contour path in the single character picture is drawn, and the character contour path is filled and the background of the single character picture is set to a transparent color to obtain a first drawn character picture. Then, step S330 is performed.

[0093] That is, the first drawn character picture is a picture with a transparent background and a single character drawn based on the single character picture.

[0094] In step S330, a plurality of different picture backgrounds are cut from the collected plurality of steel plate pictures to obtain a plurality of background pictures. Then, step S340 is performed.

[0095] In step S340, the first drawn character picture and the plurality of background pictures are randomly synthesized to obtain a plurality of randomly synthesized pictures with random backgrounds. Then, step S350 is performed.

[0096] In step S350, based on the randomly synthesized picture and the plurality of steel plate pictures, a training data set and a verification data set of the identification model are constructed to train the identification model.

[0097] In Figure 3 In the embodiment shown, since the existing font library cannot find a font as the same as the identification character of the real scene, first, the single character of the steel plate picture in the real scene is intercepted, then the same character is drawn, and then the drawn character picture and the background picture are randomly synthesized to obtain a random synthesis picture with a random background, which does not distort the character font, color, etc., and enriches the picture background and fits the real scene. The training data set and the verification data set constructed based on this help to train a recognition model with higher accuracy.

[0098] In some embodiments, as Figure 4 shown, based on the random synthesis picture and the plurality of steel plate pictures, the training data set and the verification data set of the recognition model are constructed, at least including the following steps S410-S430, which are described in detail as follows:

[0099] In step S410, a part of the random synthesis pictures is rotated by a preset angle to obtain a synthesis rotation picture. Then, step S420 is executed.

[0100] The preset angle can be set according to experience, for example, 180 degrees, etc.

[0101] In step S420, a part of the plurality of steel plate pictures is copied a preset number of times to obtain a target number of steel plate pictures. Then, step S430 is executed.

[0102] The preset number can be set according to experience, for example, 563 times, etc.

[0103] In step S430, the target number of steel plate pictures is mixed with the synthesis rotation picture and another part of the random synthesis picture to obtain the training data set of the recognition model, and another part of the plurality of steel plate pictures is used as the verification data set of the recognition model.

[0104] In the embodiment shown in Figure 4 , the picture angle of the real scene is simulated, a certain number of random synthesis pictures are obtained by rotating a certain angle to obtain synthesis rotation pictures, and a small number of real scene pictures are sampled in a repeated manner. The proportion of the real scene pictures is sampled to be multiple times of the synthesis data, so that the recognition model has a certain generalization ability after training, which greatly improves the recognition accuracy of the verification set and the real test environment.

[0105] In view of the fact that the existing font library cannot find a font as same as the identification character of the real scene, first, the single character picture of the steel plate picture in the real scene is intercepted, and then the same character is drawn, and then the drawn character picture and the background picture are randomly synthesized to obtain a random synthesis picture with a random background, which does not distort the character font, color, etc., and enriches the picture background and fits the real scene. The training data set and the verification data set constructed based on this are helpful to train a recognition model with higher accuracy.

[0106] In some embodiments, after obtaining the first drawn character picture, all the drawn single characters in all the first drawn character pictures are arranged in the same picture to obtain a second drawn character picture, and then the first drawn character picture, the second drawn character picture and the plurality of background pictures are randomly synthesized to obtain a plurality of random synthesis pictures with random backgrounds.

[0107] In some embodiments, as shown in Figure 5 The model data set is constructed, and the steps S510-S560 are included, which are described in detail as follows.

[0108] In step S510, all single character pictures are intercepted from the plurality of steel plate pictures collected. Each identification character includes a plurality of single characters, and a single character picture contains one single character. Then, step S520 is performed.

[0109] In step S520, the character outline path in the single character picture is drawn, and the character outline path is filled and the background of the single character picture is set to a transparent color to obtain a first drawn character picture. Then, step S530 is performed.

[0110] In step S530, all the drawn single characters in all the first drawn character pictures are arranged in the same picture to obtain a second drawn character picture. Then, step S540 is performed.

[0111] In step S540, a plurality of different picture backgrounds are intercepted from the plurality of steel plate pictures collected to obtain a plurality of background pictures. Then, step S550 is performed.

[0112] In step S550, the second drawn character picture and the plurality of background pictures are randomly synthesized to obtain a plurality of random synthesis pictures with random backgrounds. Then, step S560 is performed.

[0113] In step S560, based on the random synthesis picture and the plurality of steel plate pictures, a training data set and a verification data set of the recognition model are constructed to train the recognition model.

[0114] In some embodiments, as shown in Figure 6 The model data set is constructed, and the steps S610-S690 are included, which are described in detail as follows.

[0115] In step S610, the first character data and the identity code data are acquired from the character database storing the identification characters, and all the acquired first character data and the identity code data are put into a first file. Then, step S620 is performed.

[0116] The first character data includes the item code, the processing type and the flow direction field, which can be stored separately. The first character data can be acquired from the character database storing the identification characters, which can be the data of all the item codes, all the processing types and all the flow direction fields respectively exported from the respective databases, the repeated data removed, the temporary irregular data deleted, and combined into one line of data according to the rule of "flow direction-processing type-item code" to obtain the first character data. Of course, the item code, the processing type and the flow direction field can be stored together as the first character data, and the first character data can be acquired from the character database storing the identification characters, which can be the first character data exported from the character database storing the identification characters.

[0117] The identity code data can be acquired from the character database storing the identification characters, which can be all the identity codes of several typical items exported from the character database storing the identification characters, the repeated and irregular data removed.

[0118] The first character data and the identity code data can be put into a.txt format file, one line of data for one row, to obtain 285,000 lines of data.

[0119] In step S620, all the single character pictures are cut out from the collected multiple steel plate pictures. Then, step S630 is performed.

[0120] In step S630, the character contour path in the single character picture is drawn, and the character contour path is filled and the background of the single character picture is set to be transparent, to obtain a first drawn character picture. Then, step S640 is performed.

[0121] In an embodiment, in step S630, the character contour path of each single character picture is drawn by using the phtoshop tool, and the character contour path is filled with black and the background is filled with transparent color, and the single character picture is stored separately in a png format.

[0122] In step S640, the drawn single characters in all the first drawn character pictures are arranged in the same picture to obtain a second drawn character picture. Then, step S650 is performed.

[0123] The second drawn character picture can be a png format picture.

[0124] At step S650, the first drawn character picture and the second drawn character picture are processed by using a character conversion tool to obtain a second file. Then, step S660 is performed.

[0125] The second file can be a.tiff format file.

[0126] At step S660, different picture backgrounds are cut from the collected multiple steel plate pictures to obtain multiple background pictures. Then, step S670 is performed.

[0127] The number of the cut picture backgrounds is, for example, 30, and the size of the obtained background pictures can be determined based on the size of a line of characters.

[0128] At step S670, random synthesis is performed based on the first file, the second file and the multiple background pictures to obtain random synthesis pictures of multiple random backgrounds. Then, step S680 is performed.

[0129] In an embodiment, at step S670, a data synthesis tool is used to take the first file, the second file and the 30 background pictures as inputs to obtain 285,000 random synthesis pictures of random backgrounds. Parameters are adjusted to set the font size to 48 pixels, the character spacing to 2 pixels, and the distance between the characters of the picture and the four edges of the picture to 2 pixels, so that the picture proportion and style are consistent with the steel plate pictures of the real scene.

[0130] At step S680, 38% of the random synthesis pictures are rotated by 180 degrees to obtain 285,000 synthesis rotation pictures. Then, step S690 is performed.

[0131] At step S690, 5000 pictures are randomly extracted from 6000 steel plate pictures of the real scene 563 times to obtain 28,150,000 steel plate pictures, and the 28,150,000 steel plate pictures are mixed with the 285,000 synthesis rotation pictures to obtain 3,100,000 pictures as a training data set of the recognition model. The remaining 1000 steel plate pictures of the real scene are used as a validation data set of the recognition model.

[0132] In Figure 6In the embodiment shown, since the existing font library cannot find a font as same as the identification character of the real scene, the single character of the steel plate picture in the real scene is intercepted, and then the same character is drawn, and then the drawn character picture and the background picture are randomly synthesized to obtain a random synthesis picture with a random background, which does not distort the character font, color, etc., and at the same time simulates the picture angle of the real scene. A certain number of random synthesis pictures are obtained by rotating 180 degrees to obtain synthesis rotation pictures, and a small number of real scene pictures are sampled in a repeated manner, and the proportion of real scene pictures is sampled to about 5 to 10 times of the synthesis data, so that the recognition model has a certain generalization ability after training, and the recognition accuracy of the verification set and the real test environment is greatly improved.

[0133] Next, the steel plate identification character recognition system provided by the present application is described in detail.

[0134] Figure 7 The hardware architecture diagram of the steel plate identification character recognition system of an embodiment of the present application is shown.

[0135] As shown in Figure 7 The steel plate identification character recognition system 700 includes a first server 701, a second server 702, and a third server 703, and the first server 701 is in communication connection with the second server 702 and the third server 703. The first server 701 can also be connected to a mobile terminal 704 through 5G or WIFI and the like.

[0136] Among them, the first server 701 acquires the steel plate picture collected by the mobile terminal 704, and sends the steel plate picture to the second server 702. The steel plate picture contains identification characters formed on the surface of the steel plate, and the identification characters at least contain identity codes.

[0137] The second server 702 receives the steel plate picture, identifies the identification characters based on the steel plate picture, and returns the identified identification character information to the first server 701.

[0138] The first server 701 receives the identification character information identified by the second server 702, converts the identity code contained in the identified identification character into an identity code vector, and sends it to the third server 703.

[0139] The third server 703 matches the identity code vector with the code vector data in the vector database, obtains multiple pieces of code vector data with the highest similarity, and based on the multiple pieces of code vector data and the mapping relationship, obtains multiple pieces of identity code corresponding to the multiple pieces of code vector data, and forms an identity code recommendation list and sends it to the first server 701. The mapping relationship between the code vector data and the identity code is stored in the vector database.

[0140] The first server 701 returns the recognized identification character and the identity code recommendation list to the mobile terminal 704.

[0141] In some embodiments, the steel plate picture is a binary picture file. After receiving the steel plate picture collected by the mobile terminal 704, the first server 701 converts it into Base64 encoding format, and then calls the interface of the second server 702. The second server 702 receives the steel plate picture in Base64 encoding format, recognizes the identification character through the recognition model therein, and outputs the text result in Json format to the first server 701. The text result in Json format includes three rows of data, which are the first character data, the second character data, and the identity code, respectively. After receiving the text result in Json format, the first server 701 determines which row of the three rows of data is the identity code based on the character judgment rule, and then converts it into an identity code vector through the lightweight embedding model deployed therein, and requests the ANN search function of the vector database in the third server. The third server queries the identity code and the similarity score corresponding to the 10 pieces of code vector data with the highest matching degree, and forms an identity code recommendation list and returns it to the first server 701. The first server 701 returns the recognized identification character and the identity code recommendation list to the mobile terminal 704. The specific interaction logic is shown in Figure 8 .

[0142] It should be noted that the specific working process of each server can refer to the description of the corresponding steps in the aforementioned steel plate identification character recognition method, which will not be repeated here.

[0143] In summary, the recognition system of the present application solves the recognition problem of steel plate identification characters caused by factors such as light and shadow, occlusion, blur, scratches, and random angle in complex industrial environments through multi-level server cooperative processing, combined with related methods such as synthetic picture training recognition model, rule judgment, similarity retrieval recommendation, and retraining, and improves the efficiency and accuracy of part sorting and distribution.

[0144] Other embodiments of the application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. It is intended that the application cover any variations, uses, or adaptations of the application following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice in the art to which the application pertains or can relate. The specification and examples are to be regarded as exemplary only, and the true scope and spirit of the application are indicated by the appended claims.

Claims

1. A method for recognizing steel plate identification characters, characterized in that: The identification method comprises: Acquire a steel plate image captured by a mobile terminal, wherein the steel plate image includes identification characters formed on a surface of the steel plate, and the identification characters include at least an identity code; Recognizing the identification character based on the steel plate image; Converting the identity code contained in the recognized identification character into an identity code vector, and matching the identity code vector with code vector data in a vector database to obtain a plurality of code vector data with the highest similarity, wherein the vector database stores a mapping relationship between the code vector data and the identity code; Based on the plurality of pieces of encoding vector data and the mapping relationship, obtaining a plurality of identity codes corresponding to the plurality of pieces of encoding vector data, and forming an identity code recommendation list; The recognized identification character and the identity code recommendation list are returned to the mobile terminal.

2. The identification method according to claim 1, characterized in that The matching of the identity encoding vector with the encoding vector data in the vector database to obtain a plurality of encoding vector data with the highest similarity includes: Matching the identity encoding vector with encoding vector data in a vector database to obtain a plurality of encoding vector data with the highest similarity and similarity scores corresponding to each of the plurality of encoding vector data, wherein the similarity scores are positively correlated with the similarity between the encoding vector data and the identity encoding vector; The obtaining, based on the plurality of pieces of encoding vector data and the mapping relationship, a plurality of identity codes corresponding to the plurality of pieces of encoding vector data, and forming an identity code recommendation list includes: Based on the multiple pieces of coded vector data and the mapping relationship, obtaining multiple identity codes corresponding to the multiple pieces of coded vector data; Based on the multiple identity codes corresponding to the multiple pieces of encoding vector data and the similarity scores corresponding to the multiple pieces of encoding vector data, an identity code recommendation list is formed.

3. The identification method according to claim 2, characterized in that The forming of the identity code recommendation list based on the plurality of identity codes corresponding to the plurality of code vector data and the similarity scores corresponding to the plurality of code vector data includes: The multiple identity codes corresponding to the multiple encoding vector data are used as a data column, and the similarity scores corresponding to the multiple encoding vector data are used as another data column. An identity code recommendation list is formed from high to low according to the similarity scores.

4. The identification method according to claim 2, characterized in that After obtaining the plurality of pieces of encoding vector data with the highest similarity and the similarity scores corresponding to the plurality of pieces of encoding vector data, the recognition method further includes: Comparing the maximum score among the similarity scores corresponding to the plurality of pieces of encoding vector data with a score threshold; If the maximum score is below the score threshold, it is determined that the steel plate identification character recognition is incorrect, and the steel plate image and identification character recognition result are stored as model training data.

5. The identification method according to claim 1, characterized in that The identification character includes first character data, second character data, and the identity code, wherein the first character data is used to indicate at least one of the flow direction, processing type, and project code of the steel plate, and the second character data is used to indicate at least one of the material and thickness of the steel plate; After the identification characters are identified based on the steel plate image, the recognition method further includes: Based on a character judgment rule, the first character data, the second character data and the identity code are distinguished.

6. The identification method according to any one of claims 1 to 5, characterized in that: The identifying character based on the steel plate image includes: The steel plate image is input into a trained recognition model, and the identification character is recognized by the trained recognition model.

7. The identification method according to claim 6, characterized in that: The recognition model is PP-OCR.

8. The identification method according to claim 6, characterized in that: Before inputting the steel plate image into the trained recognition model, the recognition method further comprises constructing a model data set to train the recognition model through the following steps: Extracting all single-character images from the collected multiple steel plate images, each of the identification characters includes multiple single characters, and the single-character images include one of the single characters; Draw a character outline path in the single-character picture, fill the character outline path, and set the background of the single-character picture to a transparent color to obtain a first drawn character picture; Extracting multiple different picture backgrounds from the collected multiple steel plate pictures to obtain multiple background pictures; Performing random synthesis based on the first drawn character image and the multiple background images to obtain multiple randomly synthesized images of random backgrounds; Based on the randomly synthesized pictures and the multiple steel plate pictures, a training data set and a verification data set of a recognition model are constructed to train the recognition model.

9. The identification method according to claim 8, characterized in that: The step of constructing a training data set and a verification data set for a recognition model based on the random synthesized image and the plurality of steel plate images includes: Rotating a portion of the randomly synthesized image by a preset angle to obtain a synthesized rotated image; Copying a portion of the plurality of steel plate images a preset number of times to obtain a target number of steel plate images; The target number of steel plate images are mixed with the synthetic rotated image and another part of the randomly synthesized image to obtain a training data set for the recognition model, and another part of the multiple steel plate images is used as a verification data set for the recognition model.

10. A system for recognizing steel plate identification characters, characterized in that: The identification system includes a first server, a second server, and a third server, wherein: The first server acquires a steel plate image captured by the mobile terminal and sends the steel plate image to the second server, wherein the steel plate image includes identification characters formed on the surface of the steel plate, and the identification characters at least include an identity code; The second server receives the steel plate image, identifies the identification character based on the steel plate image, and returns the identified identification character information to the first server; The first server receives the identification character information, converts the identity code contained in the identified identification character into an identity code vector, and sends the identity code vector to the third server; The third server matches the identity encoding vector with the encoding vector data in the vector database to obtain multiple pieces of encoding vector data with the highest similarity, and based on the multiple pieces of encoding vector data and the mapping relationship, obtains multiple identity codes corresponding to the multiple pieces of encoding vector data, and forms an identity code recommendation list and sends it to the first server, wherein the mapping relationship between the encoding vector data and the identity codes is stored in the vector database; The first server returns the recognized identification character and the identity code recommendation list to the mobile terminal.

Images