A machine vision-based steel billet character recognition detection system and method for heating furnace

Abstract

The present invention discloses a machine vision-based billet character recognition detection system and method for a heating furnace. First, a billet cross-section recognition module is used to recognize the billet cross-section on a roller, and the actual angle between the minimum circumscribed rectangle of the character area with the largest area in the billet cross-section and the horizontal plane is determined. The billet cross-section image is rotated according to the actual angle, and its front side faces upward; then, the billet character recognition module is used to determine the characters in the billet cross-section. In this process, the characters are filtered based on the confidence of the characters, and the character area coordinates are used for line judgment to obtain spliced ​​characters; finally, the spliced ​​characters are scored, and based on the scoring results, the final recognition result of the characters in the billet cross-section is output. The method of the present invention can accurately identify billet characters.

Description

Technical Field

[0001] The present invention relates to the technical field of machine vision, and in particular to a machine vision-based steel billet character recognition detection system and method for entering a heating furnace. Background Art

[0002] At present, character recognition technology based on machine vision has been widely used in all aspects of life, such as license plate recognition, document recognition, financial bill recognition, office automation, etc.

[0003] However, due to the uneven surface of the billet and the melting and deformation of the billet number coating at high temperatures, it is difficult for machine vision-based character recognition to accurately identify the material number on the billet surface. At the same time, due to the uncontrollability of the billet transportation and lifting process, as well as the existence of machine spraying angles and handwriting angles, the orientation of the characters at both ends of the billet is relatively random. The current character recognition technology is difficult to accurately recognize inverted or tilted characters. In addition, the current billet character recognition on the market usually requires access to automated control signals to trigger the camera to take pictures and fill in the light. The cost and modification requirements are relatively high, and it is limited by the automated control signal. When the signal is wrong, the billet character recognition is inaccurate. Summary of the invention

[0004] In view of the deficiencies in the prior art, the present invention provides a system and method for detecting characters of steel billets entering a heating furnace based on machine vision.

[0005] The present invention achieves the above technical objectives through the following technical means.

[0006] A method for character recognition and detection of steel billets entering a heating furnace based on machine vision:

[0007] Obtain the roller image in actual production, and use the trained billet cross-section recognition model to determine whether there is a billet cross-section on the roller. When there is a billet cross-section on the roller, obtain the minimum circumscribed rectangle of the character area with the largest area in the billet cross-section, and determine the actual angle between the minimum circumscribed rectangle and the horizontal plane. Rotate the billet cross-section image according to the actual angle so that the front side faces upward.

[0008] The trained billet character recognition model is used to recognize characters on the billet cross-section image to obtain recognition result information of all characters on the entire billet cross-section, wherein the recognition result information includes the coordinates of the character area and the confidence of the character; the confidence of all characters is sorted from high to low, and if the intersection and union ratio of a certain character area with other character areas is greater than a set value, the other character areas are filtered, and a certain character is moved into the reserved character list A; in the reserved character list A, the characters are sorted from large to small according to the area, and if the area inclusion result of a certain character and other character areas is greater than a set value, the other character areas are filtered, and a certain character is moved into the reserved character list A. Filter, a certain character is moved into the reserved character list B; the distance between the center coordinates of any two character areas is calculated from the center coordinates of all character areas in the reserved character list B, and if the distance between the center coordinates of a certain character area and the center coordinates of other character areas is greater than the set value, a certain character area is filtered out; the character line is judged according to the y coordinate of the center point of the character area to obtain multiple lines of character results; for the same line of characters, the x coordinates of the center points of the character areas are sorted in ascending order, and the corresponding characters are spliced ​​from left to right and from top to bottom to obtain a string of spliced ​​recognition results;

[0009] For the spliced ​​character results, the scoring result is determined by using the scoring algorithm and the supplementary recognition algorithm;

[0010] Based on the scoring results, the final character recognition result of the billet cross section is output: when the same character is met for the set number of times and the score is up to standard, it is determined that the character has been accurately recognized, and the final character recognition result of the billet cross section is output; if the same character is not met for the set number of times and the score is up to standard, then when it is detected that the billet leaves the recognition area, the result with the highest score is output. If there are multiple highest scoring results, the result with the most occurrences of a certain character is output.

[0011] Furthermore, the actual angle between the minimum circumscribed rectangle and the horizontal plane is determined by:

[0012] The cross-section image of the steel billet is rotated at a set angle, and the minimum circumscribed rectangle of the character area is re-determined, and at the same time, the angle between the minimum circumscribed rectangle of the character area with the largest area and the horizontal plane is determined;

[0013] Each time the character is rotated, the angle between the smallest circumscribed rectangle of the character area with the largest area and the horizontal plane is subtracted from the corresponding rotation angle, and the result is used to form an angle list.

[0014] Divide the angle list into n intervals, take the average of all angles in the interval with the largest number of angles, and use it as the actual angle between the minimum circumscribed rectangle of the character area with the largest area and the horizontal plane.

[0015] Furthermore, the rotated billet cross-section has four orientations: up, down, left, and right. The orientation is judged by machine learning image classification. The billet cross-section images in the three orientations of down, left, and right are rotated, and finally a billet cross-section image with the front side facing up is obtained.

[0016] Furthermore, if the intersection and union ratio of the area of ​​a certain character area with other character areas is greater than a set value, the character area is filtered, specifically: the confidence of all characters is sorted from high to low, and the corresponding characters are recorded as Y1, Y2...Yi...Yn in turn, and the area A1, A2...Ai...An of each character area is calculated. The intersection and union ratio (IOU) of the areas of Y1 and Y2...Yi...Yn is calculated, that is, IOU = (A1∩Ai) / (A1∪Ai). If IOU>0.5, the character area corresponding to Yi is filtered out and Y1 is moved into the reserved character list A. The characters that are not filtered out are re-recorded as z1, z2...zi...zm. If m=1 at this time, the character is also moved into the reserved character list A. Otherwise, the above IOU filtering operation is repeated for z1, z2...zi...zm.

[0017] Furthermore, if the area inclusion result of a certain character and other character areas is greater than the set value, other character areas are filtered, specifically: the characters in the reserved character list A are sorted from large to small in terms of area, and the corresponding characters are recorded as x1, x2...xi...xk, and the corresponding areas are A1, A2...Ai...Ak. The area inclusion calculation of x1 and x2...xi...xk is performed, and the calculation formula is c= (A1∩Ai) / Ai. If c>0.9, the character area corresponding to xi is filtered out and x1 is moved into the reserved character list B. The characters that are not filtered out are re-recorded as Z1, Z2...Zi...Zm. If m=1 at this time, the character is also moved into the reserved character list B. Otherwise, the filtering operation of the above-mentioned area inclusion calculation is repeated for Z1, Z2...Zi...Zm.

[0018] Furthermore, if the distance between the center coordinates of a certain character area and the center coordinates of other character areas is greater than the set value, then the character area is filtered out. Specifically, the minimum distance Lmin of the center coordinates of all character areas is determined. If the distance between the center coordinates of a certain character area and the center coordinates of other character areas is greater than 3 times of Lmin, then the character area is filtered out.

[0019] Furthermore, the characters are divided into rows, specifically: the y coordinates of the center points of all the character regions that have been filtered three times are sorted in ascending order, and are recorded as y1, y2, ...y i …y n , if y i -y i-1 ≤H / 2, then the corresponding character areas are considered to be in the same row, otherwise, yi The corresponding character area is located at y i-1 the next line of the corresponding character area; according to the actual coding rules of the characters, determine the set number of lines and the set number of characters in each line; if the number of character lines obtained by line branching is greater than the set number of lines, sort the number of characters in each line from large to small, and filter the character lines exceeding the set number of lines; if the number of character lines obtained by line branching is less than the set number of lines, retain the number of character lines obtained by line branching; if the number of characters in the retained character line exceeds the set number of characters, sort the characters in the line in descending order according to the confidence level, and filter the characters exceeding the set number of characters; if the number of characters in the retained character line is less than the set number of characters, the characters in the line are retained; where H is the average height of the character area prediction box after three filtering.

[0020] Furthermore, the scoring algorithm performs a regular comparison between the spliced ​​billet characters and the character coding rules, and the higher the matching degree, the higher the score; the spliced ​​billet characters are compared with the actual furnace number of the billet, and the score is added or subtracted; the addition or subtraction is specifically as follows: the actual furnace number of the billet has 5 digits. When the first furnace number is 0, the last four digits are directly taken. When the first furnace number is not 0, the first two furnace numbers are converted according to the alphabetical order rule of 10=A, 11=B, 12=C... At this time, the last four digits of the spliced ​​billet characters are compared with the four-digit furnace number of the billet after the actual conversion for string text difference. If the scoring result of the regular comparison is 100 points, 2 points will be deducted for each inconsistent character match. If the scoring result of the regular comparison is less than 100 points, 2 points will be added for each consistent character match, and the final maximum score shall not exceed 100 points.

[0021] Furthermore, the supplementary recognition algorithm is: the spliced ​​image is rotated 180 degrees, 90 degrees, and 270 degrees in turn for recognition, and the character recognition process is repeated, and then scored until a score reaches 100. When the four recognition results are all below 100, the result with the highest score will be output. If the highest scores are the same, the earlier score will be output.

[0022] A character recognition and detection system for steel billets entering a heating furnace, comprising:

[0023] Model building module, used to build and train the billet cross-section recognition model and billet character recognition model;

[0024] The billet cross-section recognition module is used to determine whether there is a billet cross-section on the roller. If there is a billet cross-section, the minimum circumscribed rectangle of the character area with the largest area in the billet cross-section is obtained, and the actual angle between the minimum circumscribed rectangle and the horizontal plane is determined. The billet cross-section image is rotated according to the actual angle, with the front side facing up.

[0025] The character recognition module is used to determine the characters in the cross section of the steel billet, filter them based on the confidence of the characters, and make line judgments based on the coordinates of the character area to obtain the spliced ​​characters;

[0026] A scoring module scores the spliced ​​characters through a scoring algorithm and a supplementary recognition algorithm;

[0027] The result output module outputs the final character recognition result of the billet cross section based on the scoring result.

[0028] The beneficial effects of the present invention are:

[0029] (1) The present invention first uses a billet cross-section recognition module to identify the billet cross-section on the roller, and determines the actual angle between the minimum circumscribed rectangle of the character area with the largest area in the billet cross-section and the horizontal plane, and rotates the billet cross-section image according to the actual angle, with its front side facing up; then, the billet character recognition module is used to determine the characters in the billet cross-section. In this process, the characters are filtered based on their confidence, and the character area coordinates are used to make line judgments to obtain spliced ​​characters; finally, the spliced ​​characters are scored, and based on the scoring results, the final recognition result of the characters in the billet cross-section is output. The present invention uses machine learning to specifically identify billet characters, and uses direction correction and continuous multiple recognition correction to further improve the recognition accuracy.

[0030] (2) The machine vision-based character recognition of steel billets entering the furnace can run continuously for 24 hours, replacing the human eye for judgment, reducing the workload of workers, and integrating and interacting with the secondary system to promptly detect the inconsistency between the actual steel billet spray number and the planned spray number, and reminding manual on-site review. The present invention can detect mixed steel accidents early, reduce the situation of steel pouring in the heating furnace, effectively improve production efficiency, and reduce the energy waste caused by pouring and re-entering steel. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 This is a flow chart of character recognition detection of steel billets entering a heating furnace based on machine vision according to the present invention;

[0032] Figure 2 This is a flow chart of the steel billet cross-section identification according to the present invention. DETAILED DESCRIPTION

[0033] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments, but the protection scope of the present invention is not limited thereto.

[0034] like Figure 1 As shown, the present invention provides a method for character recognition and detection of steel billets entering a heating furnace based on machine vision, which specifically includes the following contents:

[0035] (1) Build and train the billet cross-section recognition model and billet character recognition model;

[0036] Based on YOLOv8 (an open source deep learning framework), we built a steel billet cross-section recognition model and a steel billet character recognition model.

[0037] The images of the running billet are collected by video monitoring equipment installed on both sides of the furnace roller table and transmitted to the computer. The billet cross-section image and billet character image are annotated to form the billet cross-section dataset and billet character dataset, which are used to train the billet cross-section recognition model and the billet character recognition model respectively.

[0038] (2) Obtain the roller image in actual production, determine the actual angle between the circumscribed rectangle of the character area with the largest area in the cross section of the billet on the roller and the horizontal plane, and make the cross section image of the billet face up;

[0039] In actual production, video monitoring equipment is installed on both sides of the furnace roller to collect roller images in real time and transmit them to the computer. The roller images are processed and optimized using image processing algorithms, including brightness adjustment, area division and segmentation, and grayscale conversion, which effectively improves the recognition rate of subsequent billet surfaces and reduces interference from background billets or equipment. In particular, for hot-delivered billets, the red channel in the RBG image of the hot-delivered billet is extracted before grayscale conversion during the initial processing and optimization process.

[0040] Then, by calling the trained billet cross-section recognition model, it is determined whether there is a billet cross-section on the roller. When there is a billet cross-section on the roller, the character area perception technology (CRAFT-character area perception text detector) is used to determine the boundaries of multiple character areas of the billet cross-section, and the minimum circumscribed rectangle of multiple character areas is further determined. The minimum circumscribed rectangle of the character area with the largest area is taken to determine its angle with the horizontal plane; in order to avoid the error of a single judgment, the billet cross-section image is rotated at a set angle (such as 15 degrees), and the character area boundary is re-determined, and the angle between the minimum circumscribed rectangle of the character area with the largest area and the horizontal plane is determined; the angle determined by multiple rotations Angle, minus the corresponding rotation angle, to get the angle list; divide the angle list into n intervals, take the average of all angles in the interval with the largest number of angles, and use the average as the actual angle between the minimum circumscribed rectangle of the character area with the largest area and the horizontal plane, and rotate the billet cross-section image according to the actual angle to ensure that the circumscribed rectangle is in the correct direction; the rotated billet cross-section has four directions: up, down, left and right. The machine learning image classification method (specifically: collect billet images in four directions, train the image classification model based on YOLOv8) is used to judge the direction, and the billet cross-section images in the three directions of down, left and right are rotated to finally obtain the billet cross-section image with the front side facing up.

[0041] Since steel billet character recognition requires the use of a GPU (graphics processing unit, generally a graphics card device) and is relatively time-consuming, direction judgment is used instead of directly recognizing all four directions. In most cases, only one recognition is actually performed, which effectively reduces the load on the recognition machine and increases the number of video streams that a single machine can access, or can effectively reduce the configuration requirements for a single video stream and reduce the cost of the GPU server.

[0042] (3) Perform character recognition on the cross-section image of the steel billet;

[0043] The trained billet character recognition model is called to recognize characters on the billet cross-section image to obtain recognition result information of all characters on the entire billet cross-section. The recognition result information includes character area coordinates (a plane coordinate system constructed with the upper left corner of the billet cross-section as the coordinate origin, the horizontal direction as the x-axis, and the vertical direction as the y-axis. In this embodiment, the character area coordinates refer to the coordinates of the upper left corner and the lower right corner of the prediction box where the character is located) and the confidence of the character.

[0044] Sort the confidence of all characters from high to low, and the corresponding characters are recorded as Y1, Y2...Yi...Yn in turn. Calculate the area A1, A2...Ai...An of each character area, and calculate the intersection of union (IOU) of the areas of Y1 and Y2...Yi...Yn, that is, IOU = (A1∩Ai) / (A1∪Ai). If IOU>0.5, it is determined that the overlapping part of the two character areas accounts for a large proportion, and it is likely to be two different recognition results of the same character area (for example, the number 8 may get two recognition results, 8 and B, but the confidence of one of the recognition results will be low). At this time, the character area corresponding to Yi is filtered out, and Y1 is moved into the reserved character list A. The characters that have not been filtered out are re-recorded as z1, z2...zi...zm. If m=1 at this time, that is, there is only one character left, then this character is also moved into the reserved character list A, otherwise z1, z2...zi...zm are repeated with the above IOU filtering operation.

[0045] According to the reserved character list A obtained above, they are sorted from large to small in terms of area. The corresponding characters are recorded as x1, x2...xi...xk, and the corresponding areas are A1, A2...Ai...Ak. The area inclusion calculation of x1 and x2...xi...xk is performed using the formula c= (A1∩Ai) / Ai. If c>0.9, it is considered that xi is basically included in the middle of x1, which is most likely the result of incorrect recognition of a normal character and a partial character area (for example, the number 6 may get the recognition results of 6 and the lower half 0 at the same time, and the character 4 may get the recognition results of 4 and the lower half 7 at the same time). At this time, the character area corresponding to xi is filtered out, and x1 is moved into the reserved character list B. The characters that have not been filtered out are re-recorded as Z1, Z2...Zi...Zm. If m=1 at this time, that is, only one character is left, then this character is also moved into the reserved character list B. Otherwise, the filtering operation of the above area inclusion calculation is repeated for Z1, Z2...Zi...Zm.

[0046] The center coordinates of all character regions in the reserved character list B are calculated (calculated based on the coordinates of the upper left corner and the lower right corner), and the distance between the center coordinates of any two character regions is calculated, and the minimum distance Lmin of the center coordinates of all character regions is determined. If the distance between the center coordinates of a character region and the center coordinates of other character regions is greater than 3 times Lmin, the character region is filtered out to effectively reduce edge interference. The average height of the prediction box of the character region after filtering and removal is recorded as H.

[0047] The y coordinates of the center points of all character areas after the three filterings are used for character line judgment to obtain multiple lines of character results; wherein the line judgment is specifically as follows: the y coordinates of the center points of all character areas are sorted in ascending order and recorded as y1, y2…y i …y n , if y i -y i-1 ≤H / 2, then the corresponding character areas are considered to be in the same row, otherwise, y i The corresponding character area is located at y i-1 The next line of the corresponding character area; according to the actual coding rules of the characters, determine the set number of lines and the set number of characters in each line; if the number of character lines obtained by line branching is greater than the set number of lines, sort the number of characters in each line in descending order, and filter the character lines that exceed the set number of lines; if the number of character lines obtained by line branching is less than the set number of lines, retain the number of character lines obtained by line branching.

[0048] If the number of characters in the retained character line exceeds the set number of characters, the characters in the line are sorted in descending order according to the confidence level, and the characters exceeding the set number of characters are filtered out; if the number of characters in the retained character line is less than the set number of characters, the characters in the line are retained. According to the confidence level, the wrong character recognition caused by the stripes on the boundary or the cross section of the billet can be effectively avoided.

[0049] For the same row of characters, sort the x-coordinates of the center points of the character areas from small to large.

[0050] Through the above process, the recognition results after line sorting are obtained, and the corresponding characters are spliced ​​together from left to right and from top to bottom to obtain a string of spliced ​​recognition results. Figure 2 The process from billet cross section to character recognition is demonstrated.

[0051] At the same time, in the early stage of the steel billet character recognition model training process, some incomplete but recognizable character materials were selectively added, which greatly improved the recognition rate of incomplete characters in the steel billet.

[0052] (4) Score the splicing results;

[0053] For the spliced ​​character results, a scoring algorithm and a supplementary recognition algorithm are introduced at this time, and the result with the highest score is the recognition result of this image.

[0054] 1) The scoring algorithm mainly compares the spliced ​​billet characters with the character coding rules. The higher the matching degree, the higher the score. The spliced ​​billet characters are compared with the actual furnace number of the billet, and the score is appropriately increased or decreased.

[0055] The spliced ​​billet characters are compared with the character coding rules, as follows:

[0056] ①The regular score is 100 points

[0057] a. Regular expression: the coding rules for conventional steel billets;

[0058] The first part is the last digit of the current year, such as _year_prefix = 4 in 2024;

[0059] The second part of the characters are 1-9;

[0060] The third part of the characters are 0-9;

[0061] The fourth part of the characters is 0-9 or ABCD;

[0062] The fifth part of the characters is 0-9 or ABCDEFGHJKLMN;

[0063] The sixth part of the characters is three digits;

[0064] A total of 8 characters.

[0065] b. Regular expression: The first line of the purchased steel billet is a four-digit or five-digit number, such as P400 or P2400;

[0066] The first part is P or P plus any number from 1 to 9;

[0067] The second part is 1-9;

[0068] The third part is 00;

[0069] The fourth part is 0-9 or ABCD;

[0070] The fifth part is 0-9 or ABCDEFGHJKLMN;

[0071] The sixth part is three digits;

[0072] A total of 8 or 9 characters.

[0073] ②The regular score is 90 points

[0074] Regular expression: matches when some fuzzy recognition errors occur;

[0075] The first part is three digits;

[0076] The second part is two characters, ranging from 0 to 9 or ABCDEFGHJKLMN;

[0077] The third part is three digits;

[0078] A total of 8 characters.

[0079] ③The regular score is 88 points

[0080] Regular expression: Some characters are missing or letters are not sprayed in a standard way and are not recognized, resulting in only the year + 6 digits being recognized;

[0081] The first part is the last digit of the current year;

[0082] The second part is six digits;

[0083] 7 characters in total.

[0084] ④The regular score is 85 points

[0085] Regular expression: Some characters are missing or letters are not sprayed in a standard way and are not recognized, resulting in only the year + 4 / 5 digits being recognized;

[0086] The first part is the last digit of the current year;

[0087] The second part is four or five digits;

[0088] 6 or 7 characters in total.

[0089] ⑤The regular score is 80 points

[0090] Regular expression: 8 digits are recognized, usually billets of other years or special cases where the billets are identified in the wrong direction;

[0091] The first part is 8 digits;

[0092] A total of 8 characters.

[0093] Match from top to bottom in order. If a match is found, the score is output and no further matching is performed. If all matches are successful, the number of characters will be output as the score.

[0094] The actual furnace number of the steel billet has 5 digits. When the first furnace number is 0, the last four digits are directly taken. When the first furnace number is not 0, the first two furnace numbers will be converted according to the alphabetical order of 10=A, 11=B, 12=C... For example, the furnace number 13579 will be converted to D579. At this time, the last four digits of the concatenated steel billet characters will be compared with the four-digit furnace number of the steel billet after the actual conversion for string text difference. If the regular comparison score is 100 points, 2 points will be deducted for each inconsistent character match. If the regular comparison score is less than 100 points, 2 points will be added for each consistent character match. The final maximum score shall not exceed 100 points.

[0095] 2) When the score is less than or equal to 100, the supplementary recognition algorithm will be triggered: the spliced ​​image will be rotated 180 degrees, 90 degrees, and 270 degrees in turn, and part (3) will be repeated and then scored until the score reaches 100. When the four recognition results are all less than 100, the result with the highest score will be output. If the highest scores are the same, the earlier score will be output.

[0096] (5) Based on the scoring results, the final character recognition results of the billet cross section are output;

[0097] During the actual operation, the billet will continuously move on the roller table. During this process, operations (2) to (4) are repeated. When the same character is encountered a set number of times (e.g., 3 times) and the score reaches the standard (100 points), it is determined that the character has been accurately recognized. At this time, the final character recognition result of the billet section is output.

[0098] If the same character does not continue to meet the set number of times and the score meets the standard, the result with the highest score will be output when the billet is detected to leave the recognition area. If there are multiple highest score results, the result with the most occurrences of a certain character will be output.

[0099] The departure of the billet from the identification area is determined in the following way: if the billet cross section is not identified for several consecutive times (such as 3 times), it is determined that the billet has left the identification area, thus avoiding the problem of a single billet being identified as multiple billets due to a single failure to identify.

[0100] In the above technical process, the steel billets that fail to meet the scoring standards are provided with historical query and corresponding marking functions as training data for the steel billet character recognition model.

[0101] In part (2), the changes in the center point coordinates of the billet cross section on the roller (the center point coordinates of the billet cross section are determined by the billet cross section recognition model) are recorded, which can effectively determine the moving direction of the billet cross section and identify the reverse steel situation (referring to the movement of the billet in the opposite direction); in part (3), the character orientation and the actual coding rules are determined by the billet character recognition model, and the sideways placement of the billet can be determined; the above abnormal situations are transmitted to the on-site equipment through the computer to issue alarm information reminders.

[0102] A character recognition and detection system for steel billets entering a heating furnace based on machine vision, comprising:

[0103] Model building module, used to build and train the billet cross-section recognition model and billet character recognition model;

[0104] The billet cross-section recognition module is used to determine whether there is a billet cross-section on the roller. If there is a billet cross-section, the minimum circumscribed rectangle of the character area with the largest area in the billet cross-section is obtained, and the actual angle between the minimum circumscribed rectangle and the horizontal plane is determined. The billet cross-section image is rotated according to the actual angle, with the front side facing up.

[0105] The character recognition module is used to determine the characters in the cross section of the steel billet, filter them based on the confidence of the characters, and make line judgments based on the coordinates of the character area to obtain the spliced ​​characters;

[0106] A scoring module scores the spliced ​​characters through a scoring algorithm and a supplementary recognition algorithm;

[0107] The result output module outputs the final character recognition result of the billet cross section based on the scoring result.

[0108] The entire identification and detection system of the present invention is independent and reliable, does not rely on automatic control signals, supports multi-channel video image recognition, provides multiple data interaction modes to output results to a third party, and can be efficiently integrated and interacted with the secondary system of the steel mill.

[0109] The embodiments are preferred implementations of the present invention, but the present invention is not limited to the above-mentioned implementations. Any obvious improvements, substitutions or modifications that can be made by those skilled in the art without departing from the essential content of the present invention belong to the protection scope of the present invention.

Claims

1. A method for character recognition and detection of steel billets entering a heating furnace based on machine vision, characterized in that: Obtain the roller image in actual production, and use the trained billet cross-section recognition model to determine whether there is a billet cross-section on the roller. When there is a billet cross-section on the roller, obtain the minimum circumscribed rectangle of the character area with the largest area in the billet cross-section, and determine the actual angle between the minimum circumscribed rectangle and the horizontal plane. Rotate the billet cross-section image according to the actual angle so that the front side faces upward. The trained billet character recognition model is used to recognize characters on the billet cross-section image to obtain recognition result information of all characters on the entire billet cross-section, wherein the recognition result information includes the coordinates of the character area and the confidence of the character; the confidence of all characters is sorted from high to low, and if the intersection and union ratio of a certain character area with other character areas is greater than a set value, the other character areas are filtered, and a certain character is moved into the reserved character list A; in the reserved character list A, the characters are sorted from large to small according to the area, and if the area inclusion result of a certain character and other character areas is greater than a set value, the other character areas are filtered, and a certain character is moved into the reserved character list A. Filter, a certain character is moved into the reserved character list B; the distance between the center coordinates of any two character areas is calculated from the center coordinates of all character areas in the reserved character list B, and if the distance between the center coordinates of a certain character area and the center coordinates of other character areas is greater than the set value, a certain character area is filtered out; the character line is judged according to the y coordinate of the center point of the character area to obtain multiple lines of character results; for the same line of characters, the x coordinates of the center points of the character areas are sorted in ascending order, and the corresponding characters are spliced ​​from left to right and from top to bottom to obtain a string of spliced ​​recognition results; For the spliced ​​character results, the scoring result is determined by using the scoring algorithm and the supplementary recognition algorithm; Based on the scoring results, the final character recognition results of the billet section are output: when the same character meets the set number of times and the score meets the standard, it is determined that the character has been accurately recognized, and the final character recognition result of the billet section is output; If the same character does not continue to meet the set number of times and the score meets the standard, the result with the highest score will be output when the billet is detected to leave the recognition area. If there are multiple highest score results, the result with the most occurrences of a certain character will be output.

2. The method for character recognition and detection of steel billets entering a heating furnace according to claim 1, characterized in that: The actual angle between the minimum circumscribed rectangle and the horizontal plane is determined by: The cross-section image of the steel billet is rotated at a set angle, and the minimum circumscribed rectangle of the character area is re-determined, and at the same time, the angle between the minimum circumscribed rectangle of the character area with the largest area and the horizontal plane is determined; Each time the character is rotated, the angle between the smallest circumscribed rectangle of the character area with the largest area and the horizontal plane is subtracted from the corresponding rotation angle, and the result is used to form an angle list. Divide the angle list into n intervals, take the average of all angles in the interval with the largest number of angles, and use it as the actual angle between the minimum circumscribed rectangle of the character area with the largest area and the horizontal plane.

3. The method for character recognition and detection of steel billets entering a heating furnace according to claim 1, characterized in that: After rotation, the cross-section of the billet has four orientations: up, down, left, and right. The machine learning image classification method is used to judge the orientation. The billet cross-section images in the three orientations of down, left, and right are rotated, and finally a billet cross-section image with the front side facing up is obtained.

4. The method for character recognition and detection of steel billets entering a heating furnace according to claim 1, characterized in that: If the intersection and union ratio of the area of ​​a character area with other character areas is greater than the set value, the character area will be filtered. Specifically, the confidence of all characters is sorted from high to low, and the corresponding characters are recorded as Y1, Y2...Yi...Yn in turn. The area A1, A2...Ai...An of each character area is calculated, and the intersection and union ratio (IOU) of the areas of Y1 and Y2...Yi...Yn is calculated, that is, IOU = (A1∩Ai) / (A1∪Ai). If IOU>0.5, the character area corresponding to Yi is filtered out and Y1 is moved into the reserved character list A. The characters that are not filtered out are re-recorded as z1, z2...zi...zm. If m=1 at this time, the character is also moved into the reserved character list A. Otherwise, the above IOU filtering operation is repeated for z1, z2...zi...zm.

5. The method for character recognition and detection of steel billets entering a heating furnace according to claim 1, characterized in that: If the area inclusion result of a certain character and other character areas is greater than the set value, other character areas will be filtered. Specifically, the characters in the reserved character list A are sorted from large to small in terms of area. The corresponding characters are recorded as x1, x2...xi...xk, and the corresponding areas are A1, A2...Ai...Ak. The area inclusion calculation of x1 and x2...xi...xk is performed using the calculation formula c= (A1∩Ai) / Ai. If c>0.9, the character area corresponding to xi will be filtered out and x1 will be moved into the reserved character list B. The characters that have not been filtered out will be re-recorded as Z1, Z2...Zi...Zm. If m=1 at this time, the character will also be moved into the reserved character list B. Otherwise, the filtering operation of the above area inclusion calculation will be repeated for Z1, Z2...Zi...Zm.

6. The method for character recognition and detection of steel billets entering a heating furnace according to claim 1, characterized in that: If the distance between the center coordinates of a character area and the center coordinates of other character areas is greater than the set value, the character area will be filtered out. Specifically: determine the minimum distance Lmin of the center coordinates of all character areas. If the distance between the center coordinates of a character area and the center coordinates of other character areas is greater than 3 times Lmin, the character area will be filtered out.

7. The method for character recognition and detection of steel billets entering a heating furnace according to claim 1, characterized in that: Character line division: specifically, the y coordinates of the center points of all character regions that have been filtered three times are sorted from small to large, and recorded as y1, y2…y i …y n , if y i -y i-1 ≤H / 2, then the corresponding character areas are considered to be in the same row, otherwise, y i The corresponding character area is located at y i-1 the next line of the corresponding character area; according to the actual coding rules of the characters, determine the set number of lines and the set number of characters in each line; if the number of character lines obtained by line branching is greater than the set number of lines, sort the number of characters in each line from large to small, and filter the character lines exceeding the set number of lines; if the number of character lines obtained by line branching is less than the set number of lines, retain the number of character lines obtained by line branching; if the number of characters in the retained character line exceeds the set number of characters, sort the characters in the line in descending order according to the confidence level, and filter the characters exceeding the set number of characters; if the number of characters in the retained character line is less than the set number of characters, the characters in the line are retained; where H is the average height of the character area prediction box after three filtering.

8. The method for character recognition and detection of steel billets entering a heating furnace according to claim 1, characterized in that: The scoring algorithm performs a regular comparison between the spliced ​​billet characters and the character coding rules, and the higher the matching degree, the higher the score; the spliced ​​billet characters are compared with the actual furnace number of the billet, and the score is added or subtracted; the addition or subtraction is specifically as follows: the actual furnace number of the billet has 5 digits, when the first furnace number is 0, the last four digits are directly taken, and when the first furnace number is not 0, the first two furnace numbers are converted according to the alphabetical order rule of 10=A, 11=B, 12=C..., and at this time, the last four digits of the spliced ​​billet characters are compared with the four-digit furnace number of the billet after the actual conversion for the string text difference, if the regular comparison score result is 100 points, then 2 points are deducted for each inconsistent character match, if the regular comparison score result is less than 100 points, then 2 points are added for each consistent character match, and the final maximum score does not exceed 100 points.

9. The method for character recognition and detection of steel billets entering a heating furnace according to claim 8, characterized in that: The supplementary recognition algorithm is as follows: the spliced ​​image is rotated 180 degrees, 90 degrees, and 270 degrees in turn for recognition, and the character recognition process is repeated, followed by scoring until a score reaches 100. When the four recognition results are all below 100, the result with the highest score will be output. If the highest scores are the same, the earlier score will be output.

10. A system for implementing the method for character recognition and detection of steel billets entering a heating furnace as claimed in any one of claims 1 to 9, characterized in that: include: Model building module, used to build and train the billet cross-section recognition model and billet character recognition model; The billet cross-section recognition module is used to determine whether there is a billet cross-section on the roller. If there is a billet cross-section, the minimum circumscribed rectangle of the character area with the largest area in the billet cross-section is obtained, and the actual angle between the minimum circumscribed rectangle and the horizontal plane is determined. The billet cross-section image is rotated according to the actual angle so that the front side faces upward. The character recognition module is used to determine the characters in the cross section of the steel billet, filter them based on the confidence of the characters, and make line judgments based on the coordinates of the character area to obtain the spliced ​​characters; A scoring module scores the spliced ​​characters through a scoring algorithm and a supplementary recognition algorithm; The result output module outputs the final character recognition result of the billet cross section based on the scoring result.

Images