Device and method for automatically recognizing and verifying coil marking information

Abstract

To provide a device and a method for automatically recognizing and collating coil marking information which can discriminate a printing failure other than a mismatch of marking information printed on a coil and marking command performance.SOLUTION: An automatic recognition and collation device 1 is the device for automatically recognizing and collating marking information 22 printed on a coil 20 and comprises: a camera 2 which images the marking information; a camera control unit 4 which performs control so as to adjust a zoom multiplication factor of imaging means according to a coil diameter and to image the marking information at a fixed field angle at all times; a region division unit 6 which detects a printing region and a band region from an image captured by the camera 2 and number of threads information of a band and determines overlapping between the printing region and the band region; an image recognition unit 7 which recognizes a character string of the marking information from the image captured by the camera 2 and determines presence / absence of a printing failure on the basis of that; and a collation unit 8 which compares the recognized character string with a marking command to determine whether the marking information is correct.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an apparatus and method for automatically recognizing marking information printed on a coil and comparing it with the actual results of marking commands in a steel plate manufacturing line.

Background Art

[0002] Conventionally, as a technique for comparing marking information printed on a coil with the actual results of marking commands, a method is known in which the relative relationship between the position of characters and a camera and a lighting device is always made to coincide regardless of the size of the diameter of the steel material (see, for example, Patent Document 1). Further, in order to surely detect a printed marking string, a method is known in which the marking string is sequentially illuminated and photographed from a plurality of different directions, character recognition is performed using one of the obtained image signals, and when character recognition cannot be performed, character recognition is sequentially performed using other image signals (see, for example, Patent Document 2).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the former method, regardless of the diameter of the steel material, a mechanism for moving the positions of the camera and the lighting device is required to always match the relative relationship between the position of the characters and the camera and the lighting device. Therefore, there is a problem that the device configuration becomes large and space is required. That is, these devices will be installed on the line for transporting the marked coil. However, there is often not enough space on the transport line because a transport device for sending out the coil, a binding device for binding the coil with a band, a marking device for printing marking information, and their control devices are arranged. Therefore, it is desirable that the device configuration is compact and can be installed in a small space.

[0005] In the latter method, the marking characters are sequentially illuminated and photographed from a plurality of different directions, character recognition is performed using one of the obtained image signals, and when character recognition cannot be performed, character recognition is sequentially performed using other image signals. However, there is a problem that the more devices are added to obtain good recognition accuracy, the higher the cost becomes. That is, since the mismatch between the marking information and the marking command results directly leads to quality troubles, in the latter method, in order to surely detect such a mismatch, the number of trials is ensured and the detection accuracy is improved by increasing the memory for storing the lighting and the images. However, it is desirable to reduce the cost without increasing the devices and to increase the detection accuracy.

[0006] Furthermore, in the latter method, when processing the image captured by the camera during character recognition as a binary image, there is also a problem that printing defects other than the mismatch between the printed marking information and the marking command result cannot be detected. That is, when printing the marking information on the band that binds the coil, there is a possibility of printing defects such as bleeding or dripping of ink, but these printing defects cannot be detected. When the marking information is printed on the band, the correct marking information will be lost when the band is removed. Also, when bleeding or dripping of ink occurs, even if the marking information can be correctly read immediately after printing, as time passes, the bleeding or dripping may deteriorate and the marking information may become unreadable. Therefore, it is desirable that these printing defects can be determined as non-conformities in the same way as the mismatch between the marking information and the marking command result.

[0007] Therefore, the present invention provides an apparatus and method for automatically recognizing and collating coil marking information, which has a compact device configuration with a small installation space, does not require an increase in the number of devices and has a low cost, and can also detect printing defects other than the mismatch between the printed marking information and the marking command result.

Means for Solving the Problems

[0008] To solve the above problems, the present invention provides the following [1] to [8].

[0009] [1] An apparatus for automatically recognizing and collating marking information printed on a coil, imaging means for imaging the marking information; imaging control means for adjusting the zoom ratio of the imaging means according to the coil diameter and controlling to always image the marking information at a constant angle of view; region dividing means for detecting, from the image captured by the imaging means and the number-of-strips information of the band that binds the coil, a printing region where a character string in the image is printed and a band region where the band is hung, and determining an overlap between the printing region and the band region; Image recognition means for recognizing the character string of the marking information from the image captured by the imaging means and determining the presence or absence of printing defects based on the recognition result of the character string; Collation means for comparing the recognized character string with a marking command to determine whether the marking information is correct; An apparatus for automatically recognizing and collating coil marking information, which has these.

[0010] [2] The detection of the printing area and the band area by the area division means is performed using a trained model that has been learned by deep learning with a camera imaging image as an input and the band area and the printing area as outputs, using several training images in advance as an image recognition model, for the apparatus for automatically recognizing and collating coil marking information described in [1].

[0011] [3] The recognition of the character string by the image recognition means is performed using a trained model that has been learned by deep learning with the printing area as an input and a character string as an output, using several training data in advance as an image recognition model, for the apparatus for automatically recognizing and collating coil marking information described in [1].

[0012] [4] If any of the determination of the overlap by the area division means, the determination of the printing defect by the image recognition means, and the determination of the marking information by the collation means is not in conformity, it is determined that the coil marking information is not in conformity, for the apparatus for automatically recognizing and collating coil marking information described in any of [1] to [3].

[0013] [5] A method for automatically recognizing and collating marking information printed on a coil, comprising: Adjusting the zoom magnification of imaging means for imaging the marking information according to the coil diameter, and always imaging the marking information at a constant angle of view; From the image captured by the imaging means and the information on the number of bands that bind the coil, respectively detect the printing area where the character string in the image is printed and the band area where the band is placed, and determine the overlap between the printing area and the band area; Recognize the character string of the marking information from the image captured by the imaging means, and determine the presence or absence of printing defects based on the recognition result of the character string; Compare the recognized character string with the marking command to determine whether the marking information is correct; A method for automatically recognizing and verifying coil marking information.

[0014] [6] The detection of the printing area and the band area is performed using a pre-trained model that has been learned by deep learning with several training images in advance as an image recognition model, taking the camera imaging image as the input and the band area and the printing area as the outputs, according to the method for automatically recognizing and verifying coil marking information described in [5].

[0015] [7] The recognition of the character string is performed using a pre-trained model that has been learned by deep learning with several training data in advance as an image recognition model, taking the printing area as the input and the character string as the output, according to the method for automatically recognizing and verifying coil marking information described in [5].

[0016] [8] If any of the determination of the overlap, the determination of the printing defect, and the determination of the marking information is not in compliance, it is determined that the coil marking information is not in compliance, according to the method for automatically recognizing and verifying coil marking information described in any of [5] to [7].

Advantages of the Invention

[0017] According to the present invention, by adjusting the zoom ratio based on the coil diameter information, an image of the marking information is captured at a constant angle of view. Therefore, without using a mechanism for moving the imaging means, the printing surface can be imaged regardless of the coil diameter, and a compact device configuration with a small installation space can be realized. Further, based on the captured image, the marking information can be recognized and verified by the region division means, the image recognition means, and the verification means, so that the number of devices can be reduced and the device cost can be suppressed compared with the conventional case. In addition, since the risk of failure is low, there is an effect that the maintainability is high. Furthermore, it is possible to detect printing defects such as overlap between the band region and the printing region (marking character string), ink dripping, and blurring, which were not detectable by the conventional method, and determine them as non-conforming.

Brief Description of the Drawings

[0018]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Embodiments for Carrying Out the Invention

[0019] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. FIG. 1 is a block diagram showing an example of an automatic recognition and verification device for coil marking information according to an embodiment of the present invention. As shown in FIG. 1, the automatic recognition and verification device 1 includes a camera 2 which is an image capturing means and a computer 3.

[0020] The camera 2 captures images of the band 21 and the marking information 22 of the coil 20 being conveyed by the conveying line.

[0021] The computer 3 is connected to an external line control device 10. The line control device 10 includes a database 11 that stores data such as coil diameter information, number of band information, printing command performance, and conveyance command timing regarding the coil 20, and a control unit 12 that performs control based on the output from the computer 3. Then, the computer 3 receives the data stored in the database 11, performs processing, and outputs the processing result to the control unit 12.

[0022] This computer 3 includes a camera control unit 4, an image recognition model 5, a region division unit 6, an image recognition unit 7, and a verification unit 8.

[0023] The camera control unit 4 controls the camera 2. According to the coil diameter, it controls the zoom-in and zoom-out of the camera 2 to adjust the zoom ratio, and controls the camera 2 to always capture the marking information at a constant angle of view. When capturing an image, the camera control unit 4 controls the zoom function according to the coil diameter information of the line control device 10, and controls the camera 2 to capture an image at the timing when the coil is conveyed to a predetermined position according to the conveyance command timing.

[0024] FIG. 2 is a diagram showing a method for arranging a camera and controlling a zoom function for photographing marking information at a constant angle of view for any coil diameter. First, regarding the arrangement of the camera 2, as shown in (a), by arranging the camera 2 on a straight line connecting the printed surface 23 of coils of a plurality of diameters (in this example, coils 20A, 20B, 20C) with the lowest points aligned, the printed surface 23 can be captured within the screen regardless of the coil diameter. Next, regarding the control of the zoom function, as shown in (b), based on the focal length 24A of the coil 20A with the largest coil diameter, the focal lengths to be set for each coil diameter can be calculated. For example, when the value of the focal length 24A of the coil 20A with the largest coil diameter d1 is L1, then for the coil 20B with the next larger coil diameter d2, the value L2 of the focal length 24B can be expressed as L2 = L1 + α×(d1 - d2) using L1 and a constant α. Similarly, for the coil 20C with the smallest coil diameter d3, the value L3 of the focal length 24C is L3 = L1 + α×(d1 - d3), and by giving the largest coil diameter and its focal length, the focal lengths to be set for any coil diameter can be derived.

[0025] The zoom of the camera 2 ensures an arbitrary field-of-view size by maintaining focus (auto-focus: automatic focus) and magnifying and reducing the image of the object to be photographed. There are an optical zoom and a digital zoom. In the case of the optical zoom, the lens mechanism has a function of controlling the focal length from the center point of the lens to the image sensor, but the device becomes slightly larger and the possibility of failure also increases. In the case of the digital zoom, since a high-resolution image sensor is used and a part of the image is enlarged and reduced by software, there is image degradation compared to the optical zoom, but the risk of failure is small and the device is also compact.

[0026] The image recognition model 5 corresponds to the processing in the region division unit 6 and the processing in the image recognition unit 7 described below, and is prepared in advance by learning through several Training data and a plurality of them are prepared.

[0027] The area division unit 6 uses the image recognition model 5 based on the captured image and the number-of-bands information from the line control device 10 to detect the area (print area (coordinates)) where the character string in the image is printed and the area (band area (coordinates)) where the band that binds the coil is placed, respectively, and distinguishes between them. Then, when the area division unit 6 compares the print area and the band area and detects that the print area and the band area of the coil 20 overlap, it determines that it is not suitable, assuming that it is printed on the band 21.

[0028] As the image recognition model 5 used in the area division unit 6, a plurality of models trained for each number of bands are prepared. Then, the image recognition model is switched according to the number of bands to detect the band area. Specifically, as the image recognition model 5, a model trained with a no-band image, a model trained with a single-band image, and a model trained with a two-band image are prepared, and the model to be used is selected according to the number-of-bands information received from the line control device 10. In this way, by switching the image recognition model trained for each number of bands to detect the band area, the accuracy of the area division function can be improved.

[0029] As the image recognition model 5 used in the area division unit 6, several pre-training Data can be used, and a learned model learned by deep learning with the camera imaging image as the input and the band area (coordinates) and the print area (coordinates) as the outputs can be used. The image recognition model by deep learning used at this time is not particularly limited, and for example, existing ones such as YOLO, Fast, and R-CNN can be used.

[0030] FIG. 3 is a diagram showing an example of a printing area and a band area when the number of bands is 0, 1, or 2. In the case of the coil 20D with 0 bands, as shown in (a), there is no band area, and only one printing area 25 is detected. In the case of the coil 20E with 1 band, as shown in (b), one band area 26 and two printing areas 25A and 25B divided by the band area 26 are detected. In the case of the coil 20F with 2 bands, as shown in (c), two band areas 26A and 26B and three printing areas 25C, 25D, and 25E divided by the band areas 26A and 26B are detected.

[0031] An example in which the printing area and the band area overlap as described above is shown in FIG. 4. FIG. 4 is an example in which marking information 22 is printed on the band 21 of the coil 20, and the printing area and the band area overlap. Depending on the accuracy of the marking device that marks the coil and the bundling device that binds with the band, rarely, marking information may be printed overlapping such a band. In such a case, since the marking information is lost when the band is removed, the image segmentation unit 6 detects such a defect and determines it as non-conforming.

[0032] The image recognition unit 7 takes as input the printing area detected by the area segmentation unit 6 from the camera image, and uses the image recognition model 5 to convert the marking information 22 on the captured image into string data by character recognition and recognize (read) the string. Then, based on the recognition result of the string, it determines whether there is a printing defect. Specifically, the image recognition unit 7 uses, as the image recognition model 5, one that has been pre-trained with training data in which printing defects such as ink smudging and dripping are labeled as NG and normal printing as OK, and determines it as non-conforming if the captured image contains a printing defect.

[0033] As the image recognition model 5 used in the image recognition unit 7, a pre-trained model can be used which is learned by deep learning using, as input, the printed area (coordinates) detected by the area division unit 6 from a camera image, and outputting a character string (including the OK / NG attribute for each character). The image recognition model by deep learning used at this time is not particularly limited, and for example, an existing one such as CNN can be used.

[0034] FIG. 5 is a diagram showing an example of normal printing and printing defects caused by malfunctions of the marking device. The normally printed normal print 31 in (a) has a clear character shape and is printed with a constant density from the beginning to the end of the character. The dripping-occurred print 32 printed in a state where ink dripping has occurred in (b) has a distorted character shape. The blurred-occurred print 33 printed in a state where ink blurring has occurred in (c) has a reduced ink density in the middle of the character. Since these printing defects may deteriorate the dripping and blurring over time and the marking information may be lost, if it is determined as a printing defect by the collation unit 8, it is regarded as non-conforming.

[0035] The collation unit 8 compares the character string data recognized by using the image recognition model 5 by the image recognition unit 7 with the result of the marking command sent from the line control device 10, and determines whether the marking information is correct.

[0036] The determination results of the area division unit 6, the image recognition unit 7, and the collation unit 8 as described above are sent to the control unit 12 of the line control device 10. In the control unit 12 of the line control device 10, if any one of the above determinations is non-conforming, it is determined that the coil marking information is non-conforming, and the line operator is notified to that effect, such as by issuing an alarm, and the correction of the coil marking information is urged. On the other hand, if all are conforming, it is assumed that correct marking information has been printed, and the target coil is conveyed to the next process.

[0037] Next, the operation flow when the coil marking information is automatically recognized and collated by the automatic recognition and collation device 1 configured as described above will be described. FIG. 6 is a flowchart showing the operation flow.

[0038] In the automatic recognition and collation device 1, first, the printing command record is acquired from the line control device 10 (ST1), the coil diameter information is acquired (ST2), the number of band information is acquired (ST3), and the conveyance command timing (ST4) is acquired.

[0039] Next, using the received coil diameter information, the camera control unit 4 adjusts the angle of view by zooming in and out the camera 2 (ST5). After the adjustment of the angle of view is completed, the marking information is photographed by the camera 2 (ST6).

[0040] Next, the region division unit 6 calls the image recognition model 5 of the band region corresponding to the number of bands from the received number of bands, detects the band region from the photographed image, and detects the printing region from the photographed image (ST7).

[0041] Next, the image recognition unit 7 recognizes the marking character string from the printing region (ST8).

[0042] Next, the region division unit 6 compares the band region and the printing region, and determines whether there is a marking print in the band region, that is, whether the printing region and the band region overlap (ST9). If there is a marking print in the band region, an incompatibility determination is returned to the line control device 10 (ST13). If there is no marking print in the band region, the process proceeds to the next determination.

[0043] In the next determination, the image recognition unit 7 determines whether there is a printing defect such as dripping or smearing based on the recognition result of the character string in ST8 (ST10). If it is determined that there is a printing defect, an incompatibility determination is returned to the line control device 10 (ST13). If there is no printing defect, the process proceeds to the next determination.

[0044] In the next determination, the collation unit 8 collates the character string data (marking print) recognized (read) by the image recognition unit 7 with the print command result obtained in ST1, and determines whether the print and the result match (ST11). If the print and the result match, a conformity determination is returned to the line control device 10 (ST12). On the other hand, if the print and the result do not match, a non-conformity determination is returned to the line control device 10 (ST13).

[0045] In the line control device 10, when a non-conformity determination of ST13 is returned, the line operator is notified to that effect, such as by issuing an alarm, and the correction of the coil marking information is urged. On the other hand, when a conformity determination of ST12 is returned, assuming that the correct marking information has been printed, a command to convey the target coil to the next process is issued.

[0046] According to the present embodiment, by adjusting the zoom ratio of the camera 2 according to the coil diameter (controlling zoom in and zoom out), the marking information printed on the coil 20 is always photographed at a constant angle of view. As a result, without using a mechanism for moving the camera, it is possible to capture the printed surface of the coil 20 with any coil diameter, and a compact device configuration can be realized. Specifically, by arranging the camera 2 on the straight line connecting the printed surface of the coil 20 and the lowest point of the coil 20 and adjusting the zoom ratio, the printed surface can be captured regardless of the coil diameter without moving the camera. Further, since the printed surface to be photographed is farther from the camera 2 as the diameter of the coil 20 is smaller, and the printed surface is closer to the camera 2 as the diameter of the coil is larger, the focal length for each coil diameter can be calculated by setting the focal length of the maximum coil diameter as a reference value.

[0047] Also, according to the present embodiment, by using the image recognition model 5 based on the captured image and detecting the printing area and the band area by the area division unit 6, it is possible to detect the overlap between the band area and the printing area. By recognizing the character string by the image recognition unit 7, it is possible to determine the presence or absence of printing defects. By comparing the character string data and the marking command by the collation unit 8, it is possible to determine whether the marking information is correct. Therefore, it is possible to ensure the character recognition accuracy without adding equipment as in the conventional method, realize the reduction of equipment cost and the improvement of maintainability, and detect printing defects such as the overlap between the band area and the printing area (marking character string), ink dripping, and blurring, which were impossible to detect by the conventional method, and determine them as non-conforming.

[0048] As described above, the embodiments of the present invention have been described, but these are merely examples and should not be considered restrictive. The above embodiments may be omitted, replaced, or changed in various forms without departing from the gist of the present invention.

Explanation of Reference Numerals

[0049] 1 Automatic recognition and collation device 2 Camera (imaging means) 3 Computer 4 Camera control unit (imaging control means) 5 Image recognition model 6 Area division unit (area division means) 7 Image recognition unit (image recognition means) 8 Collation unit (collation means) 10 Line control device 11 Database 12 Control unit 20, 20A, 20B, 20C, 20D, 20E, 20F Coil 21 Band 22 Marking information 23 Printing surface 24A, 24B, 24C Focal length 25, 25A, 25B, 25C, 25D, 25E Printing area 26, 26A, 26B Band area 31 Normal printing 32 Dripping occurrence printing 33 Blurring occurrence printing

Claims

1. An apparatus for automatically recognizing and verifying marking information printed on a coil conveyed on a conveying line, comprising: imaging means for imaging the marking information; computing means for recognizing and verifying the marking information based on the image captured by the imaging means, the information on the coil diameter of the coil sent from a control unit that controls the conveying line, the information on the number of bands indicating the number of bands for binding the coil applied to the coil, and the record of the marking command for the coil; having: The computing means includes: imaging control means for controlling the zoom magnification of the imaging means according to the coil diameter of the coil so that the marking information is always imaged at a constant angle of view by the imaging means; using an image recognition model selected based on the information on the number of bands from a plurality of image recognition models that have learned, for each number of bands, the function of outputting data indicating, when an image captured by the imaging means and an image captured by the imaging means are input, the printing area where a character string is printed and the band area where a band is applied in the image, respectively, to detect the printing area and the band area, and determining whether the printing area and the band area overlap; using an image recognition model that has learned the function of outputting, when the printing area detected by the area division means of the image captured by the imaging means is input, data indicating the character string recognized by character recognition and data indicating that there is a printing defect in the character string, to recognize the character string of the marking information by character recognition and determine whether there is a printing defect in the character string; collation means for comparing the recognized character string with the marking command and determining whether the marking information matches the marking command; An apparatus for automatically recognizing and verifying coil marking information.

2. The detection of the printing area and the band area by the area division means is performed using a learned model that has been learned by deep learning for each number of bands, as the image recognition model, to output data indicating the coordinates of the band area and the coordinates of the printing area when an image captured by the imaging means is input in advance. The apparatus for automatically recognizing and verifying coil marking information according to Claim 1.

3. The recognition of the character string by the image recognition means is performed using a learned model obtained by deep learning the function of outputting, as the image recognition model, data of a character string recognized from an image of the printing area and data indicating whether the character string is a printing defect or normal printing when coordinates of the printing area detected by the area division means of an image photographed by the photographing means are input in advance. The apparatus for automatically recognizing and collating coil marking information according to claim 1.

4. When any one of the determination that the printing area and the band area overlap by the area division means, the determination that there is a printing defect in the character string of the marking information by the image recognition means, and the determination that the marking information does not match the marking command by the collation means is made, it is determined that the coil marking information is inappropriate. The apparatus for automatically recognizing and collating coil marking information according to any one of claims 1 to 3.

5. A method for automatically recognizing and collating marking information printed on a coil conveyed on a conveying line, which photographs the marking information printed on the coil conveyed on the conveying line by photographing means, and based on the image photographed by the photographing means, information on the coil diameter of the coil, band number information indicating the number of bands for binding the coil applied to the coil, and the record of the marking command to the coil sent from a control unit that controls the conveying line, A zoom magnification control step of controlling the zoom magnification of the photographing means according to the coil diameter of the coil so that the marking information is always photographed at a constant angle of view by the photographing means; An overlap determination step of detecting the printing area and the band area respectively using an image recognition model selected based on the band number information from a plurality of image recognition models learned for each band number to output data indicating the printing area where a character string is printed and the band area where a band is applied in the image when the image photographed by the photographing means and the image photographed by the photographing means are input, and determining whether or not the printing area and the band area overlap. When the printing area detected in the overlap determination step of the image captured by the imaging means is input, using an image recognition model that has learned a function to output data indicating the character string recognized by character recognition and data indicating that there is a printing defect in the character string, the character string of the marking information is recognized by character recognition, and a printing defect determination step of determining whether there is a printing defect in the character string. A marking determination step of comparing the recognized character string with the marking command to determine whether the marking information matches the marking command. A method for automatically recognizing and verifying coil marking information, which has the above steps.

6. The detection of the printing area and the band area is performed using a learned model that has been previously learned by deep learning for each number of bands as an image recognition model, which outputs data indicating the coordinates of the band area and the coordinates of the printing area when an image captured by the imaging means is input. The method for automatically recognizing and verifying coil marking information according to claim 5.

7. The recognition of the character string is performed using a learned model that has been previously learned by deep learning as an image recognition model, which outputs data of the character string recognized by character recognition from the image of the printing area and data indicating whether the character string has a printing defect or a normal print when the coordinates of the printing area detected in the overlap determination step of the image captured by the imaging means are input. The method for automatically recognizing and verifying coil marking information according to claim 5.

8. If any of the determinations that the printing area and the band area overlap, that there is a printing defect in the character string of the marking information, and that the marking information does not match the marking command is made, it is determined that the coil marking information is inappropriate. The method for automatically recognizing and verifying coil marking information according to any one of claims 5 to 7.

Images