Appearance inspection method and appearance inspection device

Abstract

The present invention provides an appearance inspection method and an appearance inspection device, which can detect appearance defects with high precision. The appearance inspection method comprises: a photographing step (S1), in which a plurality of the inspected objects are photographed to obtain images; a first sorting step (S2), in which a single classification learning algorithm is used to sort the images into qualified product presumption images within a predetermined benchmark and unqualified product presumption images outside the benchmark; and a second sorting step (S3), in which an object detection algorithm is used to extract defect features from the unqualified product presumption images, and unqualified products are detected from the inspected objects by comparing the images and the features.

Description

Technical Field

[0001] The present invention relates to a method and a device for automatically inspecting the appearance of an inspected object. Background Art

[0002] Currently, various studies have been conducted on technologies for inspecting the appearance of an object to be inspected, such as a tire (for example, refer to Patent Document 1).

[0003] Patent Document 1: Japanese Patent Application Publication No. 2018-105699 Summary of the invention

[0004] However, in the above-mentioned inspection of appearance, further improvement of inspection accuracy is desired.

[0005] The present invention is proposed in view of the above actual situation, and its main purpose is to provide a visual inspection method and a visual inspection device capable of detecting appearance defects with high accuracy.

[0006] The first scheme of the present invention is a method for inspecting the appearance of an inspected object, which includes: a photographing process, in which a plurality of the inspected objects are photographed to obtain images; a first sorting process, in which a single-classification learning algorithm is used to sort the images into images of qualified products within a predetermined benchmark and images of unqualified products outside the benchmark; and a second sorting process, in which an object detection algorithm is used to extract features of defects from the unqualified product presumption images, and the features are retrieved within the images, thereby detecting unqualified products from the inspected objects.

[0007] In the appearance inspection method of this aspect, preferably, in the second sorting step, the features are classified and accumulated for each type of the defects.

[0008] In the appearance inspection method of the present embodiment, preferably, in the second sorting process, the features extracted from the image of the inspected object that was detected as the defective product in the second sorting process after being sorted as the qualified product presumption image in the first sorting process are accumulated.

[0009] In the appearance inspection method according to the present invention, preferably, the inspection object is a tire.

[0010] In the appearance inspection method according to the present invention, preferably, in the imaging step, an annular imaging region of the tire side portion is imaged.

[0011] In the appearance inspection method according to the present invention, preferably, in the imaging step, the imaging region is converted into a strip shape.

[0012] The second scheme of the present invention provides a device for inspecting the appearance of an inspected object, the device comprising: a photographing unit, which photographs a plurality of the inspected objects to obtain images; a storage unit, which stores the images; a first sorting unit, which uses a single-classification learning algorithm to sort the images into images of qualified products within a predetermined benchmark and images of unqualified products outside the benchmark; and a second sorting unit, which uses an object detection algorithm to extract features of defects from the unqualified product presumption images, retrieve the features within the images, and thereby detect defective products from the inspected objects.

[0013] In the appearance inspection device according to the present invention, preferably, the second sorting unit classifies the features for each type of the defect, and the storage unit accumulates the features for each type.

[0014] In the appearance inspection device according to the present invention, preferably, the imaging unit includes a 2D camera.

[0015] In the appearance inspection device according to the present invention, preferably, the imaging unit includes a 3D camera.

[0016] The appearance inspection method of the first scheme includes: the first sorting process, in which the algorithm of the single classification learning is used; and the second sorting process, in which the algorithm of the object detection is used. In the first sorting process, the image obtained in the shooting process is sorted into the qualified product presumption image or the unqualified product presumption image. In the second sorting process, the feature of the defect is extracted from the unqualified product presumption image. Furthermore, the feature is retrieved in the image to determine whether the defect exists in the inspected object. As a result, the unqualified product is detected from the inspected object with high accuracy.

[0017] In the first embodiment, as the images captured are accumulated as the inspection progresses, the estimated images of defective products obtained in the first sorting process increase, and the features of the defects extracted in the second sorting process also automatically increase. Therefore, by inspecting a large number of the inspected objects, the detection accuracy of the defective products can be easily improved.

[0018] The appearance inspection device of this second solution includes: the first sorting unit, which uses the algorithm of the single classification learning; and the second sorting unit, which uses the algorithm of the object detection. The first sorting unit sorts the image obtained by the photographing unit into the qualified product presumption image or the unqualified product presumption image. The second sorting unit extracts the features of the defect from the unqualified product presumption image. Further, the second sorting unit retrieves the features in the image and determines whether there is such a defect in the object to be inspected. Thus, the unqualified products are detected from the object to be inspected with high precision.

[0019] In this second solution, as the inspection progresses, the images accumulated by photographing increase, the unqualified product presumption images obtained by the first sorting unit increase, and the features of the defects extracted by the second sorting unit also automatically increase. Therefore, by inspecting a large number of objects to be inspected, it is possible to easily improve the detection accuracy of unqualified products. Description of the Drawings

[0020] Figure 1 It is a flowchart showing an example of the processing sequence of an embodiment of the appearance inspection method of this first solution.

[0021] Figure 2 It is a block diagram showing the schematic structure of the appearance inspection device of this second solution.

[0022] Figure 3 It shows Figure 2 a block diagram of the schematic structure of the operation unit of.

[0023] Figure 4 It is a diagram showing an example of a learning pattern generated by deep learning.

[0024] Figure 5 It shows Figure 1 a diagram of an example of a defect detected in the second sorting process of.

[0025] Figure 6 It shows Figure 2 a diagram of the photographing area of a tire photographed by the camera of.

[0026] Figure 7 It is a diagram showing an image transformed into a strip shape.

[0027] Reference Numeral Explanation

[0028] 1: Appearance inspection device; 2: Photographing unit; 21: Camera; 4: Processing unit (first sorting unit, second sorting unit); 42: Storage unit; 50: Conformable product estimated image; 51: Defective product estimated image; 100: Object to be inspected; 101: Tire; 102: Sidewall; 200: Defect; S: Photographing process; S2: First sorting process; S3: Second sorting process. DETAILED DESCRIPTION

[0029] Hereinafter, one embodiment of the present invention will be described with reference to the drawings.

[0030] Figure 1 An example of the processing sequence of an embodiment of the appearance inspection method of the first scheme is shown. The appearance inspection method is a method for inspecting the appearance of an object to be inspected. The so-called inspection of the appearance of the object to be inspected refers to determining whether there is a defect (abnormality) in the appearance of the object to be inspected. The appearance inspection method is implemented, for example, by visual inspection by an operator or by alternating visual inspection by an operator.

[0031] The object to be inspected is not particularly limited. In this embodiment, industrial products such as tires are used as the object to be inspected. In this case, the appearance inspection method of the first scheme is implemented on the production line of the industrial product. Then, the product determined to have a defect in appearance is not shipped but discarded, or shipped after the defect is repaired.

[0032] Hereinafter, although the appearance inspection of tires will be described, the appearance inspection of other industrial products may be applied by replacing tires with the industrial products.

[0033] The appearance inspection method includes an imaging step S1, a first sorting step S2, and a second sorting step S3. The appearance inspection method is performed using an appearance inspection device. The appearance inspection device 1 is a device for inspecting the appearance of an object to be inspected.

[0034] Figure 2 The schematic structure of the appearance inspection device 1 is shown. The appearance inspection device 1 is a device for inspecting the appearance of an object to be inspected. The appearance inspection device 1 includes an imaging unit 2 and a computing unit 3. For example, a computer device 4 is applied to the computing unit 3.

[0035] The photographing unit 2 has, for example, a camera 21 equipped with a photographing element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The camera 21 photographs a plurality of tires 101 as the object to be inspected 100 and acquires an image thereof. More specifically, the camera 21 converts light reflected by the tire 101 or the like into an electrical signal and acquires electronic data of the image. The image data acquired by the camera 21 is sent to the computer device 4 and stored.

[0036] In the photographing unit 2, a lighting device 22 for irradiating the tire 101, which is the subject of the camera 21, is provided as needed. In addition, a device (not shown) for positioning the conveyed tire 101 and rotating the photographing unit 2 is provided in the photographing unit 2. The camera 21, the lighting device 22, and the above-mentioned device are controlled by the computer device 4, for example.

[0037] Figure 3 The schematic structure of the computer device 4 is shown. The computer device 4 includes the following components: a processing unit 41 that performs various arithmetic processes and information processes; a storage unit 42 that stores programs and various information responsible for the operation of the processing unit 41; an input unit 43 that is used to input various instructions and information to the processing unit 41; and an output unit 44 that is used to output the processing results of the processing unit 41. In addition, the processing unit 41 of the present embodiment is also responsible for controlling the camera 21, the lighting device 22, and the like.

[0038] The processing unit 41 is composed of a CPU (Central Processing Unit) and a memory, for example. In addition, in the processing unit 41, instead of or in addition to the CPU, a GPU (Graphics Processing Unit), which is excellent in specific data processing, can also be applied. A large-capacity hard disk drive device is used in the storage unit 42, for example.

[0039] Devices such as a keyboard and a mouse, and a device that is connected to the photographing unit 2 and used to receive the input of image data from the photographing unit 2 are used in the input unit 43. A display device such as an LCD (Liquid Crystal Display) is used in the output unit 44, for example.

[0040] The computer device 4 may be provided with a communication unit (not shown) for exchanging various information (e.g., information for visual inspection of the tire 101) with other computer devices (not shown). The communication unit is connected to other computer devices via a LAN (Local Area Network). In this case, the operation unit 3 is composed of a network including the computer device 4 and other computer devices.

[0041] Below, refer to Figures 1 to 3 , the processing sequence of the appearance inspection method, that is, the operation of the appearance inspection device 1 is described.

[0042] exist Figure 1 In the photographing step S1 shown, the plurality of tires 101 are photographed by the camera 21 and the images thereof are acquired as electronic data. The image data acquired in the photographing step S1 is sent to the computer device 4 and stored in the storage unit 42 by the processing unit 41 .

[0043] The first sorting step S2 and the second sorting step S3 are executed by the processing unit 41 of the computer device 4 .

[0044] In the first sorting step S2, the image acquired in the photographing step S1 is sorted into a qualified product estimated image 50 or a defective product estimated image 51 using a one-class support vector machine (SVM) algorithm. The qualified product estimated image 50 is an image of a tire 101 estimated to have a good appearance by the one-class support vector machine, and is stored, for example, in a qualified product image folder 55 of the storage unit 42. On the other hand, the defective product estimated image 51 is an image of a tire 101 estimated to have a poor appearance by the one-class support vector machine (SVM), and is stored, for example, in a defective product image folder 56 of the storage unit 42.

[0045] The so-called single classification learning is an algorithm for detecting outliers from the learning value of qualified products learned without a teacher, and is a method suitable for abnormality inspection of industrial products such as tires 101, most of which are qualified products. In this embodiment, the above-mentioned learning value is obtained by learning image data of qualified tires, for example. In the learning of image data, for example, a deep learning method is used.

[0046] Figure 4 An example of a learning pattern generated by deep learning is shown. In the present embodiment, the learning pattern 700 is generated by the calculation of the processing unit 41 and stored in the storage unit 42. The learning pattern 700 can be generated by the calculation of another computer device outside the computer device 4, for example, and input to the computer device 4 and stored in the storage unit 42.

[0047] The learning mode 700 defines, for example, the image data 701a, 701b, 701c, 701d, 701e... of multiple qualified tires as the input layer 701, and the learning values 703a, 703b, 703c... of the qualified products as the output layer 703, through the intermediate layer 702 generated by machine learning.

[0048] The intermediate layer 702 of this embodiment includes a combination of multiple neurons (nodes) 704 stratified in multiple levels and optimized weighting coefficients 705 (parameters). Each neuron 704 is connected by the weighting coefficient 705. Such an intermediate layer 702 is called a Convolutional Neural Network. That is, the learning mode 700 of this embodiment includes a convolutional neural network.

[0049] In the first sorting process S2, by using single-class learning to detect the deviation values from the learning values 703a, 703b, 703c... of the qualified products, it is also possible to detect new (unlearned) defects that are difficult to imagine. Therefore, it is possible to easily collect the defective product estimation images 51 of various forms.

[0050] In the second sorting process S3, an object detection algorithm is used to detect the defective tires 101. Object detection is an algorithm for detecting an object (a defect in this embodiment) within an image of an object to be inspected. For example, a learning mode including a convolutional neural network is applied to object detection.

[0051] In the second sorting process S3 of this embodiment, the features of the defects are retrieved from the image data obtained in the photographing process S1. The features of the defects are extracted from the defective product estimation images 51 sorted and stored in the storage unit 42 in the first sorting process S2. That is, the processing unit 41 extracts the features of the defects from the defective product estimation images 51 and stores them, for example, in the defect folder 57 of the storage unit 42. In addition, the extracted features of the defects also include features in dimensions that humans cannot understand through the above-mentioned convolutional processing. Then, the processing unit 41 retrieves the defects in the image data by comparing the image data obtained in the photographing process S1 with the features of the defects, and detects defective products from the photographed tires 101. That is, when there are defects in the image data, the tire 101 that is the source of the image data is determined to be a defective product.

[0052] Figure 5 This is an example of the defect 200 detected in the second sorting process S3. In this example, it is shown on the sidewall portion 102 of the tire 101 (refer to Figure 2) Defects 200 generated locally. In this figure, it is confirmed that appearance defects 200 caused by poor rubber flow or the like have occurred at the position of the letter "E" formed on the surface of the sidewall portion 102. In addition, it is also confirmed that the same defects 200 have occurred near the display "205 / 60R16" indicating the tire size. According to the present embodiment, not only the defects 200 shown in Figure 5 are detected, but also defects such as foreign matter mixing into the sidewall portion 102 may be detected, for example.

[0053] For industrial products such as the tire 101, usually most are qualified products, and the occurrence frequency of defective products is low. Therefore, it is generally difficult to collect the characteristics of various defects 200 and store them in the storage unit 42. However, in the present embodiment, as the inspection progresses, the images captured by the camera 21 are accumulated in the photographing process S1, the number of defective product estimation images 51 obtained in the first sorting process S2 increases, and the characteristics of the defects 200 extracted and accumulated in the storage unit 42 in the second sorting process S3 also increase. Therefore, by inspecting a plurality of tires 101, it is easy to improve the detection accuracy of defective products.

[0054] In the present embodiment, in the first sorting process S2, the processing unit 41 functions as a first sorting unit that sorts the images obtained by the camera 21 into a qualified product estimation image 50 and a defective product estimation image 51 using an algorithm of single classification learning. In addition, in the second sorting process S3, the processing unit 41 functions as a second sorting unit that detects defective products from the tire 101 photographed by the camera 21 using an algorithm of object detection. That is, the first sorting unit and the second sorting unit are implemented by the processing unit 41 and software responsible for its operation, etc. In addition, an algorithm of single classification learning and an algorithm of object detection can also be introduced by applying known software.

[0055] In the second sorting process S3, it is preferable to classify and accumulate the characteristics of the defects 200 according to each type of the characteristics of the defects 200 (for example, the above-mentioned poor rubber flow, foreign matter mixing, etc.). In the present embodiment, the characteristics of the defects 200 are classified by the processing unit 41 (second sorting unit) and stored in the defect folder 57 of the storage unit 42 according to each type. By comparing the characteristics of the defects 200 classified according to each type with the image data obtained in the photographing process S1, the detection accuracy of defective products is further improved.

[0056] In the second sorting process S3, it is preferable to compare not only the defective product estimation image 51 but also the non-defective product estimation image 50 with the features of the defect 200. In this case, for the tire 101 determined to be a non-defective product in the first sorting process S2, the defect 200 is also retrieved in the second sorting process S3, improving the detection accuracy of defective products. Additionally, the tire 101 determined to be a defective product in the first sorting process S2 and the second sorting process S3 can be treated as a defective product, or the tire 101 determined to be a defective product in the first sorting process S2 or the second sorting process S3 can be treated as a defective product.

[0057] In these cases, the features of the defect 200 extracted from the image of the tire 101 that was sorted as a non-defective product estimation image in the first sorting process S2 and then detected as a defective product in the second sorting process S3 can be accumulated in the defect folder 57. With such a configuration, the features of a more diverse range of defects 200 are accumulated in the defect folder 57, further improving the detection accuracy of defective products.

[0058] On the other hand, the image of the tire 101 determined to be a non-defective product in the second sorting process S3 can be added to the images of non-defective tires in the single-classification learning of the first sorting process S2. With such a configuration, the sorting accuracy of the images in the first sorting process S2 can be further improved.

[0059] Figure 6 Represents the imaging area 103 of the tire 101 captured by the camera 21. In this figure, the imaging area 103 is depicted with dot-patterned hatching. In the present embodiment, in the imaging process S1, the annular imaging area 103 of the sidewall portion 102 of the tire 101 is imaged. Thereby, it is possible to easily inspect the sidewall portion 102 of the tire 101 for appearance defects. In this case, preferably, in the imaging process S1, the above-mentioned annular imaging area 103 is transformed into a strip shape.

[0060] Figure 7 Represents the image of the sidewall portion 102 transformed into a strip shape. The transformation of the image is executed by the processing unit 41, and the transformed image is stored in the storage unit 42. By transforming the annular image into a strip shape in the imaging process S1, the scrolling, etc. of the image in the display device becomes easier, and the operator can easily confirm appearance defects.

[0061] In addition, in the photographing process S1, it is preferable to perform a masking process on the peak marks, light point marks, etc. of the uniformity applied to the sidewall portion 102. In the present embodiment, when the object 100 to be inspected is the tire 101, the regions corresponding to the peak marks and light point marks, etc. in the image captured by the camera 21 are colored the same color as the tire 101 itself, for example, black, to perform the masking. Since these marks are special parts in terms of color but not defects of the tire 101, it is not desired to be detected as abnormal. Thus, in the present embodiment, by performing the above masking process, the presence of the above marks is ignored in the image in the first sorting process S2 and the second sorting process S3, and the misdetection of defective products is suppressed.

[0062] On the other hand, in the present embodiment, it is configured to change the position of the camera 21 to photograph the tread surface 105 of the tire 101 (refer to Figure 2 ). With such a configuration, it is also possible to inspect the appearance of the tread surface 105 for defects.

[0063] A 2D camera, or a 3D camera, or a 2D camera and a 3D camera are applied to the camera 21. According to the 2D camera, in the first sorting process S2 and the second sorting process S3, the defect 200 is mainly detected from the viewpoint of abnormalities related to color. On the other hand, according to the 3D camera, in the first sorting process S2 and the second sorting process S3, the defect 200 is mainly detected from the viewpoint of abnormalities related to shape. By combining the two, the defect 200 is detected from the viewpoints of abnormalities related to color and shape.

[0064] As described above, the appearance inspection method and the like of the present invention have been described in detail, but the present invention is not limited to the above specific embodiments and can be implemented in various modified ways.

Claims

1. A method for inspecting the appearance of an object to be inspected, the method comprising: a photographing step in which a plurality of the inspected objects are photographed to obtain images; a first sorting step, in which the image is sorted into an image of a qualified product estimated within a predetermined standard and an image of a defective product estimated outside the standard using a single-class learning algorithm; and a second sorting step, in which a defect feature is extracted from the defective product estimation image using an object detection algorithm, the image data acquired in the photographing step is compared with the defect feature from the defective product estimation image sorted in the first sorting step, and the feature is retrieved in the image, thereby detecting defective products from the inspected object, As the inspection progresses, the images captured in the capturing step are accumulated, the number of images estimated to be defective products obtained in the first sorting step increases, and the number of features of defects extracted from the images estimated to be defective products in the second sorting step also increases.

2. The appearance inspection method according to claim 1, wherein: In the second sorting step, the features are classified and accumulated for each type of the defects.

3. The appearance inspection method according to claim 1 or 2, wherein: In the second sorting step, the features extracted from the image of the inspection object detected as the defective product in the second sorting step after being sorted as the conforming product estimated image in the first sorting step are accumulated.

4. The appearance inspection method according to claim 1 or 2, wherein: The object to be inspected is a tire.

5. The appearance inspection method according to claim 4, wherein: In the imaging step, an annular imaging region of the tire side portion is imaged.

6. The appearance inspection method according to claim 5, wherein: In the imaging step, the imaging area is transformed into a strip shape.

7. A visual inspection device for inspecting the appearance of an object to be inspected, the visual inspection device comprising: an imaging unit that captures a plurality of the inspection objects to obtain images; a storage unit for storing the image; a first sorting unit that uses a single-class learning algorithm to sort the images into images estimated to be good products within a predetermined standard and images estimated to be bad products outside the standard; and a second sorting unit, which extracts features of defects from the defective product estimation image using an object detection algorithm, compares the image data acquired by the photographing unit with features of defects from the defective product estimation image selected by the first sorting unit, and retrieves the features within the image, thereby detecting defective products from the inspected object; As the inspection progresses, the images captured by the capturing unit are accumulated, the number of estimated defective product images selected by the first sorting unit increases, and the number of defect features extracted from the estimated defective product images by the second sorting unit also increases.

8. The visual inspection device according to claim 7, wherein: The second sorting unit classifies the features according to each category of the defects, The storage unit accumulates the features for each of the categories.

9. The appearance inspection device according to claim 7 or 8, wherein the photographing unit includes a 2D camera.

10. The appearance inspection device according to claim 7 or 8, wherein the photographing unit includes a 3D camera.

Images