Inspection method and inspection device

Abstract

To provide an inspection method for improving the accuracy of determining an abnormality in an object which is a pressed component obtained by press work.SOLUTION: An inspection method includes a step of acquiring n normal image vectors each including m elements, a step of acquiring an object image vector including m elements by arranging pixel values of an object image of an object M including m pixels, a step of calculating a regression model using m observation values of n or less of explanatory variables based on the n normal image vectors each including m elements and m observation values of an objective variable based on the object image vector including m elements, a step of calculating an object image reconstruction vector for approximating the object image vector on the basis of a coefficient of the regression model and the n normal image vectors, a step of calculating, as a reconstruction error vector, the difference between the object image vector and the reconstruction vector, and a step of determining the presence / absence of an abnormality in the object on the basis of the value of an element of the reconstruction error vector.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present disclosure relates to an inspection method and an inspection apparatus. [Background technology]

[0002] 2. Description of the Related Art Conventionally, there is known a technique for detecting an abnormality in an object based on an image obtained by capturing an image of the object with a camera.

[0003] For example, Patent Document 1 discloses a method for detecting defects in pressed parts that can automatically and reliably detect defects such as cracks that occur in the press process at low cost. For example, Patent Document 2 discloses a method for detecting defects in pressed parts that can suppress overlooking and erroneous detection of defects that occur in pressed parts after press processing. For example, Patent Document 3 discloses an inspection device that inspects an object for abnormalities including missing parts, incorrect placement, and defects. Such an inspection device relaxes restrictions such as secure fixation of the object and camera and highly accurate alignment of each pixel of image data acquired by capturing an image of the object. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2006-177892 A [Patent Document 2] JP 2009-294189 A [Patent Document 3] Patent No. 6241576 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the conventional techniques have not been able to provide sufficient accuracy in determining an abnormality.

[0006] The present disclosure aims to provide an inspection method and an inspection device capable of improving the accuracy of determining an abnormality in an object that is a pressed part produced by press working. [Means for solving the problem]

[0007] An inspection method according to an embodiment of the present disclosure includes: 1. An inspection method for inspecting an object, which is a pressed part that has been subjected to press processing, based on an image obtained by imaging the object, comprising: A step of using a vector obtained by arranging pixel values ​​of a normal image including m pixels as a sequence of explanatory variables, and obtaining n normal image vectors each including m elements using n normal images; ordering pixel values ​​of an object image of the object comprising m pixels to obtain an object image vector comprising m elements; A step of calculating a regression model using m observed values ​​of n or less explanatory variables based on the n normal image vectors each including m elements, and m observed values ​​of a response variable based on the object image vector each including m elements; calculating a reconstruction vector of the object image that approximates the object image vector based on the coefficients of the regression model and the n normal image vectors; calculating a difference between the object image vector and the reconstruction vector as a reconstruction error vector; determining whether or not there is an abnormality in the object based on values ​​of elements of the reconstruction error vector; Includes.

[0008] An inspection apparatus according to an embodiment of the present disclosure includes: An inspection apparatus for inspecting an object, which is a pressed part that has been subjected to press processing, based on an image obtained by imaging the object, comprising: A control unit is provided, The control unit is A vector obtained by arranging pixel values ​​of a normal image containing m pixels is used as a sequence of explanatory variables, and n normal image vectors each containing m elements are obtained using n normal images. arranging pixel values ​​of an object image of the object, the object image vector including m elements; Calculating a regression model using m observed values ​​of n or less explanatory variables based on the n normal image vectors each including m elements, and m observed values ​​of a response variable based on the object image vector each including m elements; calculating a reconstruction vector of the object image that approximates the object image vector based on the coefficients of the regression model and the n normal image vectors; Calculating a difference between the object image vector and the reconstruction vector as a reconstruction error vector; The presence or absence of an abnormality in the object is determined based on the values ​​of the elements of the reconstruction error vector. Effect of the Invention

[0009] According to an inspection method and an inspection device according to an embodiment of the present disclosure, it is possible to improve the accuracy of determining an abnormality in an object that is a pressed part produced by press working. [Brief description of the drawings]

[0010] [Figure 1] 1 is a schematic diagram illustrating a configuration of an inspection device according to an embodiment of the present disclosure. [Diagram 2] 2 is a schematic diagram for explaining an example of the operation of the inspection device of FIG. 1. FIG. [Diagram 3] 2 is a flowchart showing an example of an inspection method executed by the inspection device of FIG. 1. [Figure 4A] FIG. 2 is a diagram showing a first example of a normal image of an object. [Figure 4B] FIG. 11 is a diagram showing a second example of a normal image of an object. [Figure 4C] FIG. 13 is a diagram showing a third example of a normal image of an object. [Figure 4D] FIG. 13 is a diagram showing a fourth example of a normal image of an object. [Figure 5A] FIG. 13 is a diagram showing a first principal component extracted from a normal image by principal component analysis. [Figure 5B]FIG. 13 is a diagram showing a second principal component extracted from a normal image by principal component analysis. [Figure 5C] FIG. 13 is a diagram showing a third principal component extracted from a normal image by principal component analysis. [Figure 5D] FIG. 13 is a diagram showing a fourth principal component extracted from a normal image by principal component analysis. [Figure 6] FIG. 2 is a diagram showing an example of an object image of an object to be inspected; [Figure 7] FIG. 7 is a diagram showing an example of a reconstructed image for the object image in FIG. 6. [Figure 8A] 7 is a diagram showing an example of a first reconstruction error image for the object image in FIG. 6. FIG. [Figure 8B] 7 is a diagram showing an example of a second reconstruction error image for the object image in FIG. 6. FIG. [Figure 9A] 8B is a diagram showing an example of an image obtained by binarizing the first reconstruction error image in FIG. 8A. FIG. [Figure 9B] 8C is a diagram showing an example of an image obtained by binarizing the second reconstruction error image in FIG. 8B. FIG. [Figure 10] FIG. 8B is a graph showing a numerical example of the object image vector and the reconstruction vector associated with FIG. 8A. [Figure 11] FIG. 8B is a graph showing an example of numerical values ​​of the first reconstruction error vector related to FIG. 8A. [Figure 12] FIG. 8C is a graph illustrating an example of numerical values ​​of the second reconstruction error vector related to FIG. 8B. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] The background and problems of the prior art will now be described in more detail.

[0012] In the process of manufacturing automobile bodies, the body is often formed as a pressed product (hereinafter also referred to as a "pressed part") by pressing a metal plate such as a thin steel plate or aluminum plate. The press working is performed by using a press machine equipped with a pair of dies including a male and a female die to pressurize a metal plate placed between the pair of dies. The press working is used as a processing method for mass-producing parts of the same shape, such as car bodies.

[0013] However, in the manufacture of pressed products, defects including cracks and fractures may occur in pressed parts due to variations in the material properties of the metal plate used as the material, fluctuations in the pressure conditions of the press device, and other reasons. In addition, when pressed parts are continuously produced at high speed by press processing, heat (hereinafter referred to as "processing heat") is generated in the pressed parts and the dies used in the press processing. When processing heat is generated, the possibility of defects occurring in the pressed parts increases due to deterioration of the lubricant applied to the surfaces of the dies and materials, thermal deformation occurring in the dies, and other reasons. The causes of processing heat change depending on the season, processing conditions, and other factors. Therefore, it has been difficult to predict whether defects will occur in pressed parts.

[0014] In recent years, defects occurring in pressed parts have been greatly reduced due to improvements in material quality and advances in press processing technology, but it is difficult to completely prevent their occurrence. In the case of pressed parts for automobile bodies, even if the defects are only a few millimeters in size, it is impossible to use the pressed parts as products. Therefore, defects occurring in pressed parts are a serious problem. In particular, if defects occur continuously in pressed parts due to material defects including variations in material properties, and the parts are used as they are in the subsequent process of assembling the body, the losses in labor costs, material costs, etc., i.e., monetary losses, are very large.

[0015] For this reason, for pressed parts that are prone to defects due to press working, a process of visually inspecting the pressed parts individually may be carried out. However, the time required for processing each pressed part formed by press working is short, and they are produced in batches of, for example, about 500 pieces per hour. Therefore, it is very difficult to visually inspect all of these pressed parts. Also, there is a possibility that defects that occur in pressed parts may be overlooked in visual inspection.

[0016] In order to solve these problems, Patent Document 1 proposes a method and device for detecting defects in pressed parts. The method for detecting defects in pressed parts described in Patent Document 1 detects part defects such as cracks by analyzing the heat distribution on the part after press forming. More specifically, the defect detection method includes the steps of capturing infrared rays emitted from the part surface immediately after press forming with an infrared imaging device, analyzing the temperature distribution on the part surface that has changed due to the processing heat during press forming, and detecting defects due to press processing by comparing with a preset reference temperature distribution.

[0017] However, due to the processing heat generated by the press process, the temperature of the die rises each time the press process is repeated. Therefore, the temperature of the pressed parts after press processing rises toward the latter half of the lot. In addition, the processing heat is greatly affected by the ambient temperature. Therefore, when there is a large difference in the ambient temperature, for example, between summer and winter, a large difference occurs in the temperature distribution of the processing heat on the surface of the part. As a result, it is difficult to set a reference temperature distribution corresponding to such fluctuations, and when defects are detected based on the difference between the temperature distribution of the pressed part after press forming and the reference temperature distribution, problems such as overlooking defects or erroneously detecting non-defective defects tend to occur.

[0018] In response to this, Patent Document 2 proposes a method for detecting defects in pressed parts, including the steps of detecting a temperature abrupt change region in the surface temperature distribution of the pressed part after press working, where the temperature has changed by a predetermined temperature difference or more from the surrounding regions, by using spatial second-order differential processing, and determining whether or not a defect has occurred in the pressed part based on the detected temperature abrupt change region. In the defect detection method described in Patent Document 2, the surface temperature distribution is obtained by capturing an image of the pressed part after press working with an infrared camera.

[0019] There is a known technology for detecting abnormalities in such an object based on an image obtained from a camera that captures the object using electromagnetic waves including visible light, ultraviolet light, infrared light, X-rays, etc. Such a technology is important for managing the quality of final or intermediate products, and is widely used.

[0020] Conventionally, in order to detect an abnormality in an image, the fact that the brightness, color tone, texture, etc. of an abnormal part exhibits characteristics different from those of the other parts, i.e., normal parts, is utilized. Conventionally, a method is known for determining the presence or absence of an abnormal part and identifying the position of the abnormal part based on the difference in pixel value between the abnormal part and the normal part. For example, when the brightness of the abnormal part is lower than that of the normal part, it is possible to detect a part having a pixel value smaller than a predetermined threshold as an abnormal part.

[0021] However, such methods have problems, such as not being able to detect abnormal areas (non-detection) or judging normal areas as abnormal (over-detection) when the average level of the pixel values ​​of the image fluctuates due to, for example, fluctuations in the temperature of the object.

[0022] Another method is known in which an image of an object not containing an abnormality is used as a reference image, an image including the difference between the reference image and the image of the object to be inspected is calculated, and a portion in which the pixel value exceeds a predetermined threshold is detected as an abnormal portion. However, such a method has a problem that, for example, when the average level of the pixel value of the image fluctuates due to a change in the temperature of the object, the difference in level with the reference image becomes large, resulting in overdetection. Another problem is that the position of the object in the reference image and the object in the image to be inspected must be accurately aligned.

[0023] In contrast, the inspection device described in Patent Document 3 performs dimensional compression to reduce the dimension of image data of an object that does not contain anomalies. The inspection device calculates parameters that represent the properties of the image data of the object that does not contain anomalies. The inspection device performs dimensional compression on the image data of the object to be inspected using the parameters. The inspection device generates restored data by restoring the dimensionally compressed image data of the object to be inspected. The inspection device determines whether or not the object to be inspected is abnormal based on the magnitude of the difference between the image data of the object to be inspected and the restored data.

[0024] However, the method for detecting defects in pressed parts described in Patent Document 2 has the following problems.

[0025] In order to accurately determine the presence or absence of defects in a pressed part, it is necessary to appropriately set a binarization threshold used when detecting the presence or absence of a temperature change in a temperature sudden change area. Patent Document 2 describes the use of the proposed defect detection method for each area into which the surface temperature distribution is divided, but in this case, the binarization threshold needs to be appropriately set for each area. Therefore, a problem has arisen in that a heavy adjustment work is required. As a result, problems have arisen in areas where the binarization threshold is not appropriate, such as defects being overlooked (non-detection) or normal parts being determined to be defective (over-detection).

[0026] Furthermore, when using a material with little processing heat due to press working, such as an aluminum plate, even if there are defects, the amount of temperature change around them is small, and there has been a problem that defects cannot be detected by the method based on detecting the surface temperature distribution.

[0027] In addition, the inspection apparatus described in Patent Document 3 has the following problems because it performs dimensional compression to reduce the dimension of the image data of an object without abnormalities.

[0028] As a method of dimensional compression, the principal component analysis described in Patent Document 3 will be described. Let the number of data of normal images to be dimensionally compressed be N, and the total number of pixels for each image data be K. For example, the value of N is 100, 1000, etc., and the value of K is 307,200 if the image is 640×480 pixels. The inspection apparatus described in Patent Document 1 calculates the principal components of d dimensions by principal component analysis. Then, since d≦K, the N-dimensional image data is compressed into d dimensions by discarding d + 1 dimensions or more.

[0029] The inspection apparatus described in Patent Document 3 uses the value of each pixel in the image data as a variable and each image data as an observed value. However, as described above, generally, the number of pixels K of the image data is much larger than the number of image data N. In this case, the number of variables K in the principal component analysis becomes much larger than the observed value N, and rank deficiency of the variance-covariance matrix in the principal component analysis occurs. That is, the number of principal components that can be used is d or less that satisfies d≦N<K.

[0030] Furthermore, in the part of the image showing other than the object, that is, the background part, it often shows a constant value such as 0 in any image. In such a case, the variables for the pixel value become the same value for all the observed values. If there are a plurality of such pixels, further rank deficiency occurs due to multicollinearity. As a result, the principal component analysis is not correctly performed, and sufficient accuracy regarding the abnormality determination of the object has not been obtained.

[0031] The inspection device described in Patent Document 3 compresses images, resulting in the loss of detailed information contained in the original image by reducing the amount of detailed data. Therefore, the inspection device described in Patent Document 3 has difficulty detecting small abnormalities such as minute scratches on the target object. Therefore, sufficient accuracy in determining abnormalities in the target object has not been achieved.

[0032] In order to solve the above problems, the present disclosure aims to provide an inspection method and an inspection device capable of improving the accuracy of abnormality determination of an object that is a pressed part produced by press working. The present disclosure relates to an inspection method and an inspection device that detects abnormalities in an object from an image of the object to be inspected. The present disclosure is applicable to inspection objects such as pressed parts formed by pressing a metal plate, such as an automobile body. Hereinafter, an embodiment of the present disclosure will be mainly described with reference to the attached drawings.

[0033] (composition) 1 is a schematic diagram showing the configuration of an inspection device 10 according to an embodiment of the present disclosure. The configuration of the inspection device 10 according to an embodiment of the present disclosure will be mainly described with reference to FIG.

[0034] The inspection device 10 is connected to the production management system 1. The inspection device 10 acquires an image of an object M captured by a camera 2 including an infrared camera or the like. The object M is produced by the production management system 1. The inspection device 10 inspects the object M to be inspected based on the image. More specifically, the inspection device 10 detects an abnormality in the object M based on the image. In this specification, "an abnormality in the object M" includes, for example, scratches, cracks, stains, deformations, defects, and the like of the object M.

[0035] In this specification, the "object M" refers to products including final products and intermediate products that are industrially mass-produced, and which have small variations in the manufactured products, are substantially identical in shape, color, temperature, etc., and have small deviations in position and angle when photographed. More specifically, the object M is a pressed part that has been subjected to press processing.

[0036] The inspection device 10 has an image input unit 11, a memory unit 12, an input unit 13, an output unit 14, and a control unit 15.

[0037] The image input unit 11 includes any module capable of acquiring an image of the object M output from the camera 2. For example, the image input unit 11 includes a communication module compatible with the communication standard of the camera 2. The inspection device 10 is connected to the camera 2 via the image input unit 11. The image input unit 11 acquires an image of the object M that is captured by the camera 2 and is produced by the production management system 1. The image input unit 11 outputs the acquired image to the control unit 15.

[0038] The memory unit 12 is, for example, but not limited to, a semiconductor memory, a magnetic memory, or an optical memory. The memory unit 12 functions as a main memory device, an auxiliary memory device, or a cache memory. The memory unit 12 stores any information used in the operation of the inspection device 10. The memory unit 12 stores system programs, application programs, images acquired by the image input unit 11, etc. The memory unit 12 stores the results of calculations and judgments executed by the control unit 15 as information. The memory unit 12 has an explanatory variable storage unit 121 and a result storage unit 122 as functional blocks indicating part of its functions.

[0039] The input unit 13 includes one or more input modules that detect a user input and obtain input information based on a user operation. For example, the input unit 13 includes physical keys, capacitive keys, a touch screen that is integrated with the display of the output unit 14, a microphone that accepts voice input, and the like.

[0040] The output unit 14 includes one or more output modules that output information to notify a user. For example, the output unit 14 includes a display that outputs information as a video, a speaker that outputs information as a sound, a rotating light that outputs visual information that an abnormality has been detected in the object M, and the like.

[0041] The control unit 15 includes one or more processors. In one embodiment, the "processor" is a general-purpose processor or a dedicated processor specialized for a specific process, but is not limited thereto. The control unit 15 is communicably connected to each component of the inspection device 10 and controls the operation of the entire inspection device 10. The control unit 15 appropriately controls the timing and settings of the processing of each component of the inspection device 10. The control unit 15 has an explanatory variable generation unit 151, an image analysis unit 152, a result determination unit 153, and an input / output interface 154 as functional blocks showing some of its functions.

[0042] (Operation / Function) In the following, the operation and functions of the inspection device 10 will be mainly described.

[0043] 1, an object M to be inspected is, for example, a press part. A camera 2 captures an image of the object M and outputs the obtained image of the object M to an inspection device 10. A control unit 15 of the inspection device 10 acquires the image of the object M from the camera 2 via an image input unit 11.

[0044] The control unit 15 realizes two operations corresponding to a learning mode and an inspection mode as the operation of the inspection device 10. The learning mode is a mode for obtaining explanatory variables of a regression model described later. The control unit 15 needs to execute the learning mode at least once before executing the inspection mode. Thereafter, the control unit 15 may execute the learning mode when it is necessary to add or update explanatory variable data described later. In normal operation, the control unit 15 may mainly execute the inspection mode.

[0045] The inspection mode is a mode in which the object M to be inspected is inspected every time the object M to be inspected arrives within the imaging range of the camera 2. The control unit 15 acquires input information from the input unit 13 via the input / output interface 154, and operates the inspection device 10 according to the input information. The control unit 15 executes a switching process between the learning mode and the inspection mode according to the input information. Such input information is acquired by the input unit 13 based on an input operation by a user as an operator of the inspection device 10 using the input unit 13 including, for example, a keyboard.

[0046] When the objects M to be inspected are divided into several groups during the operation of the inspection device 10 in the learning mode, the control unit 15 generates explanatory variable data for each group. The control unit 15 acquires information on the objects M currently being produced from the production management system 1 connected to the inspection device 10. The information on the objects M includes, for example, the group name to which the objects M belong. The control unit 15 appropriately performs learning for each group.

[0047] When the object M arrives within the imaging range of the camera 2, the control unit 15 outputs a signal to the camera 2 via the image input unit 11. The camera 2 captures the object M at that timing and outputs the obtained image to the inspection device 10. The explanatory variable generation unit 151 of the control unit 15 acquires the obtained image from the camera 2 via the image input unit 11. The explanatory variable generation unit 151 generates explanatory variable data from multiple normal images belonging to the same group. The explanatory variable generation unit 151 stores the generated explanatory variable data in the explanatory variable storage unit 121 of the memory unit 12. The explanatory variable storage unit 121 stores such explanatory variable data for each group.

[0048] The reason why the explanatory variable generating unit 151 uses multiple normal images belonging to the same group when generating explanatory variable data is as follows. Referring to examples of normal images shown in Figs. 4A to 4D described later, these normal images are obtained by capturing images of a pressed part after press working with an infrared camera as the camera 2. These normal images represent the surface temperature distribution of the pressed part as a grayscale image. In this specification, the "surface" is, for example, the surface located on the outermost side in the plate thickness direction of the object M. The "surface temperature distribution" means, for example, the temperature distribution on such a surface.

[0049] The temperature of the pressed parts after press working increases toward the end of the lot because the temperature of the die rises due to the processing heat generated during press working. In addition, the surface temperature of the pressed parts increases in areas that are subject to high stress during press working and areas that experience severe friction with the die. Therefore, the surface temperature distribution of the pressed parts varies slightly for each pressed part. Therefore, multiple normal images are used to suppress the effects of this variation. The number of normal images required is adjusted taking into account the magnitude of the variation. The greater the variation, the greater the number of normal images required.

[0050] The explanatory variable generating unit 151 generates explanatory variable data from normal image vectors obtained by converting a plurality of normal images into vectors. Fig. 2 is a schematic diagram for explaining an example of the operation of the inspection device 10 in Fig. 1. As shown in Fig. 2, the explanatory variable generating unit 151 converts a normal image into a vector by arranging pixel values ​​included in the image according to a set rule and making them into vector elements.

[0051] For example, the explanatory variable generation unit 151 converts an image in which a pixels are arranged horizontally and b pixels are arranged vertically into a vector with the top left pixel as the first element and the bottom right pixel as the last element. The explanatory variable generation unit 151 can set such rules arbitrarily, but applies the same rules to all normal images and images of the object M to be inspected. If m=a×b and the number of normal images is n, then n normal image vectors each containing m elements are obtained. The explanatory variable generation unit 151 acquires these as explanatory variable data.

[0052] 1 again, when the inspection device 10 operates in the inspection mode, the control unit 15 acquires information on the object M currently being produced and to be inspected from the production management system 1 connected to the inspection device 10. The information on the object M to be inspected includes, for example, the group name to which the object M to be inspected belongs. The image analysis unit 152 of the control unit 15 reads out explanatory variable data corresponding to the object M currently being inspected from among the explanatory variable data stored in the explanatory variable storage unit 121.

[0053] When the object M to be inspected arrives within the imaging range of the camera 2, the control unit 15 outputs a signal to the camera 2 via the image input unit 11. The camera 2 captures the object M at that timing and outputs the obtained image to the inspection device 10. The image analysis unit 152 of the control unit 15 acquires the obtained image from the camera 2 via the image input unit 11.

[0054] The image analysis unit 152 converts an image of the object M to be inspected into a vector to generate an object image vector. The image analysis unit 152 uses the generated object image vector as a response variable and calculates a regression model using explanatory variable data acquired from the explanatory variable storage unit 121. The regression model includes, for example, a linear regression model.

[0055] The image analysis unit 152 calculates a reconstruction vector that approximates the object image vector by such linear regression analysis, and further calculates the difference between the object image vector and the reconstruction vector as a reconstruction error vector. The image analysis unit 152 outputs the calculated reconstruction error vector to the result determination unit 153.

[0056] The result determination unit 153 determines the presence or absence of an abnormality in the object M to be inspected, based on the reconstruction error vector calculated by the image analysis unit 152. The result determination unit 153, for example, compares the values ​​of the elements of the reconstruction error vector with a preset first threshold. The result determination unit 153 determines whether the number of elements equal to or greater than the first threshold is equal to or greater than a preset second threshold. When the result determination unit 153 determines that the number of elements equal to or greater than the first threshold is equal to or greater than the second threshold, it identifies the object M as having an abnormality.

[0057] The result determination unit 153 stores the determination result in the result saving unit 122 of the memory unit 12. The result determination unit 153 displays the determination result on the display of the output unit 14 via the input / output interface 154. In addition, when the result determination unit 153 determines that there is an abnormality in the object M, it may sound an alarm using the speaker of the output unit 14 or may turn on a rotating light of the output unit 14. In this way, the result determination unit 153 notifies the user as the operator of the inspection device 10 that an abnormality has been detected in the object M to be inspected.

[0058] Fig. 3 is a flowchart showing an example of an inspection method executed by the inspection device 10 of Fig. 1. A main flow of the inspection method using the inspection device 10 will be described with reference to Fig. 3. In such an inspection method, the object M is inspected based on an image obtained by capturing an image of the object M.

[0059] In step S100, the control unit 15 of the inspection device 10 acquires a plurality of normal images obtained by capturing an image of the object M from the camera 2 via the image input unit 11.

[0060] In step S101, the control unit 15 treats a vector obtained by arranging the pixel values ​​of a normal image containing m pixels as a series of explanatory variables, and obtains n normal image vectors, each containing m elements, using the n normal images obtained in step S100.

[0061] In step S102, the control unit 15 acquires an object image obtained by capturing an image of the object M to be inspected from the camera 2 via the image input unit 11.

[0062] In step S103, the control unit 15 acquires the surface temperature distribution of the object M from the object image acquired in step S102.

[0063] In step S104, the control unit 15 arranges pixel values ​​of the object image of the object M including m pixels to obtain an object image vector including m elements.

[0064] In step S105, the control unit 15 calculates a regression model using m observed values ​​of n or less explanatory variables based on n normal image vectors each including m elements obtained in step S101, and m observed values ​​of a dependent variable based on the object image vector each including m elements obtained in step S104.

[0065] In step S106, the control unit 15 calculates a reconstruction vector of the object image that approximates the object image vector, based on the coefficients of the regression model calculated in step S105 and the n normal image vectors acquired in step S101.

[0066] In step S107, the control unit 15 calculates the difference between the object image vector acquired in step S104 and the reconstruction vector calculated in step S106 as a reconstruction error vector.

[0067] In step S108, the control unit 15 determines whether there is an abnormality in the object M to be inspected based on the values of the elements of the reconstruction error vector calculated in step S107. If the control unit 15 determines that there is no abnormality, it executes the process of step S109. If the control unit 15 determines that there is an abnormality, it executes the process of step S110.

[0068] In step S109, when the control unit 15 determines that there is no abnormality in step S108, it causes the output unit 14 to output the determination result in step S108. For example, the control unit 15 causes the display of the output unit 14 to display the determination result.

[0069] In step S110, when the control unit 15 determines that there is an abnormality in step S108, it notifies the user that an abnormality has been detected in the object M to be inspected using the output unit 14.

[0070] In step S111, the control unit 15 determines whether the production of the object M has been completed based on the information output from the production management system 1. If the control unit 15 determines that the production has been completed, it ends the process. If the control unit 15 determines that the production has not been completed, it executes the process of step S102 again.

[0071] (First Embodiment) The inspection method according to the first embodiment of the present disclosure will be described in more detail with reference to FIG. 3.

[0072] In the inspection method according to the first embodiment, the control unit 15 further includes a step of calculating an intermediate variable vector, for example, between step S101 and step S102 of FIG. 3. The explanatory variable generation unit 151 of the control unit 15 calculates the weights to be multiplied by the n normal image vectors acquired in step S101 using, for example, any one of principal component analysis, independent component analysis, and partial least squares method. The explanatory variable generation unit 151 calculates d (d < n) intermediate variable vectors by weighting and adding the n normal image vectors acquired in step S101 using the calculated weights.

[0073] In the step of calculating the regression model in step S105 of FIG. 3, the image analysis unit 152 of the control unit 15 regards the d intermediate variable vectors obtained between step S101 and step S102 as the m observed values of the d explanatory variables. Similarly, the image analysis unit 152 regards the object image vector including the m elements acquired in step S104 as the m observed values of the target variable.

[0074] In the step of calculating the reconstruction vector in step S106 of FIG. 3, the image analysis unit 152 calculates the reconstruction vector by weighting and adding the d intermediate variable vectors with the coefficients of the regression model calculated in step S105.

[0075] FIG. 4A is a diagram showing a first example of a normal image of the object M. FIG. 4B is a diagram showing a second example of a normal image of the object M. FIG. 4C is a diagram showing a third example of a normal image of the object M. FIG. 4D is a diagram showing a fourth example of a normal image of the object M.

[0076] As described above, it is necessary to use normal image vectors obtained from a sufficient number of normal images that can suppress the influence of fluctuations in the normal images as the explanatory variables. However, if the number of explanatory variables increases accordingly, it may hinder the calculation of the regression model in the inspection mode. For example, the load of the arithmetic processing by the control unit 15 may increase, and the processing time required for the abnormality determination of the object M may become longer.

[0077] Therefore, the control unit 15 weights and adds n normal image vectors using the weights calculated in advance, and calculates d intermediate variable vectors (d < n). The control unit 15 uses the d intermediate variable vectors as the data of the d explanatory variables. Hereinafter, as an example, the content of the operation by the control unit 15 when principal component analysis is used in calculating the intermediate variable vectors will be described in detail.

[0078] Equation 1 represents a normal image vector obtained by vectorizing a normal image including m pixels.

[0079]

number

[0080] Each x in Equation 1 i If we arrange these data horizontally to construct a matrix X containing all the data as shown in Equation 2 below, the covariance matrix V of matrix X is given by Equation 3.

[0081]

number

[0082] Here, the i-th largest eigenvalue of the covariance matrix V is λ i , and the corresponding normalized eigenvector is w i Then, the following relationship holds:

[0083]

number

[0084] w i is the coupling coefficient of the i-th principal component. In other words, the principal components are found by solving the eigenvalue problem of the covariance matrix V.

[0085] The vector obtained by extracting the p-th sample value of the matrix X is expressed as shown in Equation 6.

[0086]

number

[0087] z p The value of the i-th principal component for t pi (Principal component score) is expressed as in Equation 7.

[0088]

number

[0089] If we combine Equation 7 into one vector for the m observations of Equation 1 as in Equation 8, Equation 9 holds.

[0090]

number

[0091] From Equation 9, t i It can be seen that the normal image vector included in the matrix X is a linear sum obtained by multiplying the normal image vector by the combination coefficient and adding the result. In this disclosure, the control unit 15 uses t i (i=1,2,...,d), i.e., the magnitudes of the first to dth principal components (principal component scores) are used as explanatory variables of the regression model in the inspection mode. d is a number smaller than n. Therefore, the data volume is compressed to d / n. In general, a normal image vector contains many pieces of data that are similar to each other, resulting in redundancy. The control unit 15 extracts the main components by the above-mentioned principal component analysis and compresses the information to efficiently calculate the regression model.

[0092] FIG. 5A is a diagram showing a first principal component extracted from a normal image by principal component analysis. FIG. 5B is a diagram showing a second principal component extracted from a normal image by principal component analysis. FIG. 5C is a diagram showing a third principal component extracted from a normal image by principal component analysis. FIG. 5D is a diagram showing a fourth principal component extracted from a normal image by principal component analysis. FIGs. 5A to 5D respectively show the first to fourth principal components extracted by principal component analysis from a sample of 56 normal images including the example normal images shown in FIGs. 4A to 4D.

[0093] The first principal component shown in Figure 5A represents the component most common to normal images. The second, third, and fourth principal components shown in Figures 5B, 5C, and 5D, respectively, represent the variation of normal image samples. Among these components, the lower the order component, the more major the variation, and the higher the order component, the more minor the variation. The fifth and subsequent principal components correspond to even more minor variations.

[0094] Normally, a regression model with sufficient accuracy can be calculated by using up to the 10th principal component. However, when the variation of normal image samples is large, it is possible to maintain the accuracy of the regression model against the variation of normal images by using a sufficient number of samples that represent the distribution of the variation and by using higher-order principal components, such as up to the 20th principal component.

[0095] The following mainly describes the calculation of the regression model by the image analysis unit 152 in the inspection mode. The object image vector that is the response variable of the linear regression is expressed by Equation 10.

[0096]

number

[0097] The image analysis unit 152 uses an intermediate variable vector of Equation 12 obtained by weighting and adding the normal image vector expressed by Equation 11 as explanatory variable data of the linear regression.

[0098]

number

[0099] From the intermediate variable vector of Equation 12 obtained by the principal component analysis, the explanatory variable T is expressed by Equation 13.

[0100]

number

[0101] The image analysis unit 152 obtains the linear regression equation of Equation 15, which is obtained by multiplying the explanatory variable T of Equation 13 by the coefficient b of Equation 14 and adding the result, as a predicted value of y in Equation 10.

[0102]

number

[0103] Equation 16 represents the reconstruction vector that approximates the object image vector.

[0104] In the step of calculating the reconstruction error vector in step S107 in Fig. 3, the image analysis unit 152 calculates at least one of the first reconstruction error vector and a second reconstruction error vector different from the first reconstruction error vector. The first reconstruction error vector has a large value in a portion where the object image vector is lower in temperature than the reconstruction vector in the surface temperature distribution acquired in step S103 in Fig. 3. The second reconstruction error vector has a large value in a portion where the object image vector is higher in temperature than the reconstruction vector in the surface temperature distribution. In healthy portions, the object image vector is reproduced with high accuracy by the regression model, so that the values ​​of the elements of the reconstruction error vector become small. Therefore, a portion where the first reconstruction error vector has a large value means that the temperature is lower than the surrounding healthy portions. A portion where the second reconstruction error vector has a large value means that the temperature is higher than the surrounding healthy portions.

[0105] For example, when detecting a crack in a pressed part as the object M, the image analysis unit 152 uses the first reconstruction error vector because the temperature of a void caused by the crack is lower than the temperature of a sound part. The first reconstruction error vector is obtained by subtracting the object image vector of Equation 10 from the reconstruction vector of Equation 16, and is calculated by Equation 17.

[0106]

number

[0107] For example, the image analysis unit 152 uses the second reconstruction error vector when detecting an abnormality in a part that has become hotter than the surrounding area due to processing heat, sliding between the pressed part and a die, etc. in a pressed part as the object M. The second reconstruction error vector is obtained by subtracting the reconstruction vector of Equation 16 from the object image vector of Equation 10, and is calculated by Equation 18.

[0108]

number

[0109] The image analysis unit 152 calculates at least one of the first reconstruction error vector of Equation 17 and the second reconstruction error vector of Equation 18, and outputs it to the result determination unit 153 as information.

[0110] FIG. 6 is a diagram showing an example of an object image of an object M to be inspected. In the example of FIG. 6, a crack C is found in the object M. FIG. 7 is a diagram showing an example of a reconstructed image for the object image of FIG. 6. FIG. 7 shows a reconstructed image obtained by converting the reconstruction vector for the object image of FIG. 6 back into an image. Among the multiple normal images, there is no image that includes the crack C of the object M as shown in FIG. 6. Therefore, the portion corresponding to the crack C is not reproduced in the reconstructed image shown in FIG. 7.

[0111] Fig. 8A is a diagram showing an example of a first reconstruction error image for the object image in Fig. 6. Fig. 8A shows a first reconstruction error image obtained by converting the first reconstruction error vector of Equation 17 for the object image in Fig. 6 into an image. The first reconstruction error image in Fig. 8A shows the difference from the object image in Fig. 6 with respect to the reconstructed image in Fig. 7.

[0112] In the reconstructed image of FIG. 7, the portion of the object image of the object M to be inspected other than the crack C is well approximated. Therefore, in the first reconstructed error image of FIG. 8A, the portion is erased and shown in black. On the other hand, for the crack C in the object image of the object M to be inspected, the pixel value is larger in the reconstructed image of FIG. 7 compared to the object image of FIG. 6, and the portion is shown in white in the first reconstructed error image of FIG. 8A. Therefore, it can be seen that the crack C in the object image of the object M to be inspected has been detected. In this way, since the temperature of the crack C is lower than the surrounding area, the first reconstructed error vector obtained by Equation 17 is used for its detection.

[0113] Fig. 8B is a diagram showing an example of a second reconstruction error image for the object image of Fig. 6. Fig. 8B shows a second reconstruction error image obtained by converting the second reconstruction error vector of Equation 18 for the object image of Fig. 6 into an image. The second reconstruction error image of Fig. 8B shows the difference between the object image of Fig. 6 and the reconstructed image of Fig. 7.

[0114] The second reconstruction error image in FIG. 8B shows that there are high temperature areas around the bottom end of the press part and crack C. This shows that a partial temperature rise occurs due to the deformation and sliding of the press part. In this way, the second reconstruction error vector defined by Equation 18 is used to detect anomalies when the surface temperature of the abnormal part is higher than the surrounding area. In other words, the inspection method using the second reconstruction error vector can capture signs of the occurrence of defects such as crack C, including partial temperature rises due to deformation and sliding of the press part, and is applicable to monitoring of press working.

[0115] Fig. 9A is a diagram showing an example of an image obtained by binarizing the first reconstruction error image in Fig. 8A. Fig. 9A is an image obtained by binarizing the first reconstruction error image in Fig. 8A using a first threshold value. The crack C, which has a lower temperature than the surrounding area, is represented as white pixels, and the other parts are represented as black pixels. When the number of white pixels is equal to or greater than the second threshold value, the result determination unit 153 determines that the image contains an abnormality in the object M, for example, the crack C.

[0116] Fig. 9B is a diagram showing an example of an image obtained by binarizing the second reconstruction error image in Fig. 8B. Fig. 9B is an image obtained by binarizing the second reconstruction error image in Fig. 8B using a first threshold value. High temperature parts such as the lower end part of the pressed part and the periphery of the crack C are shown as white pixels, and other parts are shown as black pixels. When the number of white pixels is equal to or greater than the second threshold value, the result determination unit 153 determines that the image contains an abnormality in the object M, for example, an abnormally high temperature part due to deformation or sliding of the pressed part.

[0117] Here, different criteria may be used for determining whether a part is colder than the surroundings or whether a part is hotter than the surroundings, and the first threshold value may be different for the cases of Figure 9A and Figure 9B, for example. Similarly, the second threshold value may be different for the cases of Figure 9A and Figure 9B, for example.

[0118] An example of the binarization process using the first threshold value to obtain the images shown in FIGS. 9A and 9B will be described in more detail.

[0119] FIG. 10 is a graph showing an example of numerical values ​​of the object image vector and the reconstruction vector related to FIG. 8A. In FIG. 10, the solid line corresponds to the object image vector. The dashed line corresponds to the reconstruction vector. FIG. 10 shows the values ​​of the elements of each vector with respect to the horizontal position on the horizontal dashed line L that overlaps with the position including the abnormality in the first reconstruction error image of FIG. 8A. That is, FIG. 10 shows the extracted elements of the object image vector and the reconstruction vector that correspond to the horizontal position on the horizontal dashed line L, and plotted with the horizontal axis representing the horizontal position and the vertical axis representing the value of the element of each vector.

[0120] Based on the plot of the object image vector shown by the solid line in Fig. 10, it can be seen that the element values ​​of the object image vector in the abnormal area R0 suddenly become small, indicating a low temperature. On the other hand, it can be seen that the element values ​​of the object image vector in the vicinity R1 of the abnormal area R0 are large, indicating a high temperature.

[0121] Detecting a portion of an object image whose pixel value is smaller than a predetermined threshold as an abnormal portion, as in the conventional technology described in paragraph

[0020] above, corresponds to judging a portion of an object image vector whose element value is small as an abnormal portion as shown in FIG. 10 of the present disclosure. In this case, in the plot of the object image vector shown by the solid line in FIG. 10, the value is small even in the healthy portion, region R2, at low temperatures, so it is difficult to appropriately set a threshold value for judging an abnormal portion. From the above, it is difficult to detect and judge an abnormality using only the object image as in the conventional technology. Therefore, in the present disclosure, the result judgment unit 153 detects the abnormal portion R0 using the first reconstruction error vector and the second reconstruction error vector.

[0122] Fig. 11 is a graph showing an example of numerical values ​​of the first reconstruction error vector related to Fig. 8A. Fig. 11 shows a plot of elements of the first reconstruction error vector corresponding to horizontal positions on the horizontal dashed line L in Fig. 8A, with the horizontal axis representing the horizontal position and the vertical axis representing the element values ​​of the first reconstruction error vector.

[0123] 10, in the healthy portion including region R2, the plot of the reconstructed vector shown by the dashed line has substantially the same element values ​​as the plot of the object image vector shown by the solid line. That is, the reconstructed vector closely approximates the object image vector. This indicates that a highly accurate approximation of the object image vector is obtained by the regression model obtained by image analysis unit 152.

[0124] This is also shown in the plot of the first reconstruction error vector shown in FIG. 11. In FIG. 11, the element values ​​of the first reconstruction error vector are approximately 0 in healthy parts. The element values ​​of the first reconstruction error vector fluctuate significantly to the negative side in high temperature parts corresponding to the neighborhood R1 in FIG. 10. The element values ​​of the first reconstruction error vector fluctuate significantly to the positive side in low temperature parts corresponding to the abnormal part R0 in FIG. 10. Therefore, the difference between the healthy part and the abnormal part R0 as a low temperature part becomes clear. The result determination unit 153 can easily detect an abnormality by setting the first threshold T1. In this way, in the present disclosure, the effect of being able to clearly distinguish between the healthy part and the low temperature part corresponding to the abnormal part by using the reconstruction error vector is obtained.

[0125] For example, the result determination unit 153 sets the first threshold T1 to 0.15 for the plot of the first reconstruction error vector shown in Fig. 11. The above-mentioned Fig. 9A shows the first reconstruction error image in Fig. 8A binarized using such a first threshold T1.

[0126] Fig. 12 is a graph showing an example of numerical values ​​of the second reconstruction error vector related to Fig. 8B. Fig. 12 shows a plot of elements of the second reconstruction error vector corresponding to horizontal positions on the horizontal dashed line L in Fig. 8B similar to Fig. 8A, with the horizontal axis representing the horizontal position and the vertical axis representing the value of the element of the second reconstruction error vector.

[0127] The second reconstruction error vector is obtained by inverting the sign of the first reconstruction error vector from the definition of Equation 18. In FIG. 12, the value of the element of the second reconstruction error vector is approximately 0 in a healthy portion. The value of the element of the second reconstruction error vector fluctuates greatly to the positive side in a high temperature portion corresponding to the neighborhood R1 in FIG. 10. The value of the element of the second reconstruction error vector fluctuates greatly to the negative side in a low temperature portion corresponding to the abnormal portion R0 in FIG. 10. Therefore, the difference between the healthy portion and the neighborhood R1 as a high temperature portion becomes clear, but since the absolute value of the fluctuation is smaller than that of the low temperature portion corresponding to the abnormal portion R0, it is necessary to set a first threshold value different from the first threshold value T1 used to detect the low temperature portion. The result determination unit 153 can easily detect the neighborhood R1 as a high temperature portion by setting a first threshold value T2 different from the first threshold value T1. In this way, in the present disclosure, the effect of being able to clearly distinguish a portion that is hotter than a healthy portion from a healthy portion by using a reconstruction error vector is obtained.

[0128] For example, the result determination unit 153 sets the first threshold T2 to 0.06 for the plot of the second reconstruction error vector shown in Fig. 12. The above-mentioned Fig. 9B shows the second reconstruction error image in Fig. 8B binarized using such a first threshold T2.

[0129] (effect) According to the first embodiment described above, it is possible to improve the accuracy of abnormality determination of the object M, which is a press part that has been press-processed. The inspection device 10 regards n normal image vectors, which are an arrangement of pixel values ​​of a normal image including m pixels, as m observed values ​​of n or less explanatory variables, and regards an object image vector including m elements as m observed values ​​of a target variable to calculate a regression model. Images used in such inspections generally contain hundreds of thousands of pixel values. Therefore, the number of observed values ​​m is much larger than the number of explanatory variables n. Therefore, the problem of rank deficiency of the variance-covariance matrix as in the conventional technology described in Patent Document 3 is suppressed. In addition, in the inspection device 10, since the value of each pixel is an observed value, even if there are multiple pixels that take a constant value regardless of the image data, they simply appear at the same position of the vector of the explanatory variable, and the problem of multicollinearity is suppressed.

[0130] In normal image data, pixel values ​​generally vary depending on the data due to positional deviation of the object M in the image, uneven lighting, temperature fluctuations, and the like. In such a case, the inspection device 10 according to the first embodiment can approximate the object image vector with high accuracy by setting the number n of normal image vectors sufficiently large and suppressing the influence of pixel value fluctuations due to data. In addition, the inspection device 10 can reduce the number of explanatory variables by calculating an intermediate variable vector in advance by weighting and adding the normal image vectors, and calculating a regression model with the intermediate variable vector as an explanatory variable and the object image vector as a target variable. Therefore, the inspection device 10 can calculate a reconstruction vector that efficiently approximates the object image vector. As a result, the load of the calculation process in the inspection device 10 is reduced, and the processing time required for abnormality determination is shortened.

[0131] The inspection device 10 generates an intermediate variable vector from the normal image vector and uses it as an explanatory variable of the regression model, thereby reducing the number of explanatory variables and suppressing the influence of the variation of the normal image sample with a small number of explanatory variables. The inspection device 10 can reduce the calculation load when calculating the regression model compared to when the normal image vector is directly used as an explanatory variable. Therefore, the inspection device 10 can be applied to a case where the manufacturing pitch of the press part as the object M to be inspected is short and it is necessary to calculate the regression model in a short time.

[0132] The inspection device 10 can minimize the amount of information lost and compress a large number of normal image vectors into a small number of intermediate variable vectors by applying principal component analysis to calculate weights to be applied to normal image vectors. The method described in Patent Document 3 also uses principal component analysis, but the application method is different from the inspection method by the inspection device 10 according to the first embodiment. In the method described in Patent Document 3, the value of each pixel in the image data is used as a variable, and each image data is used as an observation value. In general, the number of pixels m is much larger than the number of image data n, so there are problems with rank deficiency and multicollinearity of the variance-covariance matrix. In the present disclosure, each image data is used as a variable, and the value of each pixel in the image data is used as an observation value, so such problems are suppressed.

[0133] In addition, in the method described in Patent Document 3, since the value of each pixel is a variable, the principal component analysis compresses information by projecting an m-dimensional space corresponding to the number m of pixels onto a d-dimensional space corresponding to the number d of principal components. A principal component is a linear sum obtained by multiplying m pixel values ​​by coefficients (combination coefficients) and adding them up. Therefore, a total of m×d combination coefficients for the first to d-th principal components are the result of information compression.

[0134] On the other hand, in the inspection device 10 according to the first embodiment, each image data is treated as one variable, and therefore, in the principal component analysis, information compression is performed by projecting an n-dimensional space corresponding to the number n of image data onto a d-dimensional space corresponding to the number d of principal components. The principal component is a linear sum obtained by multiplying n pixel values ​​by coefficients (combination coefficients) and adding them up. Therefore, the total of n×d combination coefficients for the first to d-th principal components are the result of information compression. As a result, the inspection device 10 can compress to an overwhelmingly smaller number of coefficients than the method described in Patent Document 3.

[0135] Since the inspection device 10 compresses the number of image data, not the image, it is possible to suppress the problem of the conventional technology that detailed data contained in the original image is reduced and information is lost. In the method described in Patent Document 3, the image information is compressed by principal component analysis to reduce the number of pixels, so some of the information contained in the original image is lost. On the other hand, the inspection device 10 according to the first embodiment compresses the number of image data, that is, the number of samples of normal images, and the number of pixels is not reduced. Therefore, the information contained in the image is not lost. The reason for using principal component analysis in the present disclosure is to remove redundancy contained in the samples of normal images, not to compress the image itself. As a result, the image resolution is not reduced, so the inspection device 10 can accurately detect small abnormalities such as fine scratches on the object M without overlooking them. In other words, the inspection device 10 can improve the accuracy of abnormality determination for the object M, which is a pressed part that has been pressed.

[0136] The above is due to the fact that, while the data of each normal image is the observed value in the method described in Patent Document 3, the value of each pixel in the image data is the observed value in this disclosure. Due to such differences, the inspection device 10 according to the first embodiment has the following advantages.

[0137] The vector obtained by weighting and adding each normal image vector by the combination coefficient obtained by principal component analysis becomes the principal component value (principal component score) of each observed value. Therefore, it is possible to physically interpret that the first principal component is the most representative value series of normal image vectors, and the second and subsequent principal components represent the fluctuations of normal image vectors and complement the first principal component.

[0138] In addition, if there is a background part outside the object M in the object image, it is necessary to limit the area to be inspected. In this case, in the method described in Patent Document 3, it is necessary to multiply all pixels of the image by the combination coefficient and add them, so it is difficult to set the inspection area after the principal components are obtained. This is because the value of each pixel is a variable, and it is difficult to delete some of the variables after the principal components are obtained. On the other hand, in the present disclosure, since the value of each pixel is an observation value, the inspection device 10 can calculate the principal components even if some of the pixels are deleted. Therefore, the inspection device 10 can set the inspection area later.

[0139] The inspection device 10 can also calculate an intermediate variable vector by applying independent component analysis to the normal image vector, and calculate a regression model using the intermediate variable vector as an explanatory variable and the object image vector as a response variable.

[0140] The inspection device 10 can also calculate an intermediate variable vector by applying the partial least squares method to the normal image vector, and calculate a regression model using the intermediate variable vector as an explanatory variable and the object image vector as a response variable.

[0141] The regression models described above include linear regression models. Unlike neural networks such as deep learning, linear regression models do not require repeated learning. Therefore, the inspection device 10 can calculate a regression model in a very short time. As a result, the inspection device 10 can calculate a new regression model for each newly added object image within a short time until the subsequent new object image is added.

[0142] The intermediate variable vector is a linear sum of normal image vectors. As a result, when the inspection device 10 erroneously determines that there is an abnormality (overdetects) as a result of determining the presence or absence of an abnormality for an object image that does not contain an abnormality, the inspection device 10 adds the object image to the data of the normal image to calculate an intermediate variable vector, and uses the calculated intermediate variable vector to perform subsequent abnormality determinations. Since the intermediate variable vector is a linear sum of normal image vectors, it also includes components of the overdetected normal image vector. Therefore, the inspection device 10 can perform correct determination without overdetection even when a similar object image is subsequently inspected.

[0143] The inspection device 10 acquires the surface temperature distribution of a pressed part as an object M from an image of the pressed part captured by a camera 2 including an infrared camera. This enables the inspection device 10 to detect a portion such as a crack C whose surface temperature is lower than the surrounding area. In addition, the inspection device 10 can also detect signs of the occurrence of defects such as a crack C, including partial temperature increases due to deformation and sliding of the pressed part. Therefore, the inspection device 10 can be applied to monitoring press working.

[0144] The inspection device 10 does not need to divide the image of the object M into regions, as in the conventional technology described in Patent Document 2. Therefore, the inspection device 10 does not need to set parameters such as a binarization threshold for each region. As a result, the inspection device 10 can reduce the adjustment load. Since the inspection device 10 is not premised on detecting defects from a sudden change in the surface temperature distribution, the camera 2 used for imaging is not limited to an infrared camera, and it is also possible to select a camera 2 suitable for imaging defects, such as a visible light camera. Therefore, the inspection device 10 can be applied to materials that generate little processing heat due to press processing.

[0145] For example, in the press processing of an aluminum plate, the amount of processing heat generated is small, and the temperature difference between an abnormal part such as crack C and the surrounding healthy parts is small, so that it may be difficult to detect the abnormal part based on the surface temperature distribution. In such a case, the inspection device 10 can use an image of the surrounding structures and external light reflected on the surface of the pressed part by taking advantage of the high specularity of the surface of the pressed part. The inspection device 10 can determine the presence or absence of an abnormality in the pressed part by detecting a change in the reflection of the surrounding structures and external light caused by the slight deformation of the pressed part due to the crack C in the pressed part.

[0146] As a result, the inspection device 10 can detect even minute abnormalities including defects such as cracks C in pressed parts. In addition, the inspection device 10 can also detect high temperature areas caused by deformation and sliding of the pressed parts. As a result, the inspection device 10 can improve the quality and manufacturing efficiency of pressed parts. The inspection device 10 can detect abnormalities even in materials that generate little processing heat due to press processing, such as aluminum plates, which was difficult to do with conventional technology.

[0147] Second embodiment The inspection method according to the second embodiment of the present disclosure will be described in more detail with reference to FIG. 3. The inspection method according to the second embodiment differs from the first embodiment in that an intermediate variable vector is not used. Other configurations, functions, effects, modifications, etc. are the same as those of the first embodiment, and the corresponding explanations also apply to the inspection device 10 according to the second embodiment. In the following, the same components as those in the first embodiment are given the same reference numerals, and the explanations thereof will be omitted. The differences from the first embodiment will be mainly described.

[0148] 3, the image analysis unit 152 of the control unit 15 regards the n normal image vectors including m elements acquired in step S101 as m observed values ​​of n explanatory variables. Similarly, the image analysis unit 152 regards the object image vector including m elements acquired in step S104 as m observed values ​​of the objective variable.

[0149] In calculating a reconstruction vector in step S106 of FIG. 3, the image analysis unit 152 calculates a reconstruction vector by weighting and adding the n normal image vectors with the coefficients of the regression model calculated in step S105.

[0150] Unlike the first embodiment, the inspection device 10 according to the second embodiment uses normal image vectors as explanatory variable data as they are when the variation of normal image samples is small and the influence of the variation can be suppressed with a relatively small number of normal image samples, and when the manufacturing pitch of press parts as the object M to be inspected is relatively long and the computational load of the regression model is not a significant problem. In this case, the matrix Z of formula 19 obtained from matrix X of formula 2 is used as the explanatory variable.

[0151]

number

[0152] The image analysis unit 152 obtains the linear regression equation of Equation 21, which is obtained by multiplying the explanatory variable Z of Equation 19 by the coefficient b of Equation 20 and adding the result, as a predicted value of y in Equation 10.

[0153]

number

[0154] The subsequent processing is the same as in the first embodiment using the intermediate variable vector. Equation 23 obtained from Equation 10 and Equation 22 becomes the first reconstruction error vector. Equation 24 obtained from Equation 10 and Equation 22 becomes the second reconstruction error vector.

[0155]

number

[0156] Equation 23 is used to detect anomalies that are cooler than the surroundings, and Equation 24 is used to detect anomalies that are hotter than the surroundings.

[0157] (effect) According to the second embodiment described above, it is possible to improve the accuracy of abnormality determination of the object M, which is a pressed part that has been press-processed. The inspection device 10 can calculate a reconstruction vector of the object image that highly accurately approximates the object image vector by weighting and adding n normal image vectors with the coefficients of the obtained regression model. If the object image contains an abnormality, the inspection device 10 cannot reconstruct that part with a linear sum of normal image vectors. Therefore, the difference between the object image vector and the reconstruction vector becomes large. The inspection device 10 regards the difference between the object image vector and the reconstruction vector of the object image, that is, the residual of the regression model, as a reconstruction error vector, and determines whether or not there is an abnormality in the object M depending on the magnitude of the values ​​of the elements of the reconstruction error vector.

[0158] The inspection device 10 according to the second embodiment is capable of identifying a regression model that accurately approximates and reconstructs normal image samples based on the assumption that the variations are small for each image to be inspected. This allows the inspection device 10 to accurately detect even minute abnormalities.

[0159] (Modification) Although the present disclosure has been described based on the drawings and examples, it should be noted that those skilled in the art can make various modifications and alterations based on the present disclosure. Therefore, it should be noted that these modifications and alterations are included in the scope of the present disclosure. For example, the functions included in each configuration or each step can be rearranged so as not to be logically inconsistent, and multiple configurations or steps can be combined or divided into one.

[0160] For example, a general-purpose electronic device such as a smartphone or a computer can be configured to function as the inspection device 10 according to each of the above-mentioned embodiments. Specifically, a program describing the processing content for realizing each function of the inspection device 10 according to each of the embodiments is stored in the memory of the electronic device, and the program is read and executed by a processor of the electronic device. Therefore, the disclosure according to one embodiment can also be realized as a program executable by a processor.

[0161] Alternatively, the disclosure according to an embodiment may be realized as a non-transitory computer-readable medium storing a program executable by one or more processors to cause the inspection apparatus 10 according to each embodiment to execute each function. It should be understood that these are also included within the scope of the present disclosure.

[0162] In the above embodiments, the regression model is described as including a linear regression model, but is not limited thereto. The regression model may include any other model, such as a neural network, such as deep learning.

[0163] In each of the above embodiments, the camera 2 is not included in the inspection device 10 and is separate from the inspection device 10, but is not limited to this. The camera 2 may be included in the inspection device 10 and configured integrally with the inspection device 10. In other words, the inspection device 10 itself may have an imaging function based on the camera 2. [Explanation of symbols]

[0164] 1 Production management system 2 Camera 10 Inspection equipment 11 Image input section 12 Storage section 121 Explanatory variable storage section 122 Result storage section 13 Input section 14 Output section 15 Control section 151 Explanatory variable generation section 152 Image Analysis Unit 153 Result judgment section 154 Input / Output Interface M Object C Crack

Claims

1. 1. An inspection method for inspecting an object, which is a pressed part that has been subjected to press processing, based on an image obtained by imaging the object, comprising: arranging pixel values ​​of a normal image including m pixels, and using the n normal images to obtain n normal image vectors, each including m elements; ordering pixel values ​​of an object image of said object comprising m pixels to obtain an object image vector comprising m elements; a step of regarding the n normal image vectors each including m elements as m observed values ​​of n explanatory variables of a regression model for each m element, or regarding d intermediate variable vectors obtained by weighting and adding the n normal image vectors using weights of d (d<n) combinations calculated in advance as the m observed values ​​of the d explanatory variables, regarding the object image vector each including m elements as the m observed values ​​of the response variable of the regression model, and calculating the regression model using the explanatory variables and the response variable; calculating a reconstructed vector of the object image that approximates the object image vector by weighting and adding the explanatory variables with coefficients of the regression model; calculating a difference between the object image vector and the reconstruction vector as a reconstruction error vector; identifying the object as having an abnormality when it is determined that the number of elements of the reconstruction error vector whose element values ​​are equal to or greater than a predetermined first threshold is equal to or greater than a predetermined second threshold; Including, Testing method.

2. The inspection method according to claim 1, obtaining a surface temperature distribution of the object from the object image; and identifying the surface temperature distribution. Testing method.

3. The inspection method according to claim 2, In the step of calculating the reconstruction error vector, the reconstruction error vector is calculated to include at least one of a first reconstruction error vector whose value is large in a portion in the surface temperature distribution that is lower in temperature than a surrounding healthy portion, and a second reconstruction error vector obtained by inverting the first reconstruction error vector whose value is large in a portion in the surface temperature distribution that is higher in temperature than a surrounding healthy portion. Testing method.

4. The inspection method according to any one of claims 1 to 3, In the step of calculating the reconstructed vector, the reconstructed vector is calculated by weighting and adding the n normal image vectors by the coefficients of the regression model. Testing method.

5. The inspection method according to any one of claims 1 to 3, In the step of calculating the reconstructed vector, the reconstructed vector is calculated by weighting and adding the d intermediate variable vectors by the coefficients of the regression model. Testing method.

6. The inspection method according to any one of claims 1 to 3, The regression model comprises a linear regression model. Testing method.

7. An inspection apparatus for inspecting an object, which is a pressed part that has been subjected to press processing, based on an image obtained by imaging the object, comprising: A control unit is provided, The control unit is arranging pixel values ​​of a normal image including m pixels, and using the n normal images to obtain n normal image vectors, each of which includes m elements; arranging pixel values ​​of an object image of the object comprising m pixels to obtain an object image vector comprising m elements; the n normal image vectors each including m elements are regarded as m observed values ​​of n explanatory variables of a regression model for each m element, or d intermediate variable vectors obtained by weighting and adding the n normal image vectors using weights of d (d<n) combinations calculated in advance are regarded as the m observed values ​​of the d explanatory variables, and the object image vector each including m elements is regarded as m observed values ​​of a response variable of a regression model, and the regression model is calculated using the explanatory variables and the response variable; calculating a reconstructed vector of the object image that approximates the object image vector by weighting and adding the explanatory variables with coefficients of the regression model; Calculating a difference between the object image vector and the reconstruction vector as a reconstruction error vector; If it is determined that the number of elements of the reconstruction error vector whose element values ​​are equal to or greater than a predetermined first threshold is equal to or greater than a predetermined second threshold, it is determined that the object has an abnormality. Inspection equipment.

8. 8. The inspection device according to claim 7, The control unit acquires a surface temperature distribution of the object from the object image and identifies the surface temperature distribution. Inspection equipment.

9. 9. The inspection device according to claim 8, the control unit, in calculating the reconstruction error vector, calculates the reconstruction error vector including at least one of a first reconstruction error vector whose value is large in a portion in the surface temperature distribution that is lower in temperature than a surrounding healthy portion, and a second reconstruction error vector obtained by inverting the first reconstruction error vector whose value is large in a portion in the surface temperature distribution that is higher in temperature than a surrounding healthy portion. Inspection equipment.

10. The inspection device according to any one of claims 7 to 9, the control unit calculates the reconstructed vector by weighting and adding the n normal image vectors with coefficients of the regression model. Inspection equipment.

11. The inspection device according to any one of claims 7 to 9, The control unit calculates the reconstructed vector by weighting and adding the d intermediate variable vectors by coefficients of the regression model. Inspection equipment.

12. The inspection device according to any one of claims 7 to 9, The regression model comprises a linear regression model. Inspection equipment.

Images