Abnormality inspection system, abnormality inspection method, and program

Abstract

To provide an abnormality inspection system, an abnormality inspection method, and a program which can accurately detect the abnormality of components.SOLUTION: An abnormality inspection system S1 includes: an image acquisition section 244 for, in a manner to include the same portions of components in each different region of a plurality of pieces of consecutive imaging data, acquiring the imaging data of the components; and a determination section 245 for detecting the existence of abnormality in the plurality of pieces of consecutive imaging data so as to determine that the components have the abnormality when the abnormality has been detected in all the plurality of pieces of imaging data.SELECTED DRAWING: Figure 3A

Description

[Technical field]

[0001] The present invention relates to an abnormality inspection system, an abnormality inspection method, and a program. [Background technology]

[0002] Industrial parts need to be checked during the production process to prevent low-quality parts from being shipped. For example, Patent Literature 1 describes a technology for determining the specification classification of parts using a trained model trained by machine learning. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2020-149578 A Summary of the Invention [Problem to be solved by the invention]

[0004] For example, an inspection device for inspecting parts can take an image of the part and analyze the image data to determine abnormalities on the part surface. However, the inspection device may erroneously detect abnormalities on the part surface, such as droplets or brush marks, when these are not actually abnormal.

[0005] The present invention is intended to solve such problems, and provides an abnormality inspection system, an abnormality inspection method, and a program that are capable of accurately detecting abnormalities in parts. [Means for solving the problem]

[0006] An abnormality inspection system according to an exemplary embodiment of the present invention includes an acquisition unit that acquires imaging data of a component such that the same location of the component is included in different areas of the multiple consecutive imaging data, and a determination unit that detects the presence or absence of an abnormality in the multiple consecutive imaging data and determines that the component is abnormal when an abnormality is detected in all of the multiple imaging data.The abnormality inspection system determines abnormality of the component based on detection results related to multiple imaging data in which the same location of the component is captured under different conditions, and therefore can detect abnormality of the component more accurately than when determination is based on detection results related to a single imaging data.

[0007] In the above anomaly inspection system, the determination unit may detect the presence or absence of an anomaly by inputting a plurality of consecutive image data to a first learning model trained by using the image data of the component as teacher data. Since the anomaly inspection system detects anomalies using a trained learning model, it can determine the presence or absence of an anomaly with higher accuracy.

[0008] In the above anomaly inspection system, the determination unit may input one piece of imaging data acquired by the acquisition unit to a first learning model, and when it determines from an output result of the first learning model that an anomaly has been detected in the one piece of imaging data, input other imaging data captured consecutively to the one piece of imaging data to the first learning model, thereby detecting the presence or absence of an anomaly in all of the multiple pieces of imaging data. If no anomaly is found as a result of initially executing a determination in the first learning model using one piece of imaging data, the anomaly inspection system does not execute any further processing on the imaging data, thereby suppressing unnecessary processing and making inspection more efficient.

[0009] In the above anomaly inspection system, the first learning model may be a model that outputs a type of abnormality in the appearance of the part, and the determination unit may determine the presence or absence of an abnormality in the part according to the type of abnormality in the appearance of the part indicated by the output result of the first learning model in a plurality of consecutive imaging data. The anomaly inspection system can suppress erroneous detection of a part that has an abnormality in appearance but has no abnormality in its actual quality as abnormal. In other words, the anomaly inspection system can suppress overdetection.

[0010] In the above anomaly inspection system, the determination unit may determine the presence or absence of a suspected anomaly by inputting the imaging data acquired by the acquisition unit into a second learning model different from the first learning model that is trained by using the imaging data of the component as teacher data, and may determine the presence or absence of anomaly by inputting a plurality of consecutive imaging data including the imaging data determined to be suspected of anomaly into the first learning model. Since the anomaly inspection system determines an anomaly of a component using two different types of learning models, the accuracy of anomaly inspection can be improved.

[0011] In the above anomaly inspection system, the determination unit may input one piece of imaging data acquired by the acquisition unit to the second learning model, and when it determines from the output result of the second learning model that an appearance abnormality has been detected in the one piece of imaging data, input other imaging data captured consecutively to the one piece of imaging data to the second learning model, and determine that an abnormality is suspected when an appearance abnormality is also detected in the other imaging data. The anomaly inspection system may first perform a determination using the one piece of imaging data with the second learning model, and if no abnormality is found, no further processing is performed on the imaging data, thereby suppressing unnecessary processing and making inspection more efficient.

[0012] In the above-mentioned anomaly inspection system, the acquisition unit may acquire image data of the part such that the same part of the part is included in at least three consecutive pieces of image data. The anomaly inspection system determines an anomaly of the part based on a detection result using three or more pieces of image data in which the same part of the part is distributed in different places in the image. Therefore, in the inspection, noise associated with image characteristics such as the degree of light irradiation can be suppressed, and anomalies in the cam portion can be detected more accurately.

[0013] In the above-described anomaly inspection system, the acquisition unit may acquire image data of the curved surface of the component while the component is rotated by a rotation mechanism that rotates the component around an axis. This allows the anomaly inspection system to efficiently acquire continuous image data of the component, thereby shortening the time required for the entire inspection.

[0014] In an abnormality inspection method according to an exemplary embodiment of the present invention, an abnormality inspection system executes an acquisition step of acquiring imaging data of a component such that the same location of the component is included in different areas of a plurality of consecutive imaging data, and a determination step of detecting the presence or absence of an abnormality in the plurality of consecutive imaging data and determining that the component is abnormal when an abnormality is detected in all of the plurality of imaging data. As a result, the abnormality inspection system determines an abnormality in the component based on detection results related to a plurality of imaging data in which the same location of the component is photographed in different states, and therefore can detect an abnormality in the component more accurately than in the case of determination based on detection results related to a single imaging data.

[0015] A program according to an exemplary aspect of the present invention causes a computer to execute an acquisition step of acquiring image data of a part such that the same part of the part is included in different areas of the multiple consecutive image data, and a determination step of detecting the presence or absence of an abnormality in the multiple consecutive image data and determining that the part is abnormal when an abnormality is detected in all of the multiple image data. As a result, the computer determines the abnormality of the part based on the detection results related to the multiple image data in which the same part of the part is captured in different states, thereby making it possible to accurately detect the abnormality of the part. Effect of the Invention

[0016] The present invention provides an abnormality inspection system, an abnormality inspection method, and a program capable of accurately detecting abnormalities in parts. [Brief description of the drawings]

[0017] [Figure 1] 1 is a block diagram showing an example of an abnormality inspection system according to a first embodiment. [Figure 2A] FIG. 2 is a detailed diagram illustrating an example of an inspection device of the anomaly inspection system according to the first embodiment. [Figure 2B] FIG. 2 is a diagram showing an example of an inspection target part according to the first embodiment; [Figure 2C] 3 is a schematic diagram showing an example of the positional relationship between a cam portion, a lighting for inspecting the cam portion, and a camera for inspecting the cam portion according to the first embodiment; FIG. [Figure 2D] FIG. 2 is a diagram showing an image of a cam portion taken at timing (1) in the first embodiment. [Figure 2E] FIG. 11 is a diagram showing an image of a cam portion taken at timing (2) in the first embodiment. [Figure 2F] FIG. 11 is a diagram showing an image of a cam portion taken at timing (3) in the first embodiment. [Figure 2G] 4 is an image showing a blowhole on a surface of a cam portion actually photographed in the first embodiment. [Figure 2H]4 is an image showing droplets on a surface of a cam portion actually photographed in the first embodiment. [Figure 3A] FIG. 2 is a block diagram showing an example of a control unit according to the first embodiment. [Figure 3B] 1 is a schematic diagram showing the relationship between an inspection unit and a learning model according to the first embodiment. FIG. [Figure 3C] FIG. 2 is a schematic diagram showing an example of image clipping according to the first embodiment; [Figure 4A] 5 is a flowchart showing an example of a process performed when the abnormality inspection system according to the first embodiment inspects a cam portion. [Figure 4B] 5 is a flowchart showing an example of a process performed when the abnormality inspection system according to the first embodiment inspects a cam portion. [Figure 4C] 5 is a flowchart showing an example of a process performed when the abnormality inspection system according to the first embodiment inspects a cam portion. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] First embodiment Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0019] <Abnormality inspection system> Fig. 1 is a diagram for explaining an abnormality inspection system according to an embodiment. As shown in Fig. 1, the abnormality inspection system S1 according to the present embodiment includes an inspection device 10 and an information processing device 20. In the abnormality inspection system S1 according to the present embodiment, a user operates the information processing device 20 to cause a camera of the inspection device 10 to capture an image of a surface of an inspection target part, and causes the information processing device 20 to inspect whether or not there is an abnormality on the surface. In this example, the inspection target part is a camshaft. Details of the inspection device 10 and the information processing device 20 will be described below.

[0020] The inspection device 10 has a cam portion inspection light 11, a cam portion inspection camera 12, a journal portion inspection light 13, a journal portion inspection camera 14, and a rotation motor 15.

[0021] 2A is a detailed diagram showing an example of the inspection device 10, and each component of the inspection device 10 will be described using this diagram. As the cam portion inspection lighting 11, a cam portion inspection lighting 11A is provided on the front side of the inspection target part W, and a cam portion inspection lighting 11B is provided on the rear side thereof, and the information processing device 20 turns on each lighting when inspecting the cam portion of the inspection target part W. As the cam portion inspection cameras 12, cam portion inspection cameras 12A-12D are provided from the front side to the rear side of the inspection target part W. Each cam portion inspection camera 12 captures imaging data (images) of the cam portion by continuously capturing an image of the area of ​​the cam portion irradiated with light by the cam portion inspection lighting 11 multiple times under the control of the information processing device 20.

[0022] The journal portion inspection lighting 13 is turned on under the control of the information processing device 20 when the journal portion of the inspection target part W is inspected. As the journal portion inspection cameras 14, journal portion inspection cameras 14A-14D are provided from the front side to the rear side of the inspection target part W. Under the control of the information processing device 20, each journal portion inspection camera 14 captures the image data of the journal portion by continuously photographing the area of ​​the journal portion illuminated by the journal portion inspection lighting 13 multiple times. Note that the cam portion inspection camera 12 and the journal portion inspection camera 14 are so-called area cameras, which are provided so that the imaging elements of the cameras are planar (i.e., there are multiple imaging elements vertically and horizontally).

[0023] 2B is an enlarged view of a camshaft as the inspection target part W. This camshaft has eight cam portions C1-C8 and four journal portions J1-J4. A cam portion inspection lighting device 11 irradiates light onto the cam portions C1-C8, and a cam portion inspection camera 12 photographs the illuminated area. A journal portion inspection lighting device 13 irradiates light onto the journal portions J1-J4, and a journal portion inspection camera 14 photographs the illuminated area.

[0024] FIG. 2C is a schematic diagram showing an example of the positional relationship between the cam portion C1, the cam portion inspection illumination 11A, and the cam portion inspection camera 12A. The cam portion inspection illumination 11A and the cam portion inspection camera 12A are arranged such that the front parts of the illumination and the camera are aligned on the same line, and the cam portion C1 is arranged on the extension line. In other words, the cam portion inspection illumination 11A and the cam portion inspection camera 12A are arranged so as to have the same attitude (same phase) with respect to the cam portion C1. The cam portion inspection illumination 11A irradiates light onto the cam portion C1. The cam portion inspection camera 12A has an angle of view θ for photographing the rotation direction of the cam portion, and photographs the photographing area RC in the cam portion. The cam portion inspection illumination 11A and the cam portion inspection camera 12A are arranged in the same positional relationship not only with respect to the cam portion C1 but also with respect to other cam portions, and can photograph a predetermined photographing area of ​​the cam portion in the same manner. In addition, the cam portion inspection lighting 11B and the cam portion inspection camera 12B are also positioned in the same positional relationship with respect to each cam portion as the cam portion inspection lighting 11A and the cam portion inspection camera 12A, respectively, and can similarly photograph a specified shooting area of ​​the cam portion.

[0025] Returning to Fig. 2A, the description will be continued. The rotary motor 15, together with a clamp FC provided on the front side of the inspection target part W and a clamp RC provided on the rear side, constitutes a rotation mechanism that axially supports and rotates the shaft of the inspection target part W. After the user fixes both ends of the inspection target part W with the clamps FC and RC, thereby making the inspection target part W rotatable, the rotary motor 15 rotates the inspection target part W under the control of the information processing device 20.

[0026] In detail, when inspecting the cam portion of the inspection target part W, the information processing device 20 rotates the rotary motor 15, and while the inspection target part W rotates once with the cam portion inspection light 11 turned on, controls the cam portion inspection camera 12 to continuously photograph the cam portion multiple times. As a result, the cam portion inspection camera 12 photographs a different area of ​​the cam portion for each photograph. The information processing device 20 determines parameters related to photographing, such as the photographing interval and number of photographs (photographing time) of the cam portion inspection camera 12 and the rotation speed of the rotary motor 15, so that the cam portion inspection camera 12 photographs the entire surface of the cam portion of the inspection target part W by photographing multiple times.

[0027] After photographing the cam portion as described above, the information processing device 20 turns off the cam portion inspection lighting 11 and instead turns on the journal portion inspection lighting 13. Then, the information processing device 20 rotates the rotary motor 15, and controls the journal portion inspection camera 14 to continuously photograph the journal portion multiple times while the inspection target part W rotates once. This allows the journal portion inspection camera 14 to photograph a different area of ​​the journal portion for each photographing.

[0028] In this embodiment, the information processing device 20 determines the above-mentioned parameters related to the shooting so that, in the continuous shooting by the cam portion inspection camera 12, the same location of the cam portion is captured in three consecutive pieces of image data.

[0029] 2D-2F show images of the cam portion taken at successive times (1) to (3). Each image is taken of the area RC irradiated with light as shown in FIG. 2C. At the time (1) shown in FIG. 2D, the predetermined point SP of the cam portion is located at the upper end of the image IM. At this time, the cam portion rotates downward. Therefore, at the time (2) when time has passed from the time (1), the predetermined point SP moves downward on the image IM as shown in FIG. 2E. In FIG. 2E, the predetermined point SP is located in the central area of ​​the image IM. At the time (3) when time has further passed from the time (2), the predetermined point SP moves further downward on the image IM as shown in FIG. 2F. In FIG. 2F, the predetermined point SP is located at the lower end of the image IM.

[0030] In this way, when the cam portion inspection camera 12 photographs the cam portion, the surface of the cam portion to be photographed forms a curved surface. Therefore, in Fig. 2D-2F, the state of light that is irradiated from the cam portion inspection illumination 11 to the specified point SP, reflected at the specified point SP, and incident on the cam portion inspection camera 12 is different. As a result, the applicant has discovered the following problem.

[0031] Figure 2G is an actual image of region RC where there is a cast cavity on the surface of the cam part. A cast cavity is an example of a defect (actual abnormality) that occurs on the surface of the cam part. The images in the upper, middle, and lower rows of Figure 2G correspond to timings (1), (2), and (3) shown in Figures 2D-2F, respectively. As shown in Figure 2G, the cast cavity is clearly visible in the image at each timing. This is because the cast cavity forms a cavity with a certain depth on the surface of the cam part, and therefore reflects light differently from areas other than the cast cavity, even if the photographing location is different.

[0032] FIG. 2H is an image of an area RC where droplets (e.g., cleaning liquid) are present on the surface of the cam portion, which was actually photographed. The droplets are an example of an event (hereinafter also referred to as a false abnormality) that occurs on the surface of the cam portion and is an abnormality in appearance but not a substantial abnormality. The images in the upper, middle, and lower rows of FIG. 2F also correspond to the timings (1), (2), and (3) shown in FIG. 2D-2F, respectively. Referring to FIG. 2F, in the image photographed at timing (2) (i.e., the image in FIG. 2C where there are droplets in front of the cam portion inspection camera 12), the droplets are barely visible in the image. However, in the images photographed at timings (1) and (3), the droplets are visible. This is because the areas of the cam portion surface corresponding to the upper and lower ends of the image are shaded areas that are difficult to light due to the positional relationship between the cam portion inspection illumination 11, the cam portion inspection camera 12, and the area RC. Therefore, at timings (1) and (3), the droplets are photographed as patterns with light and dark areas similar to the shape of the blowholes shown in Figure 2G. In other words, there is a possibility of overdetection if an inspection is performed using an image in which droplets are captured at the top and bottom of the image.

[0033] In addition, as a defect other than a cast cavity, there may be a scratch on the surface of the cam part. In such a case, the scratch is clearly captured regardless of the timing of the image capture. On the other hand, as a false abnormality other than a liquid droplet, there may be a trace of deburring with a brush on the surface of the cam part. As shown in Figure 2H, this pattern is captured in an image that is almost unrecognizable at timing (2), but at timings (1) and (3), it may be captured as a pattern with a shape and light and dark areas similar to that of a linear scratch. In this case, too, if an inspection is performed using an image in which the deburring traces are captured at the top and bottom of the image, there is a possibility of overdetection.

[0034] As described below, in the present invention, the information processing device 20 is inspected so as to suppress such overdetection. The information processing device 20 acquires images of the cam portion and journal portion obtained as described above, and judges whether or not an abnormality exists in each image by executing the process described below. In particular, in this embodiment, the judgment process for detecting an abnormality in the cam portion will be described.

[0035] Next, a description will be given of the information processing device 20. The information processing device 20 includes a DB (Database) 21, a display panel 22, an input unit 23, and a control unit 24.

[0036] DB21 stores three types of learning models required for inspection. These learning models are AI (Artificial Intelligence) models that have been trained in advance using images as training data through machine learning such as deep learning. Details of these learning models will be described later. DB21 also stores all thresholds used for judgment in the inspection unit described later.

[0037] DB21 is configured with a storage device such as a flash memory, a memory card, a HDD (Hard Disk Drive), an optical disk drive, etc., but the type of storage device is not limited to these. DB21 may be provided outside the information processing device 20. In this case, the information processing device 20 may connect to DB21 via an information transmitting / receiving unit (not shown) and obtain the data stored in DB21.

[0038] The display panel 22 is an interface that enables a user to view the determination result of the presence or absence of an abnormality in the information processing device 20. The input unit 23 is an interface that enables a user to input instructions to the information processing device 20 regarding the start of an examination and the settings for the examination.

[0039] The control unit 24 controls the cam portion inspection lighting 11, the cam portion inspection camera 12, the journal portion inspection lighting 13, the journal portion inspection camera 14, and the rotary motor 15 as described above to capture images of the cam portion and journal portion of the inspection target part W. Then, the control unit 24 obtains the images captured by the cam portion inspection camera 12 and the journal portion inspection camera 14, respectively, and performs the inspection as described below.

[0040] 3A is a block diagram for explaining the configuration of the control unit 24. The control unit 24 includes a memory 241, an I / O (Input / Output) unit 242, and an information processing unit 243. Each unit of the control unit 24 will be explained below.

[0041] The memory 241 is composed of a volatile memory, a non-volatile memory, or a combination thereof. The memory 241 is not limited to one, and a plurality of memories may be provided. The volatile memory may be, for example, a RAM (Random Access Memory) such as a DRAM (Dynamic Random Access Memory) or a SRAM (Static Random Access Memory). The non-volatile memory may be, for example, a PROM (Programmable ROM), an EPROM (Erasable Programmable Read Only Memory), or a flash memory.

[0042] The memory 241 is used to store one or more instructions. Here, the one or more instructions are stored in the memory 241 as a group of software modules. The information processing unit 243 can perform the following processes by reading and executing the one or more instructions from the memory 241.

[0043] The I / O unit 242 is a hardware interface that executes input and output of information to and from the outside of the control unit 24. In this embodiment, the control unit 24 is connected to the cam portion inspection lighting 11, the cam portion inspection camera 12, the journal portion inspection lighting 13, the journal portion inspection camera 14, and the rotary motor 15, and appropriately executes input and output of information with these via the I / O unit 242.

[0044] The information processing unit 243 is composed of any processor or the like for analyzing images. In this example, the information processing unit 243 includes a GPU (Graphics Processing Unit) useful for image processing as a processor. However, the information processing unit 243 may include a CPU (Central Processing Unit), an MPU (Micro Processing Unit), an FPGA (Field-Programmable Gate Array), a DSP (Digital Signal Processor), and an ASIC (Application Specific Integrated Circuit) as a processor. The above-mentioned memory 241 may include one built into the information processing unit 243 in addition to one provided outside the information processing unit 243.

[0045] The information processing unit 243 realizes functions of an image acquisition unit 244, a determination unit 245, etc., by reading and executing software (computer programs) from the memory 241. The determination unit 245 can be further divided into the functions of a first inspection unit 246 and a second inspection unit 247.

[0046] The image acquisition unit 244 acquires images taken from the cam portion inspection camera 12 and the journal portion inspection camera 14 via the I / O unit 242. The acquired images of the cam portion are output to the first inspection unit 246.

[0047] 3B shows the first inspection unit 246 and the second inspection unit 247, and two types of learning models stored in DB 21 and accessed by the first inspection unit 246 and the second inspection unit 247 for inspection, respectively. Each inspection unit will be described in detail below with reference to FIG. 3B.

[0048] The first inspection unit 246 inputs an image of the cam portion (first image data) captured by the cam portion inspection camera 12 into the first learning model M1, and determines whether or not there is a suspected abnormality in the cam portion based on the results output from the first learning model M1.

[0049] Here, the first learning model M1 is a model that is learned by using an image of the cam portion as teacher data, and outputs a calculated value regarding the presence or absence of an abnormality in the appearance of the cam portion as an output result when an image is input. As a detailed example, the first learning model M1 performs a semantic segmentation process on the input image to calculate and output a first judgment value for the image. This first judgment value represents the normality of the cam portion shown in the image, and the larger the value, the cleaner the surface of the cam portion in the image is, and the smaller the value, the more the surface has a pattern resembling a defect such as a scratch or a shrinkage cavity. The first inspection unit 246 compares the calculated first judgment value with a threshold value TH1 stored in the DB21, and judges an image for which the first judgment value is equal to or less than the threshold value TH1 as an image for which the cam portion has an abnormality in the appearance. On the other hand, the first inspection unit 246 judges an image for which the first judgment value is greater than the threshold value TH1 as an image for which the cam portion has no abnormality in the appearance.

[0050] The first learning model M1 may divide an image into a plurality of sections and calculate the first judgment value for each section. In this case, the first inspection unit 246 may determine that the image has an appearance abnormality in the cam portion when the first judgment value in at least one section of the image is equal to or less than the threshold value TH1. Alternatively, the first inspection unit 246 may determine that the image has an appearance abnormality in the cam portion when the first judgment value in a number of sections equal to or greater than a plurality of threshold values ​​is equal to or less than the threshold value TH1.

[0051] An abnormality in appearance means that the surface of the cam part is not clean, and for example, it means that there is a pattern such as a line or a circle on the surface, or that there is a part on the surface that is different in brightness (for example, a dark part compared to the surroundings). If there is a pattern such as a circle on the surface of the cam part, or if there is a dark part on the surface, there is a possibility that a casting cavity has occurred on the surface, and if there is a line pattern on the surface, there is a possibility that there is a scratch on the surface. The scratch or casting cavity means a defect (actual abnormality) of the inspected part W. However, even if there is a pattern such as a circle on the surface of the cam part, the pattern may be caused by droplets on the surface. In addition, even if there is a line pattern on the surface, the pattern may be a mark of a brush for deburring, a check mark on the inspected part W, or a grinding stone mark generated during grinding. These patterns are naturally generated by the processing of the inspected part W in the manufacturing stage, and are not defects of the inspected part W. Therefore, there is a possibility that an image having such a false abnormal pattern may be judged as abnormal by the anomaly inspection system, resulting in an overdetection problem.

[0052] In this embodiment, when it is determined that the cam portion has an appearance abnormality in one image to be input, the first inspection unit 246 inputs the one image and two images captured consecutively to the first learning model M1. Here, the first inspection unit 246 determines where in the image the area determined to have an appearance abnormality is located in the one image, and selects the other two images showing the appearance abnormality as images to be input to the first learning model M1 according to the location.

[0053] In this example, the upper, central, and lower regions are defined as regions that divide the entire image region into thirds in the vertical direction. As shown in Figs. 2D-2F, the rotation direction of the cam part is from top to bottom of the screen. In this case, if the location where the appearance abnormality is reflected is the upper region of the image, the first inspection unit 246 selects the image taken immediately after the timing of photographing the image to be judged and immediately after that. If the location where the appearance abnormality is reflected is the central region of the image, the first inspection unit 246 selects the image taken immediately before the timing of photographing the image to be judged and immediately after that. If the location where the appearance abnormality is reflected is the lower region of the image, the first inspection unit 246 selects the image taken immediately before the timing of photographing the image to be judged and immediately before that.

[0054] The first inspection unit 246 compares the first judgment value calculated for these two images with the threshold value TH1 stored in the DB21. As described above, the first learning model M1 may divide each image into a plurality of sections and calculate the first judgment value for each section. Then, when the first judgment value for both images is equal to or less than the threshold value TH1 (i.e., when an abnormality in appearance is detected in both of the two images), the first inspection unit 246 judges that the image judged as being suspicious of an abnormality. The image judged as being suspicious of an abnormality becomes the object of re-inspection by the second inspection unit 247. Only when it is judged by this re-inspection that a defect is reflected in the image is the inspection target part W judged as being abnormal for the first time. Note that the image judged as being not suspicious of an abnormality is not inspected by the second inspection unit 247.

[0055] The second inspection unit 247 inputs the entire image (second imaging data) of the cam portion determined to be suspected of being abnormal to the second learning model M2. Note that, when the first learning model M1 is a model that divides an image into a plurality of sections and calculates a first judgment value for each section, the second inspection unit 247 may cut out the section of the image where the first judgment value is equal to or less than the threshold value TH1, and input only the image of the cut-out section to the second learning model M2.

[0056] 3C is a diagram showing an example of the process of cutting out an image by the second inspection unit 247. Due to the presence of a pattern DA suspected of being abnormal in the original image IM captured by the cam portion inspection camera 12, the first inspection unit 246 determines that the first judgment value in the section DA of the image IM is equal to or less than the threshold value TH1. At this time, the second inspection unit 247 can cut out the section DA from the image IM and input only the image of the section DA to the second learning model M2.

[0057] The second learning model M2 outputs the calculation result based on the image input as described above. The second inspection unit 247 judges whether or not there is an abnormality in the cam portion based on the output result from the second learning model M2.

[0058] Here, the second learning model M2 is a different type of AI model from the first learning model M1. The second learning model M2 is learned using an image of the cam part as teacher data, and is a model that outputs a calculated value related to the type of abnormality in the appearance of the cam part as an output result when the image is input. In detail, the second learning model M2 executes a classification process on the input image, calculates and outputs a second judgment value for the input image. This second judgment value is a matching rate between the input image and the above-mentioned false abnormality pattern, and refers to a cosine value (cosθ) when the inner product of the input image and the false abnormality pattern is calculated. The second learning model M2 calculates the second judgment value for the number of types of false abnormality patterns modeled. The larger the value of this second judgment value (the closer to 1), the more similar the pattern on the surface of the cam part in the image is to the false abnormality pattern (for example, at least one of the patterns of droplets, brush marks for deburring, check marks, or grindstone marks), indicating that there is no substantial abnormality. On the other hand, the smaller the second judgment value, the less similar the pattern on the cam portion surface is to the pattern of a false anomaly, and therefore the more likely the pattern is to be a defect such as a scratch or a blowhole.

[0059] The second inspection unit 247 compares the calculated second judgment value with the threshold value TH2 stored in the DB21. The threshold value TH2 is set as many times as the number of types of modeled false abnormality patterns. The second inspection unit 247 judges an image in which the second judgment value is equal to or less than the threshold value TH2 in all types of false abnormality patterns as an image in which the cam portion has an abnormality. On the other hand, the second inspection unit 247 judges an image in which the second judgment value is greater than the threshold value TH2 in at least one type of false abnormality pattern as an image in which the cam portion has no abnormality.

[0060] In addition, when the second inspection unit 247 inputs only an image of a section cut out from a portion of the original photographed image to the second learning model M2, the second learning model M2 performs the above-mentioned classification process on that section, calculates and outputs a second judgment value for that section.

[0061] In this embodiment, when it is determined that there is an abnormality in the cam portion in one image to be input, the second inspection unit 247 selects the one image and two images captured consecutively, and inputs them to the second learning model M2. The two selected images are the same as the two images selected by the first inspection unit 246. When the one image to be input is an image of a cut section, the second inspection unit 247 cuts out the section whose first judgment value is equal to or less than the threshold value TH1 from the two images selected by the first inspection unit 246, and inputs the two images of the section obtained by cutting into the second learning model M2. The second inspection unit 247 compares the calculated second judgment value with the threshold value TH2 stored in the DB21 for the images of the two sections. When the second judgment value is equal to or less than the threshold value TH2 for all types of false abnormality patterns in both of these two images (i.e., when abnormalities are detected in both of the two input images), the second inspection unit 247 determines that there is an abnormality in the cam portion.

[0062] The first inspection section 246 and the second inspection section 247 execute the above-mentioned process for each image captured by each cam section inspection camera 12. If the second inspection section 247 has not determined that there is an abnormality in the cam section even once, the second inspection section 247 determines that there is no abnormality in the cam section, which is the photographed inspection target. On the other hand, if there is even one image in which it is determined that there is an abnormality in the cam section, the second inspection section 247 determines that there is an abnormality in the cam section. However, the second inspection section 247 may determine that there is an abnormality in the cam section when the number of times that it is determined that there is an abnormality in the cam section is equal to or greater than a predetermined number of thresholds. If the second inspection section 247 determines that there is an abnormality in the cam section, which is the inspection target, the second inspection section 247 determines that there is an abnormality in the inspection target part W.

[0063] The second inspection unit 247 can display the above-mentioned determination results on the display panel 22 of the information processing device 20. The second inspection unit 247 may identify the cam portion determined to have an abnormality by specifying the cam portion inspection camera 12 that photographed the cam portion and the photographing direction, and display the determination result on the display panel 22.

[0064] Although not shown, the determination unit 245 may further include a function of an inspection unit that performs inspection of the journal portion using an image of the journal portion acquired by the journal portion inspection camera 14.

[0065] 4A-4C are flowcharts showing an example of a process in which the abnormality inspection system S1 inspects the cam portion, and the process will be described below with reference to Fig. 4A-4C. Note that the details of each process are as described above, and the description will be omitted as appropriate.

[0066] First, the user operates the input unit 23, whereby the information processing device 20 causes the inspection device 10 to inspect the cam portion of the inspection target part W. As this inspection is performed, the image acquisition unit 244 reads all images captured by the cam portion inspection camera 12 (step S11).

[0067] The first inspection unit 246 inputs one image from the read images to the first learning model M1. Then, it determines whether the first judgment value calculated by the first learning model M1 is equal to or smaller than the threshold value TH1 (step S12).

[0068] If the first judgment value is equal to or less than the threshold value TH1 (Yes in step S12), the first inspection unit 246 inputs two images captured consecutively to the input image into the first learning model M1. The method of selecting these two images is as described above. Then, the first inspection unit 246 compares the first judgment value calculated for these two images with the threshold value TH1, and judges whether the first judgment value of both images is equal to or less than the threshold value TH1 (step S13).

[0069] If the first judgment value for both images is equal to or less than the threshold value TH1 (Yes in step S13), the first inspection unit 246 judges that the image that was initially judged to be the target of judgment is suspected of being abnormal. In response to this judgment result, the second inspection unit 247 cuts out the section in the image that was initially judged to be the target of judgment, in which the first judgment value is equal to or less than the threshold value TH1 (step S14). The second inspection unit 247 then inputs the image of the cut-out section to the second learning model M2. The second inspection unit 247 judges whether the second judgment value calculated by the second learning model M2 is equal to or less than the threshold value TH2 for all types of false abnormality patterns (step S15).

[0070] If the second judgment value is equal to or less than the threshold value TH2 for all types of false anomaly patterns (Yes in step S15), the second inspection unit 247 inputs two images captured consecutively to the input image into the second learning model M2. The method of selecting these two images is as described above. Then, the second inspection unit 247 compares the second judgment value calculated for these two images with the threshold value TH2, and judges whether the second judgment value is equal to or less than the threshold value TH2 for all types of false anomaly patterns for both images (step S16).

[0071] If the second judgment value for both images is below the threshold value TH2 for all types of false abnormality patterns (Yes in step S16), the second inspection unit 247 judges that there is an abnormality in the cam portion based on the judgment result of the image, and the abnormality inspection system S1 terminates the inspection process (step S17).

[0072] On the other hand, in step S12, if the first judgment value is greater than the threshold value TH1 (No in step S12), the first inspection unit 246 judges that the image to be inspected is normal (no abnormality) (step S18). In step S13, if the first judgment value is greater than the threshold value TH1 in at least one of the two images (No in step S13), a similar judgment is made.

[0073] Also, in step S14, if the second judgment value is greater than the threshold value TH2 for at least one of the pseudo-abnormal patterns (No in step S15), the second inspection unit 247 judges that the image to be inspected is normal (step S18). In step S16, if the second judgment value is greater than the threshold value TH2 for at least one of the pseudo-abnormal patterns in at least one of the two images (No in step S16), a similar judgment is made.

[0074] Thereafter, the first inspection unit 246 judges whether the above-described judgment has been completed for all images read in step S11 (step S19). If there are images for which judgment has not been completed (No in step S19), the third inspection unit 248 returns to step S12 and executes processing on the unjudged images. On the other hand, if judgment has been completed for all images (Yes in step S19), the first inspection unit 246 judges that the cam portion is normal, and the abnormality inspection system S1 ends the inspection process (step S20).

[0075] In the above flow, the information processing device 20 executes inspection for one image by the first inspection unit 246 and the second inspection unit 247, and executes the inspection for each image in turn. However, the information processing device 20 may execute inspection by the first inspection unit 246 for multiple images (e.g., all images read) at once, and execute inspection by the second inspection unit 247 for images for which the first judgment value is equal to or less than the threshold value TH1.

[0076] As described above, the image acquisition unit 244 of the abnormality inspection system S1 acquires images of the cam portion so that the same part of the part is included in different areas of the multiple consecutive images. The second inspection unit 247 (determination unit 245) then detects the presence or absence of abnormality in the multiple consecutive images, and determines that the cam portion is abnormal if abnormalities are detected in all of the multiple images.

[0077] As shown in Figures 2G and 2H, when a false abnormality pattern appears on the surface of the cam part, depending on the position of the pattern when the image was taken, the anomaly inspection system may erroneously determine that it is a defect such as a blowhole (overdetection). If a learning model is trained to suppress such overdetection, the learning model may conversely erroneously determine that the defective part is a false abnormality.

[0078] The anomaly inspection system S1 according to the present invention judges anomalies in a cam part based on detection results for multiple images taken in different states of the same part of the cam part. Therefore, even if a false abnormality pattern is present at the top or bottom of one image, if the pattern is photographed in the center of another image, the anomaly inspection system S1 can judge that the false abnormality pattern is not abnormal based on that image. Therefore, the anomaly inspection system S1 can accurately detect anomalies in the cam part. Furthermore, in image inspection of the cam part, which is a curved surface, such effects can be achieved by continuously photographing images with a relatively inexpensive area camera so that the photographed areas of the cam part overlap in the previous and next images, without using a line camera, which is highly accurate but expensive. Therefore, the configuration of the anomaly inspection system S1 can be made more inexpensive.

[0079] The second inspection unit 247 may also detect the presence or absence of anomalies by inputting multiple consecutive images to the second learning model M2 that has been trained using images of the cam portion as teacher data. The anomaly inspection system detects anomalies using a trained learning model, so it can determine the presence or absence of anomalies with higher accuracy.

[0080] The second inspection unit 247 may input one image to the second learning model M2, and when it determines from the output result of the second learning model M2 that an abnormality has been detected in that image, it may input another image captured consecutively to the image to the second learning model M2, thereby detecting the presence or absence of an abnormality in all of the multiple images. If no abnormality is found as a result of initially performing a determination using one image, the abnormality inspection system S1 does not perform any further processing on that image, thereby suppressing unnecessary processing and making the inspection more efficient.

[0081] The second learning model M2 may be a model that outputs the type of abnormality in the appearance of the cam portion, and the second inspection unit 247 may determine the presence or absence of an abnormality in the cam portion according to the type of abnormality in the appearance of the cam portion indicated by the output result of the second learning model M2 in a plurality of consecutive images. This allows the abnormality inspection system S1 to suppress erroneous detection of an abnormality in the cam portion when there is no abnormality in the actual quality of the cam portion, even if there is an abnormality in the appearance of the cam portion.

[0082] Furthermore, the first inspection unit 246 may input the image acquired by the image acquisition unit 244 to the first learning model M1 to determine whether or not there is a suspected abnormality in the image, and the second inspection unit 247 may input consecutive images including an image determined to be suspected of being abnormal to the second learning model M2 to determine whether or not there is an abnormality. The abnormality inspection system S1 determines whether or not there is an abnormality in a part using two different types of learning models, thereby improving the accuracy of the abnormality inspection.

[0083] Moreover, the first inspection unit 246 inputs one image to the first learning model M1, and when it is determined from the output result of the first learning model M1 that an abnormality in appearance has been detected in that image, it inputs another image captured consecutively to that image to the first learning model M1. Then, when an abnormality in appearance is also detected in the other image, it may be determined that there is a suspicion of an abnormality in the cam portion. The abnormality inspection system first performs a determination in the first learning model M1 using one image, and if there is no abnormality as a result, it does not perform any further processing on the captured data, so that unnecessary processing can be suppressed and the inspection can be made more efficient. Furthermore, the first inspection unit 246 inputs multiple images to the first learning model M1 to determine whether there is a suspicion of an abnormality, so that it is possible to reduce the number of images determined to be suspicious of an abnormality (i.e., images to be inspected by the second inspection unit 247). Although the second learning model M2 has high accuracy, the inference time per input image is much longer than that of the first learning model M1, so that the time required for the entire inspection can be reduced by reducing the number of images to be inspected by the second inspection unit 247.

[0084] Furthermore, the image acquisition unit 244 may acquire images of the cam portion such that the same location of the cam portion is included in at least three consecutive images. The abnormality inspection system S1 determines an abnormality in the cam portion based on a detection result using three or more images in which the same location of the cam portion is distributed in different locations in the images. Therefore, in the inspection, noise associated with image characteristics such as the degree of light irradiation can be suppressed, and therefore, an abnormality in the cam portion can be detected more accurately.

[0085] Furthermore, the image acquiring unit 244 may acquire images of the curved surface of the cam portion while the cam portion is rotated by a rotation mechanism that pivots and rotates the cam portion (the inspection target part W). This allows the abnormality inspection system S1 to efficiently acquire continuous images of the cam portion, thereby shortening the time required for the entire inspection.

[0086] The present invention is not limited to the above-described embodiment, and can be modified as appropriate without departing from the spirit and scope of the present invention.

[0087] For example, in the embodiment, a two-stage inspection of the cam portion by the first inspection unit 246 and the second inspection unit 247 has been described, but three or more stages of inspection may be performed as the inspection of the cam portion. For example, an inspection other than that by the second inspection unit 247 may be performed either before or after the inspection by the first inspection unit 246.

[0088] In the inspection, the entire surface of the cam portion is not necessarily photographed for inspection, but only a portion of the surface may be photographed for inspection. The same is true for the journal portion. In addition, at least one of the cam portion inspection camera 12 and the journal portion inspection camera 14 used for inspection may be a different type of camera, such as a line camera, instead of an area camera.

[0089] In the embodiment, only one of the process of step S13 by the first inspection unit 246 and the process of step S16 by the second inspection unit 247 may be executed. In addition, in step S13 or S16, the number of images to be additionally determined may be one, not two. For example, in step S13, if the location where the appearance abnormality is reflected is the upper region of the image, the first inspection unit 246 may additionally select an image captured immediately after the image to be determined is captured. Similarly, if the location where the appearance abnormality is reflected is the lower region of the image, the first inspection unit 246 may additionally select an image captured immediately before the image to be determined is captured. In either case, since the location where the appearance abnormality is reflected is assumed to be the central region of the image in the newly selected image, if the appearance abnormality is a pattern of a false abnormality, the first inspection unit 246 can easily accurately determine that fact. This makes it possible to perform the entire inspection more efficiently.

[0090] The information processing device 20 may determine various parameters related to the above-mentioned photographing so that the same part of the cam part is captured in two or four or more consecutive images in the continuous photographing of the cam part inspection camera 12. In this case, one or more images captured consecutively to the image that was the subject of the previous judgment may be selected as an additional image to be judged in step S13 or S16. At this time, the first inspection unit 246 or the second inspection unit 247 may select all images including the area judged to have an abnormal appearance in the image that was the subject of the previous judgment, or may select some of those images. For example, the first inspection unit 246 judges where the area judged to have an abnormal appearance is located in the image related to the judgment subject of step S12. Then, the first inspection unit 246 may select one or more images in which the area is captured closer to the center of the image (a photographed area where overdetection is avoided) by comparing with the image. The second inspection unit 247 can also execute the same process.

[0091] The object on which the first inspection unit 246 and the second inspection unit 247 perform inspection may not be a cam portion-like shape, but may be a part having a perfectly circular cross-sectional shape, such as a journal portion. The part to be inspected may not be a camshaft, but may be other types of parts. Even in this case, the anomaly inspection system S1 can continuously capture images of the curved or flat surface of the part, and perform the inspection described in the embodiment on the continuously captured images.

[0092] In the embodiment, the inspection target part W rotates, and the camera of the anomaly inspection system S1 continuously captures the inspection target part W during rotation to obtain an image. However, the method of obtaining an image is not limited to this. For example, when the inspection target part has a flat surface and the camera continuously captures the flat surface while the illumination of the anomaly inspection system S1 irradiates the flat surface with light, it may be the inspection target part side or the camera side that moves during capture. In this way, when images including the same part on the surface of the part are continuously captured, it may be either the inspection target part or the camera (capturing unit) that moves during capture. In either case, the positional relationship between the inspection target part and the camera continues to change relatively during capture, so that the anomaly inspection system S1 can easily continuously capture images including the same part on the surface of the part.

[0093] Depending on the positional relationship between the camera, lighting, and inspection target part of the anomaly inspection system S1, and the characteristics of the inspection target part (for example, whether the photographed area is curved or flat, and if it is curved, the value of its curvature), the degree to which the false anomaly pattern is reflected in the image may differ from that shown in FIG. 2G. Even in such a case, at least one of the first inspection unit 246 or the second inspection unit 247 may select an image in which the area determined to have an appearance anomaly is located in the photographed area based on information on the photographed area in which overdetection is avoided that is stored in advance in DB21 in step S13 or S16. In this way, the problem related to the present invention can be solved.

[0094] In the abnormality inspection system S1, when the presence or absence of an abnormality of the cam portion of the inspection target part W is determined as described in the embodiment, the user may input the presence or absence of an actual abnormality of the cam portion, or the type of pattern of the cam portion shown in the image (for example, a pattern indicating a defect, or a type of pattern indicating a false abnormality) to the information processing device 20 via the input unit 23. This allows the first inspection unit 246 and the second inspection unit 247 to correct at least one of the first learning model M1 and the second learning model M2 based on the data thus fed back.

[0095] As described above, one or more processors included in the anomaly inspection system in the above-mentioned embodiments execute one or more programs including instructions for causing a computer to execute the algorithms described using the drawings. This process realizes the processes described in each embodiment.

[0096] The program includes a set of instructions (or software code) that, when loaded into a computer, causes the computer to perform one or more functions described in the embodiments. The program may be stored in a non-transitory computer-readable medium or a tangible storage medium. By way of example and not limitation, the computer-readable medium or tangible storage medium includes random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technology, CD-ROM, digital versatile disk (DVD), Blu-ray® disk or other optical disk storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device. The program may be transmitted on a transitory computer-readable medium or communication medium. By way of example and not limitation, the transitory computer-readable medium or communication medium includes electrical, optical, acoustic, or other forms of propagated signals. The program may also take the form of, for example, an application program.

[0097] Although the present invention has been described above with reference to the embodiment, the present invention is not limited to the above. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention. [Explanation of symbols]

[0098] S1 Anomaly Inspection System 10 Inspection equipment 11 Cam inspection lighting 12 Cam inspection camera 13 Journal inspection lighting 14 Journal inspection camera 15 RPM motor 20 Information processing device 21 DB 22 Display panel 23 Input section 24 Control section 241 Memory 242 I / O section 243 Information processing unit 244 Image acquisition unit 245 Judgment Department 246 First Inspection Department 247 Second Inspection Department M1 First learning model M2 Second learning model

Claims

1. an acquisition unit that acquires image data of a curved surface portion of the component captured while the component is rotated by a rotation mechanism that supports the component and rotates the component such that the same location of the component is included in different regions of each of a plurality of consecutive image data; a determination unit that detects the presence or absence of an appearance abnormality in one piece of imaging data, and when an appearance abnormality is detected in the one piece of imaging data, determines in which of three types of regions in the one piece of imaging data the location in which the appearance abnormality is reflected is present: a region on one end side of the one piece of imaging data, a region on the other end side, or a central region located between the one end side and the other end side, a rotation direction of the component when the acquisition unit acquires the one piece of imaging data is a direction from the region on the one end side toward the region on the other end side, the determination unit detects the presence or absence of the appearance abnormality in the two imaging data, where there is a possibility that a location where the appearance abnormality is reflected is present in two types of areas other than the area where the location where the appearance abnormality is reflected is present in the one imaging data out of the three types of areas, based on a result of the determination, the one imaging data and the two imaging data constitute continuous imaging data, and determine that the part is suspected of having an abnormality when the appearance abnormality is detected in all of the two imaging data. Anomaly inspection system.

2. the determination unit detects the presence or absence of an abnormality in appearance by inputting the one piece of image data and the two pieces of image data into a first learning model trained by using the image data of the component as teacher data. The anomaly inspection system according to claim 1 .

3. the determination unit inputs the one piece of imaging data acquired by the acquisition unit into a second learning model different from the first learning model, and when it is determined using an output result of the second learning model that a matching rate between the one piece of imaging data and a pattern of a false abnormality is equal to or lower than a predetermined threshold, inputs the two pieces of imaging data into the second learning model, and when it is determined using the output result of the second learning model that a matching rate between the imaging data and a pattern of a false abnormality is equal to or lower than the predetermined threshold for each of the two pieces of imaging data, determines that the part has an abnormality. The anomaly inspection system according to claim 2 .

4. The first learning model performs a semantic segmentation process on the input imaging data; The second learning model performs a classification process on the input imaging data. The anomaly inspection system according to claim 3 .

5. the acquisition unit acquires imaging data of the component such that the same location of the component is included in at least three consecutive imaging data. The abnormality inspection system according to claim 1 .

6. an acquisition step of acquiring image data of a curved surface portion of the component, the image being captured while the component is rotated by a rotation mechanism that rotates the component around an axis such that the same location of the component is included in different regions of each of a plurality of consecutive image data; a first determination step of detecting the presence or absence of an appearance abnormality in one piece of image data, and, if an appearance abnormality is detected in the one piece of image data, determining in which of three types of regions in the one piece of image data the location in which the appearance abnormality is reflected is present: a region on one end side of the one piece of image data, a region on the other end side, or a central region located between the region on the one end side and the region on the other end side, and a rotation direction of the part when acquiring the one piece of image data in the acquisition step is a direction from the region on the one end side toward the region on the other end side; a second determination step of detecting the presence or absence of the appearance abnormality in two types of image data, where a location where the appearance abnormality is reflected may be present in two types of areas other than the area where the location where the appearance abnormality is reflected in the one image data out of the three types of areas based on a result of the determination in the first determination step, the one image data and the two image data constitute continuous image data, and determining that the part is suspected of having an abnormality when the appearance abnormality is detected in all of the two image data; The anomaly inspection system performs an anomaly inspection method.

7. an acquisition step of acquiring image data of a curved surface portion of the component, the image being captured while the component is rotated by a rotation mechanism that rotates the component around an axis such that the same location of the component is included in different regions of each of a plurality of consecutive image data; a first determination step of detecting the presence or absence of an appearance abnormality in one piece of image data, and, if an appearance abnormality is detected in the one piece of image data, determining in which of three types of regions in the one piece of image data the location in which the appearance abnormality is reflected is present: a region on one end side of the one piece of image data, a region on the other end side, or a central region located between the region on the one end side and the region on the other end side, and a rotation direction of the part when acquiring the one piece of image data in the acquisition step is a direction from the region on the one end side toward the region on the other end side; a second determination step of detecting the presence or absence of the appearance abnormality in two types of image data, where a location where the appearance abnormality is reflected may be present in two types of areas other than the area where the location where the appearance abnormality is reflected in the one image data out of the three types of areas based on a result of the determination in the first determination step, the one image data and the two image data constitute continuous image data, and determining that the part is suspected of having an abnormality when the appearance abnormality is detected in all of the two image data; A program that causes a computer to execute the following.

Images