Abnormality inspection system, abnormality inspection method, and program

The dual-learning model approach in the abnormality inspection system addresses the challenge of accurately detecting abnormalities in cams with non-circular cross-sections, reducing false positives and enhancing inspection efficiency.

JP7589656B2Active Publication Date: 2024-11-26TOYOTA JIDOSHA KK
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
JP2021120602
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2021-07-21
Publication Date
2024-11-26
Estimated Expiration
2041-07-21

AI Technical Summary

Technical Problem

Existing inspection systems struggle to accurately detect abnormalities in components with non-circular cross-sections, such as cams, due to irregular changes in light irradiation positions, leading to potential false positives and inefficiencies.

Method used

An abnormality inspection system utilizing two distinct learning models to analyze imaging data of cams, where the first model identifies suspected abnormalities and the second model confirms or refines the type of abnormality, reducing false positives and improving accuracy.

Benefits of technology

The system enhances the accuracy of abnormality detection in cams by using multiple models to validate findings, minimizing false detections and optimizing inspection efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007589656000001
    Figure 0007589656000001
  • Figure 0007589656000002
    Figure 0007589656000002
  • Figure 0007589656000003
    Figure 0007589656000003
Patent Text Reader

Abstract

To provide an abnormality inspection system, an abnormality inspection method, and a program which can accurately detect the abnormality of cam parts.SOLUTION: An abnormality inspection system S1 includes: an image acquisition section 244 (first acquisition section) for acquiring imaging data of cam parts; a first inspection section 245 for inputting the imaging data of the cam parts into a first learning model M1 so as to inspect whether the cam parts are suspected of having abnormality; and a second inspection section 246 for inputting the imaging data of the cam parts which have been determined to be suspected of having the abnormality by the first inspection section 245 into a second learning model M2 so as to inspect the existence of the abnormality of the cam parts.SELECTED DRAWING: Figure 3A
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to an abnormality inspection system, an abnormality inspection method, and a program. [Background technology]

[0002] Industrial parts need to be properly inspected in the production process to prevent low-quality parts from being shipped. For example, Patent Literature 1 describes a technology for determining the specification classification of parts using a trained model trained by machine learning. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2020-149578 A Summary of the Invention [Problem to be solved by the invention]

[0004] Inspection systems that inspect parts can determine abnormalities on the surface of a part by photographing the part and analyzing the images. If the cross section of the part is a perfect circle, the relative positions of the part's surface, the camera photographing the part, and the lighting that illuminates the surface do not change, so the position where the light is irradiated on the image does not change. Therefore, the inspection system can determine whether there are abnormalities in a part by analyzing multiple images of each part of the part using AI (Artificial Intelligence) or other methods.

[0005] However, for parts such as cams whose cross sections are not perfectly round (i.e. parts whose surface has different distances from the axis of rotation and is not uniform), the relative positions of the camera photographing the part, the light illuminating the surface, and the part change, so the position where the light is irradiated in the image changes. As a result, the appearance of the surface in the image captured by the camera changes irregularly. Therefore, the above-mentioned method cannot be used, and the inspection system must analyze each captured image one by one. In such cases, the inspection system may erroneously determine that a part on the surface of the cam that is not actually abnormal is abnormal.

[0006] The present invention is intended to solve such problems, and provides an abnormality inspection system, an abnormality inspection method, and a program capable of accurately detecting abnormalities in a cam portion. [Means for solving the problem]

[0007] An abnormality inspection system according to an exemplary embodiment of the present invention includes a first acquisition unit that acquires imaging data of a cam portion, a first inspection unit that inspects whether or not the cam portion is suspected of having an abnormality by inputting the imaging data of the cam portion to a first learning model, and a second inspection unit that inspects whether or not the cam portion has an abnormality by inputting the imaging data of the cam portion determined to be suspected of having an abnormality by the first inspection unit to a second learning model different from the first learning model. Since the abnormality inspection system determines abnormalities in the cam portion using two different types of learning models, it is possible to increase the accuracy of the abnormality inspection and accurately detect abnormalities in the cam portion.

[0008] In the above anomaly inspection system, the first learning model is a model whose output result is the presence or absence of an abnormality in the appearance of the cam portion, and the first inspection unit may determine that the cam portion is suspected of having an abnormality when the output result indicates that the cam portion has an abnormality in appearance, and the second learning model is a model whose output result is the type of abnormality in the appearance of the cam portion, and the second inspection unit may determine the presence or absence of an abnormality in the cam portion according to the type of abnormality in the appearance of the cam portion indicated by the output result. Since the second inspection unit does not execute a judgment process when there is no abnormality in the appearance of the cam portion, inspection by the second inspection unit is executed only when necessary. Therefore, the anomaly inspection system can execute inspection efficiently.

[0009] In the above anomaly inspection system, when the output result of the second learning model indicates that the appearance abnormality of the cam portion is at least one of the patterns of liquid droplets, marks of a brush for deburring, check marks, and grinding wheel marks, the second inspection unit may determine that there is no abnormality in the cam portion. This makes it possible for the anomaly inspection system to suppress overdetection, since it does not determine that a mere appearance abnormality that does not affect the actual quality of the part is ultimately abnormal.

[0010] In the above anomaly inspection system, the second inspection unit may extract a section of the imaging data that is determined by the first inspection unit to be suspected of being abnormal, and input the imaging data of the extracted section to the second learning model. This allows the second inspection unit to exclude unnecessary sections from its judgment, thereby suppressing the influence of noise in the inspection by the second inspection unit and improving the accuracy of the inspection.

[0011] The abnormality inspection system may further include a second acquisition unit that acquires imaging data of a journal unit provided on the same part as the cam unit, and a third inspection unit that inspects the journal unit for abnormalities by inputting the imaging data of the journal unit into a third learning model different from the first and second learning models, and the journal unit may be inspected only by the third inspection unit. This allows the journal unit that is not a cam unit to be inspected only once instead of multiple times in part inspection, thereby reducing the total amount of calculation processing required in part inspection.

[0012] In the above-mentioned anomaly inspection system, the first acquisition unit may acquire imaging data of the cam portion captured in a state where the cam portion is irradiated with light by a first illumination, and the second acquisition unit may acquire imaging data of the journal portion captured in a state where the journal portion is irradiated with light by a second illumination different from the first illumination. Since the cam portion and the journal portion are irradiated with light by illumination that matches their respective shapes, the first acquisition unit and the second acquisition unit can acquire imaging data of a quality suitable for inspection. Therefore, the anomaly inspection system can more accurately detect anomalies in the cam portion.

[0013] In the above-mentioned anomaly inspection system, the first acquisition unit may acquire image data of the cam portion captured in a state in which the cam portion is rotated by a rotation mechanism that axially supports and rotates the shaft of the cam portion. This allows the anomaly inspection system to efficiently acquire images of different locations of the cam portion, thereby shortening the time required for the entire inspection.

[0014] In an abnormality inspection method according to an exemplary embodiment of the present invention, an abnormality inspection system executes an acquisition step of acquiring imaging data of a cam portion, a first inspection step of inspecting whether or not there is a suspected abnormality in the cam portion by inputting the imaging data of the cam portion to a first learning model, and a second inspection step of inspecting whether or not there is an abnormality in the cam portion by inputting the imaging data of the cam portion determined to be suspected to be abnormal to a second learning model different from the first learning model. Since the abnormality inspection system determines an abnormality in the cam portion using two different types of learning models, it is possible to increase the accuracy of the abnormality inspection and to accurately detect an abnormality in the cam portion.

[0015] A program according to an exemplary embodiment of the present invention causes a computer to execute an acquisition step of acquiring imaging data of a cam portion, a first inspection step of inspecting whether or not there is a suspicion of abnormality in the cam portion by inputting the imaging data of the cam portion to a first learning model, and a second inspection step of inspecting whether or not there is an abnormality in the cam portion by inputting the imaging data of the cam portion determined to be suspected of being abnormal to a second learning model different from the first learning model. With this program, the computer determines abnormality in the cam portion using two different types of learning models, thereby improving the accuracy of the inspection for abnormality and enabling accurate detection of abnormality in the cam portion. Effect of the Invention

[0016] The present invention provides an abnormality inspection system, an abnormality inspection method, and a program capable of accurately detecting an abnormality in a cam portion. [Brief description of the drawings]

[0017] [Figure 1] 1 is a block diagram showing an example of an abnormality inspection system according to a first embodiment. [Figure 2A] FIG. 2 is a detailed diagram illustrating an example of an inspection device of the anomaly inspection system according to the first embodiment. [Figure 2B] FIG. 2 is a diagram showing an example of an inspection target part according to the first embodiment; [Figure 2C]3 is a schematic diagram showing an example of the positional relationship between a cam portion, a lighting for inspecting the cam portion, and a camera for inspecting the cam portion according to the first embodiment; FIG. [Figure 3A] FIG. 2 is a block diagram showing an example of a control unit according to the first embodiment. [Figure 3B] 1 is a schematic diagram showing the relationship between an inspection unit and a learning model according to the first embodiment. FIG. [Figure 3C] FIG. 2 is a schematic diagram showing an example of image clipping according to the first embodiment; [Figure 4A] 5 is a flowchart showing an example of a process performed when the abnormality inspection system according to the first embodiment inspects a cam portion. [Figure 4B] 5 is a flowchart showing an example of a process performed when the abnormality inspection system according to the first embodiment inspects a cam portion. [Figure 4C] 5 is a flowchart showing an example of a process performed when the abnormality inspection system according to the first embodiment inspects a joint portion. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] First embodiment Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0019] <Abnormality inspection system> Fig. 1 is a diagram for explaining an abnormality inspection system according to an embodiment. As shown in Fig. 1, the abnormality inspection system S1 according to the present embodiment includes an inspection device 10 and an information processing device 20. In the abnormality inspection system S1 according to the present embodiment, a user operates the information processing device 20 to cause a camera of the inspection device 10 to capture an image of a surface of an inspection target part, and causes the information processing device 20 to inspect whether or not there is an abnormality on the surface. In this example, the inspection target part is a camshaft. Details of the inspection device 10 and the information processing device 20 will be described below.

[0020] The inspection device 10 has a cam portion inspection light 11, a cam portion inspection camera 12, a journal portion inspection light 13, a journal portion inspection camera 14, and a rotation motor 15.

[0021] 2A is a detailed diagram showing an example of the inspection device 10, and each component of the inspection device 10 will be described using this diagram. As the cam portion inspection lighting 11, a cam portion inspection lighting 11A is provided on the front side of the inspection target part W, and a cam portion inspection lighting 11B is provided on the rear side thereof, and the information processing device 20 turns on each lighting when inspecting the cam portion of the inspection target part W. As the cam portion inspection cameras 12, cam portion inspection cameras 12A-12D are provided from the front side to the rear side of the inspection target part W. Each cam portion inspection camera 12 captures imaging data (images) of the cam portion by continuously capturing an image of the area of ​​the cam portion irradiated with light by the cam portion inspection lighting 11 multiple times under the control of the information processing device 20.

[0022] The journal portion inspection lighting 13 is turned on under the control of the information processing device 20 when the journal portion of the inspection target part W is inspected. As the journal portion inspection cameras 14, journal portion inspection cameras 14A-14D are provided from the front side to the rear side of the inspection target part W. Under the control of the information processing device 20, each journal portion inspection camera 14 captures the image data of the journal portion by continuously photographing the area of ​​the journal portion illuminated by the journal portion inspection lighting 13 multiple times. Note that the cam portion inspection camera 12 and the journal portion inspection camera 14 are so-called area cameras, which are provided so that the imaging elements of the cameras are planar (i.e., there are multiple imaging elements vertically and horizontally).

[0023] 2B is an enlarged view of a camshaft as the inspection target part W. This camshaft has eight cam portions C1-C8 and four journal portions J1-J4. A cam portion inspection lighting device 11 irradiates light onto the cam portions C1-C8, and a cam portion inspection camera 12 photographs the illuminated area. A journal portion inspection lighting device 13 irradiates light onto the journal portions J1-J4, and a journal portion inspection camera 14 photographs the illuminated area.

[0024] FIG. 2C is a schematic diagram showing an example of the positional relationship between the cam portion C1, the cam portion inspection illumination 11A, and the cam portion inspection camera 12A. The cam portion inspection illumination 11A and the cam portion inspection camera 12A are arranged such that the front parts of the illumination and the camera are aligned on the same line, and the cam portion C1 is arranged on the extension line. In other words, the cam portion inspection illumination 11A and the cam portion inspection camera 12A are arranged so as to have the same attitude (same phase) with respect to the cam portion C1. The cam portion inspection illumination 11A irradiates light onto the cam portion C1. The cam portion inspection camera 12A has an angle of view θ for photographing the rotation direction of the cam portion, and photographs the photographing area RC in the cam portion. The cam portion inspection illumination 11A and the cam portion inspection camera 12A are arranged in the same positional relationship not only with respect to the cam portion C1 but also with respect to other cam portions, and can photograph a predetermined photographing area of ​​the cam portion in the same manner. In addition, the cam portion inspection lighting 11B and the cam portion inspection camera 12B are also positioned in the same positional relationship with respect to each cam portion as the cam portion inspection lighting 11A and the cam portion inspection camera 12A, respectively, and can similarly photograph a specified shooting area of ​​the cam portion.

[0025] Returning to Fig. 2A, the description will be continued. The rotary motor 15, together with a clamp FC provided on the front side of the inspection target part W and a clamp RC provided on the rear side, constitutes a rotation mechanism that axially supports and rotates the shaft of the inspection target part W. After the user fixes both ends of the inspection target part W with the clamps FC and RC, thereby making the inspection target part W rotatable, the rotary motor 15 rotates the inspection target part W under the control of the information processing device 20.

[0026] In detail, when inspecting the cam portion of the inspection target part W, the information processing device 20 rotates the rotary motor 15, and while the inspection target part W rotates once with the cam portion inspection illumination 11 turned on, controls the cam portion inspection camera 12 to continuously photograph the cam portion multiple times. This allows the cam portion inspection camera 12 to photograph a different area of ​​the cam portion for each photograph.

[0027] After photographing the cam portion as described above, the information processing device 20 turns off the cam portion inspection lighting 11 and instead turns on the journal portion inspection lighting 13. Then, the information processing device 20 rotates the rotary motor 15, and controls the journal portion inspection camera 14 to continuously photograph the journal portion multiple times while the inspection target part W rotates once. This allows the journal portion inspection camera 14 to photograph a different area of ​​the journal portion for each photographing.

[0028] The information processing device 20 determines the photographing interval and number of photographs (photographing time) of the cam portion inspection camera 12, and the rotation speed of the rotary motor 15, so that the cam portion inspection camera 12 photographs the entire surface of the cam portion of the inspection target part W by photographing multiple times. The information processing device 20 also determines the photographing interval and number of photographs (photographing time) of the journal portion inspection camera 14, and the rotation speed of the rotary motor 15, so that the journal portion inspection camera 14 photographs the entire surface of the journal portion of the inspection target part W by photographing multiple times. The information processing device 20 acquires the images of the cam portion and journal portion obtained as described above, and determines whether or not there is an abnormality in each image by executing the processing described below.

[0029] In addition, in the images successively taken by the cam portion inspection camera 12, the area of ​​the cam portion photographed in one image and the area of ​​the cam portion photographed in the previous and next images may or may not overlap. The same can be said about the photographing of the journal portion by the journal portion inspection camera 14.

[0030] Next, a description will be given of the information processing device 20. The information processing device 20 includes a DB (Database) 21, a display panel 22, an input unit 23, and a control unit 24.

[0031] DB21 stores three types of learning models required for inspection. These learning models are AI (Artificial Intelligence) models that have been trained in advance using images as training data through machine learning such as deep learning. Details of these learning models will be described later. DB21 also stores all thresholds used for judgment in the inspection unit described later.

[0032] DB21 is configured with a storage device such as a flash memory, a memory card, a HDD (Hard Disk Drive), an optical disk drive, etc., but the type of storage device is not limited to these. DB21 may be provided outside the information processing device 20. In this case, the information processing device 20 may connect to DB21 via an information transmitting / receiving unit (not shown) and obtain the data stored in DB21.

[0033] The display panel 22 is an interface that enables a user to view the determination result of the presence or absence of an abnormality in the information processing device 20. The input unit 23 is an interface that enables a user to input instructions to the information processing device 20 regarding the start of an examination and the settings for the examination.

[0034] The control unit 24 controls the cam portion inspection lighting 11, the cam portion inspection camera 12, the journal portion inspection lighting 13, the journal portion inspection camera 14, and the rotary motor 15 as described above to capture images of the cam portion and journal portion of the inspection target part W. Then, the control unit 24 obtains the images captured by the cam portion inspection camera 12 and the journal portion inspection camera 14, respectively, and performs the inspection as described below.

[0035] 3A is a block diagram for explaining the configuration of the control unit 24. The control unit 24 includes a memory 241, an I / O (Input / Output) unit 242, and an information processing unit 243. Each unit of the control unit 24 will be explained below.

[0036] The memory 241 is composed of a volatile memory, a non-volatile memory, or a combination thereof. The memory 241 is not limited to one, and a plurality of memories may be provided. The volatile memory may be, for example, a RAM (Random Access Memory) such as a DRAM (Dynamic Random Access Memory) or a SRAM (Static Random Access Memory). The non-volatile memory may be, for example, a PROM (Programmable ROM), an EPROM (Erasable Programmable Read Only Memory), or a flash memory.

[0037] The memory 241 is used to store one or more instructions. Here, the one or more instructions are stored in the memory 241 as a group of software modules. The information processing unit 243 can perform the following processes by reading and executing the one or more instructions from the memory 241.

[0038] The I / O unit 242 is a hardware interface that executes input and output of information to and from the outside of the control unit 24. In this embodiment, the control unit 24 is connected to the cam portion inspection lighting 11, the cam portion inspection camera 12, the journal portion inspection lighting 13, the journal portion inspection camera 14, and the rotary motor 15, and appropriately executes input and output of information with these via the I / O unit 242.

[0039] The information processing unit 243 is composed of any processor or the like for analyzing images. In this example, the information processing unit 243 includes a GPU (Graphics Processing Unit) useful for image processing as a processor. However, the information processing unit 243 may include a CPU (Central Processing Unit), an MPU (Micro Processing Unit), an FPGA (Field-Programmable Gate Array), a DSP (Digital Signal Processor), and an ASIC (Application Specific Integrated Circuit) as a processor. The above-mentioned memory 241 may include one built into the information processing unit 243 in addition to one provided outside the information processing unit 243.

[0040] The information processing unit 243 realizes functions of an image acquisition unit 244, a first inspection unit 245, a second inspection unit 246, a third inspection unit 247, etc. by reading and executing software (computer program) from the memory 241. The image acquisition unit 244 acquires images captured by the cam portion inspection camera 12 and the journal portion inspection camera 14 via the I / O unit 242. The acquired image of the cam portion is output to the first inspection unit 245, and the image of the journal portion is output to the third inspection unit 247.

[0041] Fig. 3B shows the first inspection unit 245 to the third inspection unit 247 and three types of learning models stored in the DB 21 and accessed by the first inspection unit 245 to the third inspection unit 247 for inspection. Each inspection unit will be described in detail below with reference to Fig. 3B.

[0042] The first inspection unit 245 inputs an image (first image data) of the cam portion captured by the cam portion inspection camera 12 into the first learning model M1, and determines whether or not there is a suspected abnormality in the cam portion based on the result output from the first learning model M1.

[0043] Here, the first learning model M1 is a model that is learned by using an image of the cam portion as teacher data, and outputs a calculated value regarding the presence or absence of an abnormality in the appearance of the cam portion as an output result when an image is input. As a detailed example, the first learning model M1 performs a semantic segmentation process on the input image to calculate and output a first judgment value for the image. This first judgment value represents the normality of the cam portion shown in the image, and the larger the value, the cleaner the surface of the cam portion in the image is, and the smaller the value, the more a pattern resembling a defect such as a scratch or a shrinkage cavity is shown on the surface. The first inspection unit 245 compares the calculated first judgment value with a threshold value TH1 stored in the DB21, and judges an image for which the first judgment value is equal to or less than the threshold value TH1 as an image for which the cam portion has an abnormality in the appearance. On the other hand, the first inspection unit 245 judges an image for which the first judgment value is greater than the threshold value TH1 as an image for which the cam portion has no abnormality in the appearance.

[0044] The first learning model M1 may divide an image into a plurality of sections and calculate the first judgment value for each section. In this case, the first inspection unit 245 may determine that the image has an appearance abnormality in the cam portion when the first judgment value in at least one section of the image is equal to or less than the threshold value TH1. Alternatively, the first inspection unit 245 may determine that the image has an appearance abnormality in the cam portion when the first judgment value in a number of sections equal to or greater than a plurality of threshold values ​​is equal to or less than the threshold value TH1.

[0045] An abnormality in appearance means that the surface of the cam part is not clean, and for example, it means that there is a pattern such as a line or a circle on the surface, or that there is a part on the surface that is different in brightness (for example, a dark part compared to the surroundings). If there is a pattern such as a circle on the surface of the cam part, or if there is a dark part on the surface, there is a possibility that a casting cavity has occurred on the surface, and if there is a line pattern on the surface, there is a possibility that there is a scratch on the surface. The scratch or casting cavity means a defect (substantial abnormality) of the inspected part W. However, even if there is a pattern such as a circle on the surface of the cam part, the pattern may be caused by droplets on the surface (for example, cleaning liquid). In addition, even if there is a line pattern on the surface, the pattern may be a mark of a brush for deburring, a check mark on the inspected part W, or a grinding stone mark generated during grinding. These patterns are naturally generated by the processing of the inspected part W in the manufacturing stage, and are not defects of the inspected part W. Therefore, there is a possibility that an image having such a pattern that has an abnormality in appearance but is not a substantial abnormality (hereinafter also referred to as a false abnormality) may be judged as an abnormality by the anomaly inspection system, which is an overdetection problem.

[0046] In the present invention, when it is determined that the cam portion has an appearance abnormality in the image, the first inspection unit 245 determines that the cam portion is suspected of having an abnormality. The image determined to be suspected of having an abnormality is subject to re-inspection by the second inspection unit 246. Only when it is determined through this re-inspection that the image shows a defect is the inspection target part W determined to be abnormal for the first time. Note that the image determined to be free of suspicion of abnormality is not subject to inspection by the second inspection unit 246.

[0047] The second inspection unit 246 inputs the entire image (second imaging data) of the cam portion determined to be suspected of being abnormal to the second learning model M2. Note that, when the first learning model M1 is a model that divides an image into a plurality of sections and calculates a first judgment value for each section, the second inspection unit 246 may cut out the section of the image where the first judgment value is equal to or less than the threshold value TH1, and input only the image of the cut-out section to the second learning model M2.

[0048] 3C is a diagram showing an example of the process of cutting out an image by the second inspection unit 246. Due to the presence of a suspected abnormal pattern DA in the original image IM captured by the cam portion inspection camera 12, the first inspection unit 245 determines that the first judgment value in the section DA of the image IM is equal to or less than the threshold value TH1. At this time, the second inspection unit 246 can cut out the section DA from the image IM and input only the image of the section DA to the second learning model M2.

[0049] The second learning model M2 outputs the calculation result based on the image input as described above. The second inspection unit 246 judges whether or not there is an abnormality in the cam portion based on the output result from the second learning model M2.

[0050] Here, the second learning model M2 is a different type of AI model from the first learning model M1. The second learning model M2 is learned using an image of the cam part as teacher data, and is a model that outputs a calculated value related to the type of abnormality in the appearance of the cam part as an output result when the image is input. In detail, the second learning model M2 executes a classification process on the input image, calculates and outputs a second judgment value for the input image. This second judgment value is a matching rate between the input image and the above-mentioned false abnormality pattern, and refers to a cosine value (cosθ) when the inner product of the input image and the false abnormality pattern is calculated. The second learning model M2 calculates the second judgment value for the number of types of false abnormality patterns modeled. The larger (closer to 1) the second judgment value is, the more similar the pattern on the surface of the cam part in the image is to the false abnormality pattern (for example, at least one of the patterns of droplets, brush marks for deburring, check marks, or grindstone marks), indicating that there is no substantial abnormality. On the other hand, the smaller the second judgment value, the less similar the pattern on the cam portion surface is to the pattern of a false anomaly, and therefore the more likely the pattern is to be a defect such as a scratch or a blowhole.

[0051] The second inspection unit 246 compares the calculated second judgment value with the threshold value TH2 stored in the DB21. The threshold value TH2 is set as many times as the number of types of modeled false abnormality patterns. The second inspection unit 246 judges an image in which the second judgment value is equal to or less than the threshold value TH2 for all types of false abnormality patterns as an image in which the cam portion has an abnormality. On the other hand, the second inspection unit 246 judges an image in which the second judgment value is greater than the threshold value TH2 for at least one type of false abnormality pattern as an image in which the cam portion has no abnormality.

[0052] In addition, when the second inspection unit 246 inputs only an image of a section cut out from a portion of the original photographed image to the second learning model M2, the second learning model M2 performs the above-mentioned classification process on that section, calculates and outputs a second judgment value for that section.

[0053] The first inspection unit 245 and the second inspection unit 246 execute the above-mentioned process for each image captured by each cam portion inspection camera 12. Then, if there is no image in which the second inspection unit 246 has determined that there is an abnormality in the cam portion, the second inspection unit 246 determines that there is no abnormality in the cam portion, which is the photographed inspection target. On the other hand, if there is at least one image in which it has been determined that there is an abnormality in the cam portion, the second inspection unit 246 determines that there is an abnormality in the cam portion, which is the photographed inspection target. However, the second inspection unit 246 may determine that there is an abnormality in the cam portion, which is the photographed inspection target, when there are a predetermined number of images determined to have an abnormality in the cam portion or more. If the second inspection unit 246 has determined that there is an abnormality in the cam portion, which is the inspection target, the second inspection unit 246 determines that there is an abnormality in the inspection target part W.

[0054] The second inspection unit 246 can display the above-mentioned determination results on the display panel 22 of the information processing device 20. The second inspection unit 246 may identify the cam portion determined to have an abnormality by identifying the cam portion inspection camera 12 that photographed the cam portion and the photographing direction, and display the determination result on the display panel 22.

[0055] The third inspection unit 247 also inputs the image of the journal part (third imaging data) captured by the journal part inspection camera 14 to the third learning model M3, and judges whether or not there is an abnormality in the journal part based on the result output from the third learning model M3. For example, the third inspection unit 247 joins together multiple images of the same part of the journal part from all the captured images of the journal part in the order of the timing of capture to generate one image. Then, it generates one image for each different part so as to cover the surface of the journal part, and inputs the images to the third learning model M3.

[0056] Here, the third learning model M3 is an AI model different from the first learning model M1 and the second learning model M2. The third learning model M3 is a model that is learned by using images of the journal part as teacher data, and is a model that outputs a calculated value regarding the presence or absence of an abnormality in the journal part as an output result when an image is input. In detail, the third learning model M3 executes a segmentation process on the input image, calculates and outputs a third judgment value for the image. This third judgment value represents the normality of the journal part reflected in the image, and the larger the value, the cleaner the surface of the cam part in the image is, and the smaller the value, the more a pattern resembling a defect such as a scratch or a blowhole is reflected on the surface. The third inspection unit 247 compares the calculated third judgment value with a threshold value TH3 stored in the DB21, and judges an image for which the third judgment value is equal to or less than the threshold value TH3 as an image for which the journal part has an abnormality. On the other hand, the third inspection unit 247 judges an image for which the third judgment value is greater than the threshold value TH3 as an image for which the journal part has no abnormality.

[0057] The third inspection unit 247 can execute the above-mentioned image judgment process for each location on the journal surface photographed by each cam inspection camera 12. If there is no image in which the journal is judged to have an abnormality in the third inspection unit 247, the third inspection unit 247 judges that there is no abnormality in the photographed journal that is the inspection target. On the other hand, if there is even one image in which the journal is judged to have an abnormality, the third inspection unit 247 judges that there is an abnormality in the photographed journal that is the inspection target. However, the third inspection unit 247 may also judge that there is an abnormality in the photographed journal that is the inspection target when there are more than one threshold value of images judged to have an abnormality in the journal. If the third inspection unit 247 judges that there is an abnormality in the journal that is the inspection target, it judges that there is an abnormality in the inspection target part W.

[0058] The third inspection unit 247 can display the above-mentioned determination results on the display panel 22 of the information processing device 20. Note that the third inspection unit 247 may identify a journal part determined to have an abnormality by identifying the journal part inspection camera 14 that photographed the journal part and the photographing direction, and display the determination result on the display panel 22.

[0059] 4A and 4B are flowcharts showing an example of a process in which the abnormality inspection system S1 inspects the cam portion, and this process will be described below with reference to Fig. 4A and 4B. Note that the details of each process are as described above, and the description will be omitted as appropriate.

[0060] First, the user operates the input unit 23, whereby the information processing device 20 causes the inspection device 10 to inspect the cam portion of the inspection target part W. As this inspection is performed, the image acquisition unit 244 reads all images captured by the cam portion inspection camera 12 (step S11).

[0061] The first inspection unit 245 inputs one image from the read images to the first learning model M1. Then, it determines whether the first judgment value calculated by the first learning model M1 is equal to or smaller than the threshold value TH1 (step S12).

[0062] If the first judgment value is equal to or less than the threshold value TH1 (Yes in step S12), the second inspection unit 246 cuts out the section of the image in which the first judgment value is equal to or less than the threshold value TH1 (step S13). Then, the second inspection unit 246 inputs the image of the cut-out section to the second learning model M2. The second inspection unit 246 determines whether the second judgment value calculated by the second learning model M2 is equal to or less than the threshold value TH2 for all types of false anomaly patterns (step S14).

[0063] If the second judgment value is less than or equal to the threshold value TH2 for all types of false abnormality patterns (Yes in step S14), the second inspection unit 246 judges that there is an abnormality in the cam portion based on the judgment result of the image, and the abnormality inspection system S1 terminates the inspection process (step S15).

[0064] On the other hand, in step S12, if the first judgment value is greater than the threshold value TH1 (No in step S12), the first inspection unit 245 judges that the image to be inspected is normal (no abnormality) (step S16). Also, in step S14, if the second judgment value is greater than the threshold value TH2 in at least one type of false abnormality pattern (No in step S14), the second inspection unit 246 also judges that the image to be inspected is normal (step S16).

[0065] Thereafter, the first inspection unit 245 judges whether the above-described judgment has been completed for all images read in step S11 (step S17). If there are images for which judgment has not been completed (No in step S17), the third inspection unit 247 returns to step S12 and executes processing on the unjudged images. On the other hand, if judgment has been completed for all images (Yes in step S17), the first inspection unit 245 judges that the cam portion is normal, and the abnormality inspection system S1 ends the inspection process (step S18).

[0066] In the above flow, the information processing device 20 executes inspection for one image by the first inspection unit 245 and the second inspection unit 246, and executes the inspection for each image in turn. However, the information processing device 20 may execute inspection for a plurality of images (e.g., all images read) collectively by the first inspection unit 245, and execute inspection by the second inspection unit 246 for images for which the first judgment value is equal to or less than the threshold value TH1.

[0067] Fig. 4C is a flowchart showing an example of a process in which the abnormality inspection system S1 inspects a joint portion, and this process will be described below with reference to Fig. 4C. Note that the details of each process are as described above, and the description will be omitted as appropriate.

[0068] First, the user operates the input unit 23, whereby the information processing device 20 causes the inspection device 10 to inspect the journal portion of the inspection target part W. As this inspection is performed, the image acquisition unit 244 reads all images captured by the journal portion inspection camera 14 (step S21).

[0069] The third inspection unit 247 uses the images of all the journal parts that have been read to generate one image for each location on the surface of the journal part. The details are as described above. The third inspection unit 247 then inputs the image to the third learning model M3 and determines whether the third judgment value calculated by the third learning model M3 is equal to or smaller than the threshold value TH3 (step S22).

[0070] If the third judgment value is equal to or less than the threshold value TH3 (Yes in step S22), the third inspection unit 247 judges that there is an abnormality in the journal portion based on the judgment result of the image, and the abnormality inspection system S1 ends the inspection (step S23).

[0071] On the other hand, if the third judgment value is greater than the threshold value TH3 (No in step S22), the third inspection unit 247 judges that the image to be inspected is normal (step S24).

[0072] Thereafter, the third inspection unit 247 judges whether the above-described judgment has been completed for the images of all different locations on the surface of the journal portion (step S25). If there are images for which judgment has not been completed (No in step S25), the third inspection unit 247 returns to step S22 and executes processing on the unjudged images. On the other hand, if judgment has been completed for all images (Yes in step S25), the third inspection unit 247 judges that the journal portion is normal, and the abnormality inspection system S1 ends the inspection (step S26).

[0073] As described above, in the abnormality inspection system S1, the first inspection unit 245 inspects whether or not there is a suspected abnormality in the cam portion by inputting the imaging data of the cam portion to the first learning model M1. Then, the second inspection unit 246 inspects whether or not there is a suspected abnormality in the cam portion by inputting the imaging data of the cam portion determined to be abnormal to the second learning model M2.

[0074] When inspecting a part such as a cam part, which is a special part that is not a perfect circle, the above-mentioned method of inspecting the journal part cannot be applied, so the anomaly inspection system needs to inspect the many captured images of the cam part one by one. The applicant has newly discovered a problem that in such a case, when the inspection system judges the cam part using one type of AI model, the AI ​​model is trained to suppress overlooking defects such as blowholes and scratches, which results in a high overdetection rate in the judgment. The reason for this is that the threshold value used for anomaly judgment in the AI ​​model is trained to suppress overlooking, so that the AI ​​model judges as anomaly even patterns such as droplets and grinding stone marks that have similar shapes and brightness to the defect patterns (false anomaly patterns).

[0075] On the other hand, the applicant has newly discovered that, when an AI model is trained to suppress the overdetection rate, the AI ​​model is able to recognize false abnormality patterns more accurately, but the rate at which false abnormalities are overlooked increases. This is because the AI ​​model may erroneously determine that a defect pattern in a cam section is a false abnormality pattern, or that an image containing both a defect pattern and a false abnormality pattern is not abnormal.

[0076] The anomaly inspection system S1 according to the present invention judges anomalies in the cam portion using two different learning models, and therefore can improve the accuracy of the anomaly inspection even in a situation where the patterns reflected in the image to be judged are diverse. Therefore, the anomaly inspection system S1 can accurately detect anomalies in the cam portion. In addition, since the accuracy of the anomaly inspection can be improved by software processing, the imaging unit that captures the part to be inspected can be a relatively inexpensive one such as an area camera.

[0077] The second learning model M2 may be a model with higher accuracy than the first learning model M1. In this case, by using the first learning model M1, the number of images input to the second learning model M2, which requires a long processing time, can be reduced, thereby reducing the time required for the entire calculation.

[0078] The first learning model M1 is a model that outputs the presence or absence of an abnormality in the appearance of the cam portion, and the first inspection unit 245 may determine that the cam portion is suspected of having an abnormality when the output result indicates that the cam portion has an abnormality in appearance. The second learning model M2 is a model that outputs the type of abnormality in the appearance of the cam portion, and the second inspection unit 246 may determine the presence or absence of an abnormality in the cam portion according to the type of abnormality in the appearance of the cam portion indicated by the output result. The second inspection unit 246 does not execute a determination process when there is no abnormality in the appearance of the cam portion, and therefore executes the inspection only when necessary. Therefore, the abnormality inspection system S1 can execute the inspection efficiently.

[0079] Furthermore, when the output result of the second learning model M2 indicates that the abnormality in the appearance of the cam portion is at least one of the patterns of liquid droplets, marks of a brush for deburring, check marks, and grinding wheel marks, the second inspection unit 246 may determine that there is no abnormality in the cam portion. This allows the abnormality inspection system S1 to suppress overdetection because it does not determine that a mere abnormality in appearance that does not affect the actual quality of the part is ultimately abnormal.

[0080] The second inspection unit 246 may also extract sections of the imaging data that are determined by the first inspection unit 245 to be suspected of being abnormal, and input the imaging data of the extracted sections to the second learning model M2. This allows the second inspection unit to exclude unnecessary sections from its judgment, thereby reducing the number and types of noise that may occur in the inspection by the second inspection unit, and improving the accuracy of the inspection.

[0081] In addition, in the inspection of the target inspection part W, the journal part that is not a cam part may be inspected once by the third inspection unit 247 instead of multiple times. As described above, for parts such as cam parts, the accuracy of the judgment is limited when a segmentation or classification method using one type of AI model is used for inspection, but since the shape of the journal part is a perfect circle, such a problem does not arise. Therefore, by limiting the number of inspections to one, the total amount of calculation processing required for the inspection of the entire target part can be reduced, and efficient inspection can be performed. Note that the inspection parameters in the third learning model M3 can also be set according to the target inspection time, first-pass rate, and other numerical values.

[0082] Furthermore, the image acquisition unit 244 (first and second acquisition units) may acquire imaging data of the cam portion photographed with light irradiated onto the cam portion by the cam portion inspection illumination 11 (first illumination), and may acquire imaging data of the journal portion photographed with light irradiated onto the journal portion by the journal portion inspection illumination 13 (second illumination). Since the cam portion and the journal portion are irradiated with light by illumination suited to their respective shapes, the image acquisition unit 244 can acquire imaging data of a quality suitable for inspection. Therefore, the abnormality inspection system S1 can more accurately detect abnormalities in the cam portion.

[0083] Furthermore, the image acquiring unit 244 may acquire image data of the cam portion captured in a state in which the cam portion is axially supported and rotated by the rotary motor 15. This allows the abnormality inspection system S1 to efficiently acquire images of different locations of the cam portion, thereby shortening the time required for the entire inspection.

[0084] The present invention is not limited to the above-described embodiment, and can be modified as appropriate without departing from the spirit and scope of the present invention.

[0085] For example, the part to be inspected may not be a camshaft but may be another type of part that has a cam portion that is not perfectly round. Even in this case, the abnormality inspection system S1 can perform the inspection described in the embodiment on the cam portion of the part.

[0086] In the embodiment, a two-stage inspection of the cam portion by the first inspection unit 245 and the second inspection unit 246 has been described, but three or more stages of inspection may be performed as the inspection of the cam portion. For example, an inspection other than that by the second inspection unit 246 may be performed either before or after the inspection by the first inspection unit 245.

[0087] In the inspection, the cam portion does not have to have its entire surface photographed and inspected, but only a part of its surface may be photographed and inspected. This also applies to the journal portion.

[0088] In the abnormality inspection system S1, when the presence or absence of an abnormality of the cam part of the inspection target part W is judged as described in the embodiment, the user may input the presence or absence of an actual abnormality of the cam part or the type of pattern of the cam part reflected in the image (for example, a pattern indicating a defect or a type of pattern indicating a false abnormality) to the information processing device 20 by the input unit 23. This allows the first inspection unit 245 and the second inspection unit 246 to correct at least one of the first learning model M1 and the second learning model M2 based on the data fed back in this way. Similarly, when the presence or absence of an abnormality of the journal part of the inspection target part W is judged, the user may input the presence or absence of an actual abnormality of the journal part or the type of pattern of the journal part reflected in the image to the information processing device 20 by the input unit 23. This allows the third inspection unit 247 to correct the third learning model M3 based on the data fed back in this way.

[0089] As described above, one or more processors included in the anomaly inspection system in the above-mentioned embodiments execute one or more programs including instructions for causing a computer to execute the algorithms described using the drawings. This process realizes the processes described in each embodiment.

[0090] The program includes a set of instructions (or software code) that, when loaded into a computer, causes the computer to perform one or more functions described in the embodiments. The program may be stored in a non-transitory computer-readable medium or a tangible storage medium. By way of example and not limitation, the computer-readable medium or tangible storage medium includes random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technology, CD-ROM, digital versatile disk (DVD), Blu-ray® disk or other optical disk storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device. The program may be transmitted on a transitory computer-readable medium or communication medium. By way of example and not limitation, the transitory computer-readable medium or communication medium includes electrical, optical, acoustic, or other forms of propagated signals. The program may also take the form of, for example, an application program.

[0091] Although the present invention has been described above with reference to the embodiment, the present invention is not limited to the above. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention. [Explanation of symbols]

[0092] S1 Anomaly Inspection System 10 Inspection equipment 11 Cam inspection lighting 12 Cam inspection camera 13 Journal inspection lighting 14 Journal inspection camera 15 RPM motor 20 Information processing device 21 DB 22 Display panel 23 Input section 24 Control section 241 Memory 242 I / O section 243 Information processing unit 244 Image acquisition unit 245 1st Inspection Department 246 2nd Inspection Department 247 3rd Inspection Department M1 First learning model M2 Second learning model M3 Third Learning Model

Claims

1. A first acquisition unit that acquires imaging data of the cam portion; a first inspection unit that inputs imaging data of the cam portion into a first learning model that is a model whose output result indicates the presence or absence of an abnormality in the appearance of the cam portion, and determines that the cam portion is suspected of having an abnormality when the output result indicates that the cam portion has an abnormality in appearance; a second inspection unit that inputs imaging data of the cam portion that is determined to be suspected of being abnormal by the first inspection unit into the second learning model, which is different from the first learning model and has an output result indicating a type of abnormality in the appearance of the cam portion, and determines whether or not there is an abnormality in the cam portion according to the type of abnormality in the appearance of the cam portion indicated by the output result; A second acquisition unit that acquires imaging data of a journal portion provided on the same part as the cam portion; a third inspection unit that inspects the presence or absence of an abnormality in the journal unit by inputting imaging data of the journal unit into the third learning model, which is different from the first and second learning models and outputs the presence or absence of an abnormality in the journal unit; The journal portion is inspected only by the third inspection unit. Anomaly inspection system.

2. The first learning model performs a semantic segmentation process on the input image data of the cam portion; The second learning model performs classification processing on the input image data of the cam portion; The third learning model performs a segmentation process on the input image data of the journal section. The anomaly inspection system according to claim 1 .

3. When the output result of the second learning model indicates that the abnormality in the appearance of the cam portion is at least one of a droplet, a brush mark for deburring, a check mark, and a grindstone mark, the second inspection unit determines that there is no abnormality in the cam portion. The abnormality inspection system according to claim 1 or 2.

4. The second inspection unit extracts a section of the imaging data that is determined by the first inspection unit to be suspected of having an abnormality, and inputs the imaging data of the extracted section into the second learning model. The abnormality inspection system according to claim 1 .

5. The first acquisition unit acquires imaging data of the cam portion captured in a state in which the cam portion is irradiated with light by a first illumination, The second acquisition unit acquires imaging data of the journal portion captured in a state in which the journal portion is irradiated with light by a second illumination different from the first illumination. The abnormality inspection system according to claim 1 .

6. The first acquisition unit acquires imaging data of the cam portion captured in a state in which the cam portion is rotated by a rotation mechanism that axially supports and rotates a shaft of the cam portion. The abnormality inspection system according to claim 1 .

7. a first acquisition step of acquiring image data of the cam portion; a first inspection step of inputting imaging data of the cam portion into a first learning model which is a model whose output result indicates the presence or absence of an abnormality in the appearance of the cam portion, and judging that the cam portion is suspected of having an abnormality when the output result indicates that the cam portion has an abnormality in appearance; a second inspection step of inputting imaging data of the cam portion determined to be suspected of having an abnormality in the first inspection step into a second learning model different from the first learning model, the second learning model having an output result indicating a type of abnormality in the appearance of the cam portion, and determining whether or not there is an abnormality in the cam portion according to the type of abnormality in the appearance of the cam portion indicated by the output result; a second acquisition step of acquiring image data of a journal portion provided on the same part as the cam portion; a third inspection step of inspecting the presence or absence of an abnormality in the journal portion by inputting imaging data of the journal portion into a third learning model different from the first and second learning models and outputting the presence or absence of an abnormality in the journal portion as an output result; The journal portion is inspected only by the third inspection step. An anomaly inspection method performed by the anomaly inspection system.

8. a first acquisition step of acquiring image data of the cam portion; a first inspection step of inputting imaging data of the cam portion into a first learning model which is a model whose output result indicates the presence or absence of an abnormality in the appearance of the cam portion, and determining that the cam portion is suspected of having an abnormality when the output result indicates that the cam portion has an abnormality in appearance; a second inspection step of inputting imaging data of the cam portion determined to be suspected of having an abnormality in the first inspection step into a second learning model different from the first learning model, the second learning model having an output result indicating a type of abnormality in the appearance of the cam portion, and determining whether or not there is an abnormality in the cam portion according to the type of abnormality in the appearance of the cam portion indicated by the output result; a second acquisition step of acquiring image data of a journal portion provided on the same part as the cam portion; a third inspection step of inspecting the presence or absence of an abnormality in the journal portion by inputting imaging data of the journal portion into a third learning model different from the first and second learning models and outputting the presence or absence of an abnormality in the journal portion as an output result; The journal portion is inspected only by the third inspection step. A program that causes a computer to execute an anomaly inspection method.

Citation Information

Patent Citations

  • Inspection device, inspection system, and method of manufacturing goods

    JP2018146442A

  • Identification device, identification method, and program

    JP2019106090A

  • Component discrimination device and component discrimination method

    JP2020149578A

  • Color filter inspection device, inspection device, color filter inspection method, and inspection method

    WO2020175666A1

  • System and method for ai visual inspection

    WO2021062536A1