Inspection apparatus and inspection method

Abstract

To improve inspection precision of appearance of a mold.SOLUTION: An inspection device (1) includes one or multiple processors for executing inspection steps for inspecting appearance of a mold (9) by using a learned model (LM) constructed by machine learning. Input of the learned model (LM) includes an inspection image obtained by imaging appearance of the mold (9). Output of the learned model (LM) is information showing an inspection result of appearance of the mold (9).SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a technique for inspecting the appearance of a mold.

Background Art

[0002] Patent Document 1 describes an inspection apparatus for inspecting the appearance of a mold. This inspection apparatus generates a difference image between an inspection image obtained by imaging the mold and a reference image provided in advance, and determines whether the mold is normal based on the generated difference image.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The above-described inspection apparatus may determine that a mold that is actually normal is abnormal, or determine that a mold that is actually abnormal is normal, and there is room for improving the inspection accuracy.

[0005] For example, the above-described inspection apparatus determines whether the mold is normal after removing pseudo-defects among the defects obtained by analyzing the difference image. Pseudo-defects include, for example, those generated when the light irradiated during imaging is reflected by the mold release agent attached to the mold. The above-described inspection apparatus removes pseudo-defects by defining the characteristics of such pseudo-defects in advance. However, it is difficult to define all the characteristics of pseudo-defects in advance. Therefore, the above-described inspection apparatus cannot remove pseudo-defects showing characteristics that have not been defined in advance. In addition, it is difficult to define characteristics that are not shown by the original defects as characteristics of pseudo-defects. Therefore, when the original defect shows the characteristics of the pseudo-defects defined in advance, the above-described inspection apparatus removes the original defect.

[0006] One aspect of the present invention aims to realize a technique for improving the inspection accuracy of the appearance of a mold.

Means for Solving the Problems

[0007] An inspection apparatus according to one aspect of the present invention includes one or more processors that execute an inspection step and a warning step, and an output device disposed so as to be viewable from a core setting area where a core is set in the mold. In the inspection step, the one or more processors inspect the appearance of the mold using a learned model constructed by machine learning. In the warning step, when the output of the learned model indicates that there is a defect in the appearance of the mold, the one or more processors output warning information to the output device based on the information indicating the inspection result.

[0008] Also, an inspection method according to one aspect of the present invention is an inspection method including execution of an inspection step and a warning step by one or more processors. In the inspection step, the one or more processors inspect the appearance of the mold using a learned model constructed by machine learning. In the warning step, when the output of the learned model indicates that there is a defect in the appearance of the mold, the one or more processors output warning information to an output device disposed so as to be viewable from a core setting area where a core is set in the mold, based on the information indicating the inspection result.

[0009] In the inspection apparatus and the inspection method, the input of the learned model includes an inspection image obtained by imaging the appearance of the mold. The output of the learned model is information indicating the inspection result of the appearance of the mold.

Advantages of the Invention

[0010] According to one aspect of the present invention, it is possible to realize a technique for improving the inspection accuracy of the appearance of a mold.

Brief Description of the Drawings

[0011]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Embodiments for Carrying Out the Invention

[0012] 〔Inspection System S〕 The inspection system S according to an embodiment of the present invention will be described with reference to FIG. 1. FIG. 1 is a diagram showing the configuration of the inspection system S.

[0013] The inspection system S is a system for inspecting the appearance of the mold 9. In the present embodiment, the inspection system S particularly inspects the appearance of the product surface of the mold 9. If there are defects such as mold release failure on the product surface of the mold 9, the quality of the casting cast using the mold 9 will deteriorate. By inspecting the product surface of the mold 9 with the inspection system S, it is possible to determine whether the subsequent process can be carried out, and as a result, the occurrence of defective products can be suppressed. Here, the product surface of the mold 9 refers to the surface of the mold 9 whose shape is transferred to the product.

[0014] As shown in FIG. 1, the inspection system S includes an inspection device 1, a machine learning device 2, a camera 3, and a lighting 4. Further, the inspection device 1 is communicably connected to each of a sand property measuring device 92, a molding machine 93, and a sensor group 98. Also, the machine learning device 2 is communicably connected to each of the sand property measuring device 92, the molding machine 93, and the sensor group 98.

[0015] The camera 3 captures an image of the product surface of the mold 9 to generate an inspection image. The product surface of the mold 9 is not exposed to the outside after the upper mold corresponding to the upper part of the mold 9 and the lower mold corresponding to the lower part of the mold 9 are mated. Therefore, the camera 3 cannot capture an image of the product surface of the mold 9 after the upper mold and the lower mold are mated. Therefore, the camera 3 is installed at a position where it can capture an image of the product surface of the mold 9 before the upper mold and the lower mold are mated. Also, when a core is installed in the mold 9, after the core is installed, the camera 3 cannot photograph the portion of the product surface of the mold 9 hidden by the core. In this case, the camera 3 is installed at a position where it can capture an image of the product surface of the mold 9 before the core is installed. For example, the camera 3 is installed at a position where it can capture an image of the product surface of the mold 9 near the conveyance path from the molding machine 93 to a core setting area 95 described later.

[0016] The lighting 4 irradiates light onto the product surface of the mold 9 when the camera 3 captures an image. Note that the configurations of the camera 3 and the lighting 4 may adopt the configurations described in Patent Document 1 mentioned above.

[0017] The inspection device 1 is a device for implementing the inspection method M1. The inspection method M1 is a method for inspecting the product surface of the mold 9 using a learned model LM constructed by machine learning based on at least the inspection image acquired from the camera 3. As the learned model LM, for example, a neural network model such as a convolutional neural network or a recurrent neural network, or an algorithm such as a support vector machine can be used. Details of the configuration of the inspection device 1 and the flow of the inspection method M1 will be described later with reference to the drawings.

[0018] The machine learning device 2 is a device for implementing the machine learning method M2. The machine learning method M2 is a method for constructing a learning dataset DS using at least the inspection image acquired from the camera 3 and constructing a learned model LM by machine learning (supervised learning) using the learning dataset DS. Details of the configuration of the machine learning device 2 and the flow of the machine learning method M2 will be described later with reference to the drawings.

[0019] 〔Input of the learned model LM〕 The input of the learned model LM includes the inspection image. In addition, the input of the learned model LM further includes some or all of the sand information, molding information, conveyance information, and environmental information. These input data will be described.

[0020] (Inspection image) The inspection image is an image generated by the camera 3 capturing the product surface of the mold 9. The inspection image includes the product surface of the mold 9 as the subject.

[0021] (Sand information) The sand information is information indicating the foundry sand constituting the mold 9. The sand information includes some or all of the information indicating the type and properties of the foundry sand constituting the mold 9. Examples of the information indicating the properties of the foundry sand include the moisture content, compactability value, sand temperature, air permeability, and compressive strength contained in the foundry sand. The sand information is output from the sand property measuring device 92.

[0022] (Molding information) The molding information is information indicating the molding status of the mold 9. Examples of the molding status include the status of the mold 9 during molding and the status of the molding machine 93. An example of information indicating the status of the mold 9 is, for example, mold release agent information. The mold release agent information is information regarding the mold release agent applied to the mold 9. The mold release agent information includes, for example, information regarding each of the type, application amount, and application frequency of the mold release agent. Information indicating the status of the molding machine 93 includes device setting information. The setting information includes, for example, information indicating each of the squeeze pressure, sand filling time, pressure during sand filling, sand input weight, and the usage status of the molding machine. Examples of the usage status of the molding machine include the number of molding times and the wear status of parts. An example of a part that may wear is a seal part of a carrier plate. The molding information is output from the molding machine 93. Note that the mold release agent may be applied to the mold 9 by one or both of an operator and the molding machine 93. In this case, the mold release agent information may be input by the operator.

[0023] (Transportation information) The transportation information is information indicating the transportation status of the mold 9. In the casting line C described later, the mold 9 is transported downstream from the molding machine 93. The downstream side is the side of the pouring machine 97 described later as seen from the molding machine 93. The transportation information includes, for example, information indicating an external force applied to the mold 9 during this transportation. The transportation information is output from any one of the sensor group 98.

[0024] Note that an external force applied to the mold 9 may also be applied to the mold 9 other than during transportation. In this case, the input of the learned model LM may include information indicating an external force applied to the mold 9 other than during transportation. Examples of such processes other than during transportation include sand cutting, gas hole drilling, sprue forming, frame inversion, and fixed table cart setting. In this case, the sensor group 98 includes sensors that measure the external force in some or all of these processes.

[0025] (Environmental information) The environmental information is information indicating the environment around the molding machine 93 or the inspection device 1. When the environmental information indicates the environment around the inspection device 1, the inspection device 1 is installed around the casting line C described later. The environmental information includes information indicating, for example, part or all of the temperature, humidity, and concentration of floating dust. The environmental information is output from any one of the sensor group 98.

[0026] 〔Output of the learned model LM〕 The output of the learned model LM is information indicating the inspection result regarding the product surface of the mold 9. The information indicating this inspection result includes either or both of the information indicating the presence or absence of defects and the information indicating the regions of one or more defects. In the present embodiment, the information indicating the inspection result will be described as including both of these pieces of information. These pieces of information will be described.

[0027] (Information indicating the presence or absence of defects) The information indicating the presence or absence of defects indicates, for example, the presence or absence of mold release failure. The information indicating the presence or absence of defects is, for example, binary information indicating presence or absence.

[0028] (Information indicating the regions of defects) The information indicating the regions of defects indicates, for example, the locations on the product surface of the mold 9 where mold release failure has occurred. Note that when a defect has occurred at one location on the product surface of the mold 9, the learned model LM outputs information indicating one region of the defect, and when defects have occurred at multiple locations, the learned model LM outputs information indicating the regions of each of the multiple defects. The information indicating the region of each defect includes information indicating the position, size, and shape of that defect.

[0029] (Information indicating the position of a defect) The information indicating the position of a defect is represented, for example, by the position of the image of the defect in the inspection image. As a specific example, the information indicating the position of the defect may be the coordinates of any vertex of a rectangle that encloses (e.g., circumscribes) the image of the defect in the inspection image.

[0030] (Information indicating the size of a defect) The information indicating the size of the defect is represented, for example, by the size of the defect image in the inspection image. As a specific example, the information indicating the size of the defect may be the area of the rectangle described above.

[0031] (Information indicating the shape of the defect) The information indicating the shape of the defect is represented, for example, by the shape of the defect image in the inspection image. As a specific example, the information indicating the shape of the defect may be the aspect ratio of the rectangle described above.

[0032] Note that the information indicating the defect area may be any information that allows the user to recognize the position, size, and shape of the defect, and does not necessarily have to be expressed as a numerical value. For example, the information indicating the defect area may be expressed by superimposing a rectangle encompassing the defect image on the inspection image.

[0033] [Phase from the introduction to the practical use of the inspection system S] The inspection system S goes through a preparation phase, a trial phase, and then reaches the practical use phase. Briefly explaining the content of the preparation phase, the trial phase, and the practical use phase, it is as follows.

[0034] (1) Preparation phase In the preparation phase, the operator inspects the product surface of the mold 9, and the machine learning device 2 generates a learning dataset DS and constructs a learned model LM.

[0035] Specifically, the camera 3 generates an inspection image by imaging the product surface of the mold 9 and inputs it into the machine learning device 2. The sand property measuring device 92 inputs sand information related to the mold 9 into the machine learning device 2. The molding machine 93 inputs molding information related to the mold 9 into the machine learning device 2. The sensor group 98 inputs conveyance information and environment information into the machine learning device 2. Further, the operator inspects the product surface of the mold 9 and inputs information indicating the inspection result by the operator into the machine learning device 2 in association with the inspection image. The information indicating the inspection result by the operator includes information indicating the presence or absence of defects. For example, when the operator determines that there is one or more defects on the product surface of the mold 9, the operator inputs information indicating that there are defects into the machine learning device 2. Further, the information indicating the inspection result by the operator includes, in addition to the information indicating the presence or absence of defects, information indicating the area of one or more defects. An example of a method for inputting information indicating the area of one or more defects will be described. For example, the operator inputs information indicating the position, size, and shape of each defect into the machine learning device 2. As a specific example of the operation for inputting this information, an operation of drawing a rectangle on the inspection image can be cited. In this case, the operator draws a rectangle including the image of each defect on the inspection image displayed on a display (not shown) or the like using an input device. The machine learning device 2 acquires information indicating the position, size, and shape of each defect from the rectangle drawn by the operator. In this way, the machine learning device 2 acquires, as information indicating the inspection result by the operator, the information indicating the presence or absence of defects input by the operator and the information indicating the position, size, and shape of each defect input by the operator. Each time the inspection by the operator is performed, the machine learning device 2 creates teacher data Di having (i) the inspection image acquired from the camera 3 and (ii) the sand information, molding information, conveyance information, and environment information input from each device as input data, and (iii) the information indicating the inspection result by the operator as correct data, and adds the created teacher data Di to the learning data set DS. In the preparation phase, the sand information, molding information, conveyance information, and environment information have been described as being input from each device into the machine learning device 2. However, some or all of this information may be input into the machine learning device 2 by the user.In this case, the operator who conducts the inspection and the user who inputs the data may be the same or different. The preparation phase may end when a predetermined period (for example, one week, one month, or one year, etc.) has elapsed since the start of the preparation phase, or may end when the number of inspections by the operator in the preparation phase reaches a predetermined number (for example, 100 times, 1000 times, or 10000 times, etc.).

[0036] During the preparation phase, the operator inspects the product surfaces of the molds 9 respectively shaped using a plurality of different types of models. In other words, the learning dataset DS includes teacher data Di corresponding to each of the plurality of different types of models.

[0037] When the preparation phase ends, the machine learning device 2 constructs a learned model LM by machine learning using the learning dataset DS. Since the learning dataset DS includes teacher data Di corresponding to each of the plurality of different types of models, the constructed learned model LM functions in common for the plurality of different types of models. The constructed learned model LM is transferred from the machine learning device 2 to the inspection device 1.

[0038] (2) Trial phase In the trial phase, the operator inspects the product surface of the mold 9, and the inspection device 1 also inspects the product surface of the mold 9.

[0039] Specifically, the camera 3 generates an inspection image of the product surface of the mold 9 and inputs it to the inspection device 1. The sand property measuring device 92 inputs sand information related to the mold 9 to the inspection device 1. The molding machine 93 inputs molding information related to the mold 9 to the inspection device 1. The sensor group 98 inputs conveyance information and environmental information to the inspection device 1. The inspection device 1 inputs the inspection image obtained from the camera 3, the sand information, molding information, conveyance information, and environmental information input from each device to the learned model LM, and outputs information indicating the inspection result obtained from the learned model LM.

[0040] Also, the operator inspects the product surface of the mold 9. The operator compares the inspection result by the operator with the inspection result indicated by the output of the inspection device 1 to evaluate the inspection accuracy of the inspection device 1. For example, in the trial phase, 200 molds 9 are inspected. If the inspection results by the inspection device 1 and the inspection results by the operator match for 192 molds 9, the inspection accuracy is evaluated to be 96%. In the trial phase, sand information, molding information, conveyance information, and environmental information have been described as being input from each device to the inspection device 1. However, some or all of this information may be input to the inspection device 1 by the user. In this case, the operator who performs the inspection and the user who inputs the data may be the same or different. The trial phase may end when a predetermined period (for example, one week, one month, or one year, etc.) has elapsed since the start of the trial phase, or may end when the number of inspections of the mold 9 in the trial phase reaches a predetermined number (for example, 100 times, 1000 times, or 10000 times, etc.). If the inspection accuracy evaluated in the trial phase is insufficient, the above-described preparation phase and trial phase are carried out again. In this case, in the preparation phase carried out again, additional learning for the learned model LM may be performed. If the inspection accuracy evaluated in the trial phase is sufficient, the practical phase described later is carried out.

[0041] Here, there are two cases for evaluating the inspection accuracy and switching phases: (i) the case where the inspection device 1 performs the evaluation, or (ii) the case where the operator performs the evaluation. In case (i), the inspection device 1 evaluates its own inspection accuracy by comparing the inspection results input by the operator into the inspection device 1 with its own inspection results. Then, for example, if the inspection accuracy is equal to or lower than a predetermined threshold value, the inspection device 1 determines that it is necessary to perform the preparation phase again. And the inspection device 1 performs a phase switch according to the result of that determination. On the other hand, in case (ii), the operator evaluates the inspection accuracy of the inspection device 1 by comparing the inspection results output by the inspection device 1 with his / her own inspection results. Then, for example, if the inspection accuracy is equal to or lower than a predetermined threshold value, the operator determines that it is necessary to perform the preparation phase again. And the inspection device 1 performs a phase switch according to the result of that determination.

[0042] (3) Practical phase In the practical phase, the inspection device 1 inspects the product surface of the mold 9.

[0043] Specifically, the camera 3 generates an inspection image by imaging the product surface of the mold 9 and inputs it into the inspection device 1. The sand property measuring device 92 inputs sand information related to the mold 9 into the inspection device 1. The molding machine 93 inputs molding information related to the mold 9 into the inspection device 1. The sensor group 98 inputs conveyance information and environmental information into the inspection device 1. The inspection device 1 inputs the inspection image obtained from the camera 3, the sand information, molding information, conveyance information, and environmental information input from each device into the learned model LM, and outputs information indicating the inspection results obtained from the learned model LM. The learned model LM used by the inspection device 1 in the practical phase has been confirmed to have sufficient inspection accuracy in the trial phase. In the practical phase, inspection by the operator can be omitted. Therefore, the operator can be freed from the labor of inspection, and the casting line C can be operated efficiently.

[0044] In addition, in the present embodiment, a configuration is adopted in which an inspection image and some or all of sand information, molding information, conveyance information, and environment information are input to the learned model LM. However, the present invention is not limited to this. Among these data, the inspection image has a dominant influence on the inspection result of the product surface of the mold 9. Other sand information, molding information, conveyance information, and environment information are data for improving the inspection accuracy of the product surface of the mold 9 and do not necessarily have to be adopted as the input to the learned model LM. That is, any configuration may be adopted as long as at least the inspection image is input to the learned model LM.

[0045] Further, in the present embodiment, a configuration is adopted in which one or both of information indicating the presence or absence of a defect and information indicating the region of one or more defects are output from the learned model LM. However, the present invention is not limited to this. For example, the output of the learned model LM may include information indicating the degree of a defect. The degree of a defect is, for example, the degree to which the defect affects the quality of the product. The degree of such an influence may vary depending on, for example, the location where the defect occurs. Note that the degree of a defect is not limited to the above-described example.

[0046] 〔Configuration of Inspection Apparatus〕 The configuration of the inspection apparatus 1 will be described with reference to FIG. 2. FIG. 2 is a block diagram showing the configuration of the inspection apparatus 1.

[0047] The inspection apparatus 1 is realized using a general-purpose computer and includes a processor 11, a primary memory 12, a secondary memory 13, an input / output interface 14, a communication interface 15, and a bus 16. The processor 11, the primary memory 12, the secondary memory 13, the input / output interface 14, and the communication interface 15 are interconnected via the bus 16.

[0048] The secondary memory 13 stores the test program P1 and the learned model LM. The processor 11 expands the test program P1 and the learned model LM stored in the secondary memory 13 onto the primary memory 12. Then, the processor 11 executes each step included in the test method M1 according to the instructions included in the test program P1 expanded onto the primary memory 12. The learned model LM expanded onto the primary memory 12 is used when the processor 11 executes the test step M12 (described later) of the test method M1. Note that the fact that the test program P1 is stored in the secondary memory 13 means that the source code or the executable file obtained by compiling the source code is stored in the secondary memory 13. Also, the fact that the learned model LM is stored in the secondary memory 13 means that the parameters defining the learned model LM are stored in the secondary memory 13.

[0049] Examples of devices that can be used as the processor 11 include a CPU (Central Processing Unit), a GPU (Graphic Processing Unit), a DSP (Digital Signal Processor), an MPU (Micro Processing Unit), an FPU (Floating point number Processing Unit), a PPU (Physics Processing Unit), a microcontroller, or a combination thereof. The processor 11 may also be referred to as an "arithmetic unit".

[0050] Examples of devices that can be used as the primary memory 12 include, for example, semiconductor RAM (Random Access Memory). The primary memory 12 is sometimes referred to as the "main memory device". Examples of devices that can be used as the secondary memory 13 include, for example, flash memory, HDD (Hard Disk Drive), SSD (Solid State Drive), ODD (Optical Disk Drive), FDD (Floppy (registered trademark) Disk Drive), or combinations thereof. The secondary memory 13 is sometimes referred to as the "auxiliary storage device". Note that the secondary memory 13 may be built into the inspection device 1, or may be built into another computer (for example, a computer that constitutes a cloud server) connected to the inspection device 1 via the input / output interface 14 or the communication interface 15. In this embodiment, the storage in the inspection device 1 is realized by two memories (the primary memory 12 and the secondary memory 13), but it is not limited to this. That is, the storage in the inspection device 1 may be realized by one memory. In this case, for example, a certain storage area of the memory may be used as the primary memory 12, and another storage area of the memory may be used as the secondary memory 13.

[0051] An input / output interface 14 is connected to an input device and / or an output device. Examples of the input / output interface 14 include interfaces such as USB (Universal Serial Bus), ATA (Advanced Technology Attachment), SCSI (Small Computer System Interface), and PCI (Peripheral Component Interconnect). Examples of the input device connected to the input / output interface 14 include a keyboard, a mouse, a touch pad, a microphone, or a combination thereof. Data acquired from the user in the inspection method M1 is input into the inspection apparatus 1 via these input devices and stored in the primary memory 12. Examples of the output device connected to the input / output interface 14 include a display, a projector, a printer, a speaker, headphones, or a combination thereof. Information provided to the user in the inspection method M1 is output from the inspection apparatus 1 via these output devices. Note that the inspection apparatus 1 may incorporate a keyboard that functions as an input device and a display that functions as an output device, like a laptop computer. Alternatively, the inspection apparatus 1 may incorporate a touch panel that functions as both an input device and an output device, like a tablet computer.

[0052] The communication interface 15 is connected, either wired or wirelessly, to other computers via a network. Examples of the communication interface 15 include interfaces such as Ethernet (registered trademark) and Wi-Fi (registered trademark). Examples of available networks include a PAN (Personal Area Network), a LAN (Local Area Network), a CAN (Campus Area Network), a MAN (Metropolitan Area Network), a WAN (Wide Area Network), a GAN (Global Area Network), or an internetwork including these networks. The internetwork may be an intranet, an extranet, or the Internet. Data (e.g., a learned model LM) acquired by the inspection device 1 from other computers (e.g., the machine learning device 2) in the inspection method M1, and data provided by the inspection device 1 to other computers in the inspection method M1 are transmitted and received via these networks.

[0053] For example, a camera 3, a sand property measuring device 92, a molding machine 93, and each of the sensor groups 98 are connected to the communication interface 15. Data acquired from each of these devices in the inspection method M1 is input to the inspection device 1 via the communication interface 15 and stored in the primary memory 12. Note that some or all of these devices are not limited to being connected to the communication interface 15 and may be connected to the input / output interface 14.

[0054] In the present embodiment, a configuration is adopted in which the inspection method M1 is executed using a single processor (processor 11), but the present invention is not limited to this. That is, a configuration may be adopted in which the inspection method M1 is executed using a plurality of processors. In this case, the plurality of processors that execute the inspection method M1 in cooperation may be provided in a single computer and configured to be communicable with each other via a bus, or may be provided distributedly in a plurality of computers and configured to be communicable with each other via a network. As an example, a mode in which a processor incorporated in a computer constituting a cloud server and a processor incorporated in a computer owned by a user of the cloud server execute the inspection method M1 in cooperation can be considered.

[0055] Also, in the present embodiment, a configuration is adopted in which the learned model LM is stored in a memory (secondary memory 13) incorporated in the same computer as the processor (processor 11) that executes the inspection method M1, but the present invention is not limited to this. That is, a configuration may be adopted in which the learned model LM is stored in a memory incorporated in a computer different from the processor that executes the inspection method M1. In this case, the computer in which the memory storing the learned model LM is incorporated is configured to be communicable with the computer in which the processor that executes the inspection method M1 is incorporated via a network. As an example, a mode in which the learned model LM is stored in a memory incorporated in a computer constituting a cloud server and a processor incorporated in a computer owned by a user of the cloud server executes the inspection method M1 can be considered.

[0056] Also, in this embodiment, a configuration is adopted in which the learned model LM is stored in a single memory (secondary memory 13), but the present invention is not limited to this. That is, a configuration may be adopted in which the learned model LM is distributed and stored in a plurality of memories. In this case, the plurality of memories for storing the learned model LM may be provided in a single computer (which may or may not be a computer with a built-in processor for executing the inspection method M1), or may be distributed and provided in a plurality of computers (which may or may not include a computer with a built-in processor for executing the inspection method M1). As an example, a configuration in which the learned model LM is distributed and stored in the memories built into each of the plurality of computers constituting a cloud server can be considered.

[0057] 〔Flow of the inspection method〕 The flow of the inspection method M1 will be described with reference to FIG. 3. FIG. 3 is a flowchart showing the flow of the inspection method M1.

[0058] The inspection method M1 includes an acquisition step M11, an inspection step M12, a determination step M13, and a warning step M14.

[0059] The acquisition step M11 is a step in which the processor 11 acquires the data to be input to the learned model LM. In the acquisition step M11, the processor 11 acquires an inspection image from the camera 3 and stores it in the primary memory 12. Also, in the acquisition step M11, the processor 11 acquires sand information, molding information, conveyance information, and environment information from the sand property measuring device 92, the molding machine 93, and the sensor group 98, and stores them in the primary memory 12.

[0060] Inspection step M12 is a step in which the processor 11 inspects the product surface of the mold 9 using the learned model LM. In inspection step M12, the processor 11 reads the inspection image, sand information, molding information, conveyance information, and environmental information from the primary memory 12 and inputs them into the learned model LM. Then, the processor 11 writes the information indicating the inspection result output from the learned model LM into the primary memory 12. The information indicating the inspection result includes information indicating the presence or absence of defects and information indicating the regions of one or more defects.

[0061] Judgment step M13 is a step in which the processor 11 determines whether there are defects in the mold 9 based on the output of the learned model LM. If it is determined that there are defects in this step, warning step M14 is executed. If it is determined that there are no defects, inspection method M1 ends.

[0062] Warning step M14 is a step in which the processor 11 outputs warning information based on the output of the learned model LM. In warning step M14, the processor 11 generates warning information based on the information indicating the inspection result output from the learned model LM.

[0063] The warning information is, for example, an image in which a figure indicating the boundary line of the defective region is superimposed on the inspection image. Also, the warning information includes, for example, the occurrence frequency (number of occurrences, occurrence rate, etc.) of defects in a plurality of molds molded using a mold of the same type as the mold 9.

[0064] Also, the warning information includes, for example, the occurrence frequency (number of occurrences, occurrence rate, etc.) of defects for each location in a plurality of molds molded using a mold of the same type as the mold 9. The locations where defects occur are managed, for example, in units of blocks obtained by dividing the inspection image.

[0065] The processor 11 outputs the generated warning information using an output device. For example, if the output device includes a display, the processor 11 generates an image showing the warning information and displays the image on the display.

[0066] When warning information is output, the operator can skip the subsequent processes in the casting line C for the corresponding mold 9.

[0067] 〔Configuration of Machine Learning Device〕 The configuration of the machine learning device 2 will be described with reference to FIG. 4. FIG. 4 is a block diagram showing the configuration of the machine learning device 2.

[0068] The machine learning device 2 is realized using a general-purpose computer and includes a processor 21, a primary memory 22, a secondary memory 23, an input / output interface 24, a communication interface 25, and a bus 26. The processor 21, the primary memory 22, the secondary memory 23, the input / output interface 24, and the communication interface 25 are interconnected via the bus 26.

[0069] The secondary memory 23 stores a machine learning program P2 and a learning dataset DS. The learning dataset DS is a set of teacher data D1, D2,.... The processor 21 expands the machine learning program P2 stored in the secondary memory 23 onto the primary memory 22. Then, the processor 21 executes each step included in the machine learning method M2 according to the instructions included in the machine learning program P2 expanded on the primary memory 22. The learning dataset DS stored in the secondary memory 23 is constructed in the learning dataset construction step M21 (described later) of the machine learning method M2 and is used in the learned model construction step M22 (described later) of the machine learning method M2. Also, the learned model LM constructed in the learned model construction step M22 of the machine learning method M2 is also stored in the secondary memory 23. Note that the fact that the machine learning program P2 is stored in the secondary memory 23 means that the source code or the executable file obtained by compiling the source code is stored in the secondary memory 23. Also, the fact that the learned model LM is stored in the secondary memory 23 means that the parameters defining the learned model LM are stored in the secondary memory 23.

[0070] Examples of devices that can be used as the processor 21 include a central processing unit (CPU), a graphic processing unit (GPU), a digital signal processor (DSP), a micro processing unit (MPU), a floating point number processing unit (FPU), a physics processing unit (PPU), a microcontroller, or a combination thereof. The processor 21 is also sometimes called an "arithmetic unit."

[0071] Furthermore, an example of a device that can be used as the primary memory 22 is a semiconductor random access memory (RAM). The primary memory 22 is sometimes referred to as a "main storage device." Furthermore, an example of a device that can be used as the secondary memory 23 is a flash memory, a hard disk drive (HDD), a solid state drive (SSD), an optical disk drive (ODD), a floppy disk drive (FDD), or a combination thereof. The secondary memory 23 is sometimes referred to as an "auxiliary storage device." The secondary memory 23 may be built into the machine learning device 2, or may be built into another computer (e.g., a computer constituting a cloud server) connected to the machine learning device 2 via the input / output interface 24 or the communication interface 25. While the present embodiment implements storage in the machine learning device 2 using two memories (the primary memory 22 and the secondary memory 23), this is not limiting. That is, storage in the machine learning device 2 may be implemented using a single memory. In this case, for example, one storage area of the memory may be used as the primary memory 22, and another storage area of the memory may be used as the secondary memory 23.

[0072] An input / output interface 24 has an input device and / or an output device connected thereto. Examples of the input / output interface 24 include interfaces such as USB (Universal Serial Bus), ATA (Advanced Technology Attachment), SCSI (Small Computer System Interface), and PCI (Peripheral Component Interconnect). Examples of the input device connected to the input / output interface 24 include a keyboard, a mouse, a touch pad, a microphone, or a combination thereof. Data acquired from the user in the machine learning method M2 is input into the machine learning device 2 via these input devices and stored in the primary memory 22. Examples of the output device connected to the input / output interface 24 include a display, a projector, a printer, a speaker, headphones, or a combination thereof. Information provided to the user in the machine learning method M2 is output from the machine learning device 2 via these output devices. Note that the machine learning device 2 may incorporate, like a laptop computer, a keyboard functioning as an input device and a display functioning as an output device, respectively. Alternatively, the machine learning device 2 may incorporate, like a tablet computer, a touch panel functioning as both an input device and an output device.

[0073] Other computers are connected to the communication interface 25 via a network, either wired or wirelessly. Examples of the communication interface 25 include interfaces such as Ethernet (registered trademark) and Wi-Fi (registered trademark). Available networks include a personal area network (PAN), a local area network (LAN), a campus area network (CAN), a metropolitan area network (MAN), a wide area network (WAN), a global area network (GAN), or an internetwork including these networks. The internetwork may be an intranet, an extranet, or the Internet. Data (e.g., the trained model LM) provided by the machine learning device 2 to other computers (e.g., the inspection device 1) is transmitted and received via these networks.

[0074] For example, the camera 3, the sand property measuring device 92, the molding machine 93, and the sensor group 98 are each connected to the communication interface 25. Data acquired from each of these devices in the machine learning method M2 is input to the machine learning device 2 via the communication interface 25 and stored in the primary memory 22. Note that some or all of these devices are not limited to those connected to the communication interface 25, and may also be connected to the input / output interface 24.

[0075] In the present embodiment, a configuration is adopted in which the machine learning method M2 is executed using a single processor (processor 21), but the present invention is not limited to this. That is, a configuration may be adopted in which the machine learning method M2 is executed using a plurality of processors. In this case, the plurality of processors that execute the machine learning method M2 in cooperation may be provided in a single computer and configured to be communicable with each other via a bus, or may be provided distributedly in a plurality of computers and configured to be communicable with each other via a network. As an example, a mode in which a processor built into a computer constituting a cloud server and a processor built into a computer owned by a user of the cloud server execute the machine learning method M2 in cooperation can be considered.

[0076] Also, in the present embodiment, a configuration is adopted in which the learning data set DS is stored in a memory (secondary memory 23) built into the same computer as the processor (processor 21) that executes the machine learning method M2, but the present invention is not limited to this. That is, a configuration may be adopted in which the learning data set DS is stored in a memory built into a computer different from the processor that executes the machine learning method M2. In this case, the computer in which the memory storing the learning data set DS is built is configured to be communicable with the computer in which the processor that executes the machine learning method M2 is built via a network. As an example, a mode in which the learning data set DS is stored in a memory built into a computer constituting a cloud server and a processor built into a computer owned by a user of the cloud server executes the machine learning method M2 can be considered.

[0077] Furthermore, although this embodiment employs a configuration in which the training dataset DS is stored in a single memory (secondary memory 23), the present invention is not limited to this. That is, a configuration in which the training dataset DS is distributed and stored in multiple memories may be employed. In this case, the multiple memories that store the training dataset DS may be provided in a single computer (which may or may not be a computer incorporating a processor that executes the machine learning method M2), or may be distributed and stored in multiple computers (which may or may not include a computer incorporating a processor that executes the machine learning method M2). As an example, a configuration in which the training dataset DS is distributed and stored in memories incorporated in each of multiple computers that constitute a cloud server may be considered.

[0078] Furthermore, in this embodiment, a configuration is adopted in which inspection method M1 and machine learning method M2 are executed using different processors (processor 11 and processor 21), but the present invention is not limited to this. That is, inspection method M1 and machine learning method M2 may be executed using the same processor. In this case, by executing machine learning method M2, the trained model LM is stored in memory built into the same computer as this processor. Then, this processor uses the trained model LM stored in this memory when executing inspection method M1.

[0079] [Machine learning method flow] The flow of the machine learning method M2 will be described with reference to Fig. 5. Fig. 5 is a flowchart showing the flow of the machine learning method M2.

[0080] The machine learning method M2 includes a learning dataset construction step M21 and a trained model construction step M22.

[0081] The learning data set construction step M21 is a step in which the processor 21 constructs a learning data set DS, which is a collection of training data D1, D2, . . .

[0082] Each training data Di (i = 1, 2, ...) includes input data including an inspection image, sand information, molding information, transportation information, and environmental information, as well as supervised data including information indicating the presence or absence of defects and information indicating the area of one or more defects. Specifically, in the learning dataset construction step M21, the processor 21 acquires an inspection image from the camera 3. The processor 21 also acquires sand information, molding information, transportation information, and environmental information from the sand property measuring device 92, the molding machine 93, and the sensor group 98. This information relates to the mold 9 included as a subject in the inspection image. The processor 21 also acquires information indicating the presence or absence of defects and information indicating the area of one or more defects input by the user as information indicating the worker's inspection results for the mold 9 to be inspected. The processor 21 stores the training data Di, which includes the input data acquired from each device (inspection image, sand information, molding information, transportation information, and environmental information) and the supervised data input by the user (information indicating the worker's inspection results), in the secondary memory 23. The processor 21 repeats the above process to construct a training dataset DS.

[0083] The trained model construction step M22 is a step in which the processor 21 constructs the trained model LM. In the trained model construction step M22, the processor 21 constructs the trained model LM by supervised learning using the training dataset DS. Then, the processor 21 stores the constructed trained model LM in the secondary memory 23.

[0084] [Configuration of Casting Line C] The configuration of a portion of a casting line C to which the inspection system S is applied will be described with reference to FIG. 6. FIG. 6 is a block diagram showing a portion of the schematic configuration of the casting line C. The casting line C is an example of a portion of a system that transports a plurality of molds 9 along a transport path and produces castings by pouring molten metal into each mold 9. The molds 9 used in the casting line C are green sand molds, and the molding method is flask molding. However, the molding method in the casting line C may also be flaskless molding. Furthermore, the casting line to which the inspection system S is applied may also be a line that uses self-hardening molds.

[0085] 6, the casting line C includes a kneading machine 91, a sand property measuring device 92, a molding machine 93, a conveying device 94, a core setting station 95, a mold matching device 96, a pouring machine 97, and sensors 98a and 98b. The sensors 98a and 98b are examples of sensors included in the sensor group 98 shown in FIG.

[0086] The kneader 91 is a device for kneading foundry sand.

[0087] The sand property measuring device 92 is a device for measuring the properties of the foundry sand mixed by the mixer 91.

[0088] The molding machine 93 is a device that uses a model to manufacture the casting mold 9. The molding machine 93 forms the upper and lower molds of the casting mold 9.

[0089] The transfer device 94 is a device that transfers the mold 9 along the transfer path from the molding machine 93 to the downstream side (towards the pouring machine 97). The transfer path is constituted by, for example, a roller conveyor (not shown) or a rail (not shown) laid from the molding machine 93 to the pouring machine 97. A plurality of mold frames F are arranged at equal intervals at positions P1 to P14 on the transfer path. The transfer device 94 transfers these mold frames F one by one to the downstream side. When one frame is transferred to the downstream side, the mold frames F arranged at positions P1 to P13 are arranged at positions P2 to P14. The mold frame F arranged at position P14 is arranged at a position (not shown) one frame downstream from position P14. A new mold frame F is arranged at position P1. A camera 3 is installed near the transfer path from the molding machine 93 to the core setting area 95 (here, near position P5).

[0090] An operator stays at the core setting area 95. The core setting area 95 is a place where the operator sets cores in the mold 9 before mold alignment.

[0091] The mold alignment device 96 is a device that aligns the upper mold and the lower mold.

[0092] The pouring machine 97 is a device that pours molten metal into the mold 9.

[0093] The sensor 98a is a group of sensors that detect the environment around the molding machine 93 or the inspection device 1. The sensor 98a is constituted by, for example, a temperature sensor, a humidity sensor, and a measuring instrument that measures the concentration of floating dust. Each sensor constituting the sensor 98a is installed around the molding machine 93 or the inspection device 1.

[0094] The sensor 98b is a group of sensors that detect the external force applied to the mold 9. The sensor 98b is constituted by, for example, a plurality of acceleration sensors and is attached to each mold frame F.

[0095] 〔Casting process C100 implemented by the casting line C〕 The casting process C100 performed by the casting line C will be described with reference to FIG. 7. FIG. 7 is a flowchart illustrating each step of a part of the casting process C100. As shown in FIG. 7, the casting process C100 includes a kneading process C101, a molding process C102, conveying processes C103, C105, C107, C109, an inspection process C104, a core set process C106, a mold alignment process C108, and a pouring process C110.

[0096] In the kneading process C101, the kneader 91 kneads the casting sand. Specifically, for example, the kneader 91 adds a binder, moisture, other additives (such as starch and coal powder), or a combination thereof to the casting sand and kneads it. Further, the sand property measuring device 92 measures the properties of the kneaded casting sand and transmits the measurement results to the inspection device 1 as the above-described sand information.

[0097] In the molding process C102, the molding machine 93 fills the casting sand kneaded in the kneading process C101 into the mold frame F. Further, the molding machine 93 pressurizes and solidifies the casting sand in the mold frame F to mold the upper mold or the lower mold of the mold 9. Specifically, when a model of the upper mold is provided in the mold frame F, the molding machine 93 molds the upper mold, and when a model of the lower mold is provided, the molding machine 93 molds the lower mold. For example, the molding machine 93 molds the upper mold and the lower mold alternately. Further, when the process is completed, the molding machine 93 outputs a signal indicating that it is transportable to the transport device 94. Also, the molding machine 93 transmits the mold release agent information and the setting information of the molding machine 93 obtained during the process to the inspection device 1 as the above-described molding information. Further, the sensors among the sensors 98a installed around the molding machine 93 transmit the temperature, humidity, or concentration of floating dust around the molding machine 93 detected during the process to the inspection device 1 as the above-described environmental information. Also, the sensors among the sensors 98a installed around the inspection device 1 transmit the temperature, humidity, or concentration of floating dust around the inspection device 1 detected during the process to the inspection device 1 as the above-described environmental information.

[0098] In the conveying process C103, the conveying device 94 conveys each mold frame F downstream from the molding machine 93 side. Note that the conveying device 94 executes the process when it receives signals indicating that it can convey from the molding machine 93, the mold alignment device 96, and the pouring machine 97, respectively. Further, the sensor 98b detects an external force applied during conveyance to the upper mold or the lower mold with the mold frame F for which the molding process C102 has been completed. The sensor 98b transmits the detection result to the inspection device 1 as the above-described conveyance information. When the upper mold or the lower mold with the mold frame F is conveyed to the position P5, the next inspection process C104 is performed.

[0099] In the inspection process C104, the camera 3 images the product surface of the upper mold or the lower mold with the mold frame F at the position P5, and transmits the captured image to the inspection device 1 as an inspection image. The inspection device 1 executes the above-described inspection method M1 using the information acquired from the sand property measuring device 92, the molding machine 93, the sensors 98a and 98b, and the camera 3 in the processes C101 to C104. When the inspection device 1 determines that there is a defect in the product surface of the upper mold or the lower mold with the mold frame F in the inspection method M1, it outputs warning information to the output device.

[0100] The operation of the conveying device 94 in the conveying process C105 is the same as that in the conveying process C103. When the upper mold or the lower mold with the mold frame F for which the inspection process C104 has been completed is conveyed to the position P6, the next core subset process C106 is performed.

[0101] In the core subset process C106, when the inside of the mold frame F at the core subset area 95 (positions P6 to P8) is the lower mold, the operator sets the core in the lower mold.

[0102] If a warning is output in the inspection step C104, the worker visually checks whether the subsequent steps can be carried out on the top or bottom mold with flask F for which the warning was output. If the worker determines that the steps can be carried out, he places a core in the bottom mold with flask F. If the worker determines that the steps cannot be carried out, he registers the top or bottom mold with flask F as a defective flask. This prevents the subsequent steps from being carried out on the top or bottom mold with flask F. Details of defective flasks will be provided later.

[0103] The worker may decide that the subsequent process can be carried out after making the necessary adjustments to the top or bottom mold with flask F for which the warning information was output. In this case, the worker makes the necessary adjustments to the top or bottom mold with flask F, and then sets the core in the bottom mold if it is a bottom mold. Examples of necessary adjustments include cleaning the product surface of the top or bottom mold with flask F and applying a mold release agent to the product surface.

[0104] The operation of the transport device 94 in the transport step C107 is the same as in the transport step C103. Once the upper or lower mold with the flask F attached has completed the core setting step C106, it is transported to position P10, where the next mold matching step C108 is carried out. Note that before being transported to position P10, the upper mold with the flask F is inverted by a flask inverting device (not shown) so that the product surface of the upper mold faces downward.

[0105] In the mold matching step C108, the mold matching device 96 aligns the upper mold with the flask F and the lower mold with the flask F. This results in a mold 9 with the flask F, in which the upper mold and the lower mold are aligned. Furthermore, upon completion of this step, the mold matching device 96 outputs a signal to the transport device 94 indicating that transport is possible.

[0106] The operation of the transfer device 94 in the transfer step C109 is the same as in the transfer step C103. When the mold 9 with the molding flask F that has completed the matching step C108 is transferred to position P14, the next pouring step C110 is carried out.

[0107] In the pouring process C110, the pouring machine 97 pours the molten metal into the mold 9 with the flask F located at position P14. When this process is completed, the pouring machine 97 outputs a signal to the transfer device 94 indicating that the transfer is possible.

[0108] Thereafter, the molten metal poured into the mold 9 with the flask F is cooled to form a casting, which is then removed by dismantling the mold 9 with the flask F.

[0109] In order to maintain the quality of the castings cast on the casting line C, if a defect occurs on the product surface of the mold 9, it is desirable not to use that mold 9 for subsequent processes.

[0110] Therefore, in the casting process C100, the inspection device 1 performs an inspection process C104 in which the product surface of the mold 9 molded in the molding process C102 is inspected. If warning information is output to the output device in the inspection process C104, the worker registers the top or bottom mold with the flask F for which the warning information was output as a defective flask. This makes it possible to skip the subsequent processes (mold matching process C108 and pouring process C110) for that top or bottom mold with the flask F.

[0111] (Details of defective frames) Here, we will explain the details of defective flasks. A defective flask is a mold 9 (or a flask F in which the mold 9 is placed) that has a defect on its product surface (e.g., a mold-out defect) and is judged to be unable to produce a non-defective product. In processes prior to the mold matching process C108, the upper or lower mold with the flask F attached is registered as a defective flask. Registering a defective flask means, for example, recording information indicating the defective flask in shift data. Shift data is information that a line controller (not shown) stores in a memory area corresponding to the position (P1 to P14) of the flask F. When a flask F is transported (shifted) by one flask, the shift data recorded in the memory area is also shifted and stored in a memory area corresponding to the destination position. The shift data includes, for example, data indicating the molding history (molding information), data indicating the transport status (transport information), data indicating the state of the molten metal, data indicating the alloy material input history, etc., or identification information of the mold 9 to which these data are linked.

[0112] For example, in the core setting process C106, if the worker determines that the top or bottom mold with the flask F for which the warning information was output is a defective flask, the worker performs an operation to register the top or bottom mold with the flask F as a defective flask. For example, the registration operation may be the pressing of a judgment switch. In response to the worker's operation to register, the line controller records information indicating that the corresponding top or bottom mold with the flask F is a defective flask in the shift data.

[0113] In the processes subsequent to the core setting process C106 (such as the mold matching process C108 and pouring process C110), the devices that perform each process (such as the mold matching device 96 and pouring machine 97) refer to the shift data. By referring to the shift data, each device skips the process for the upper or lower mold with flask F registered as a defective flask, or for the mold 9 with flask F after these have been matched.

[0114] (When the core setting process C106 is performed by a core setting device) In the above description of the casting process C100, a worker sets the core. However, the core setting station 95 may be provided with a core setting device (not shown) that automatically sets the core. In this case, the core setting device sets the core in the lower mold with the flask F. In this case, if the inspection device 1 determines in the inspection process C104 that there is a defect on the product surface of the upper mold or lower mold with the flask F, instead of or in addition to outputting a warning to the output device, the inspection device 1 may register the upper mold or lower mold with the flask F as a defective flask. For example, the inspection device 1 transmits request information to the line controller requesting registration of a defective flask. In response to receiving the request information, the line controller records information indicating that the corresponding upper mold or lower mold with the flask F is a defective flask in the shift data. By referring to the shift data, the core setting device does not set a core in the lower mold with the flask F registered as a defective flask (i.e., for which warning information has been output).

[0115] [Variation 1] An embodiment of the present invention described above can be modified so that the input of the trained model further includes a reference image.

[0116] The configuration of an inspection system Sa according to this modification will be described with reference to Fig. 8. Fig. 8 is a block diagram showing the configuration of the inspection system Sa. The inspection system Sa includes an inspection device 1a, a machine learning device 2a, a camera 3, and a light 4.

[0117] (Configuration of inspection device 1a) The inspection device 1a differs from the inspection device 1 in that it uses a trained model LMa instead of the trained model LM to inspect the product surface of the mold 9. In all other respects, it is the same as the inspection device 1.

[0118] (Input and output of the trained model LMa) The input of the learned model LMa includes a reference image in addition to the inspection image, sand information, molding information, conveyance information, and environmental information similar to those of the learned model LM. The reference image is an image obtained by imaging the product surface of a normal mold 9 with the camera 3. The normal mold 9 has no defects on the product surface. The mold 9 included as a subject in the reference image is molded using the same type of mold as the mold 9 included as a subject in the inspection image. The output of the learned model LMa is the same as that of the learned model LM.

[0119] (Flow of the inspection method by the inspection device 1a) The inspection method by the inspection device 1a will be described with reference to FIG. 3. The inspection method by the inspection device 1a is described by modifying the acquisition step M11 and the inspection step M12 included in the inspection method M1 shown in FIG. 3 as follows. In other respects, it is the same as the inspection method M1.

[0120] The acquisition step M11 is modified to acquire the inspection image, sand information, molding information, conveyance information, environmental information, and reference image and store them in the primary memory 12. The processor 11 acquires, for example, a reference image specified by the user. The specification of the reference image by the user is realized, for example, by the user inputting the storage location of the reference image in the secondary memory 13 to the inspection device 1a.

[0121] The inspection step M12 is modified to read the inspection image, sand information, molding information, conveyance information, environmental information, and reference image from the primary memory 12 and input them to the learned model LMa.

[0122] (Configuration of the machine learning device 2a) The machine learning device 2a differs from the machine learning device 2 in that it constructs a learned model LMa instead of the learned model LM using a learning dataset DSa instead of the learning dataset DS for the machine learning device 2. In other respects, it is the same as the machine learning device 2.

[0123] (Flow of the machine learning method by the machine learning device 2a) The machine learning method by machine learning device 2a will be described with reference to Fig. 5. The machine learning method by machine learning device 2a can be explained by modifying each step included in machine learning method M2 shown in Fig. 5 as follows. In all other respects, it is the same as machine learning method M2.

[0124] The training dataset construction step M21 is modified to construct a training dataset DSa instead of the training dataset DS. The training dataset DSa is a collection of training data Da-i (i = 1, 2, ...). The processor 21 stores the training data Da-i in the secondary memory 23, which includes, as input data, (i) an inspection image captured by the camera 3, (ii) information acquired from each device (sand information, molding information, transportation information, environmental information), and (iii) a reference image designated by the user, and (iv) information indicating the inspection results by the worker (information indicating the presence or absence of defects and information indicating the area of one or more defects). The training dataset DSa is constructed by the processor 21 repeating the above process.

[0125] The trained model construction step M22 is modified to construct the trained model LMa by supervised learning using the training data set DSa. The processor 21 stores the constructed trained model LMa in the secondary memory 23.

[0126] This modified example uses a trained model LMa that references a reference image as input data in addition to the inspection image, and therefore can obtain information that indicates inspection results with higher accuracy.

[0127] [Variation 2] One embodiment of the present invention described above can be modified so that the trained model used to inspect the product surface of the mold 9 is selected depending on the type of model used to create the mold 9.

[0128] The configuration of the inspection system Sb according to this modification example will be described with reference to FIG. 9. FIG. 9 is a block diagram showing the configuration of the inspection system Sb. The inspection system Sb includes an inspection device 1b, a machine learning device 2b, a camera 3, and illumination 4.

[0129] (Configuration of inspection device 1b) The inspection device 1b is different from the inspection device 1 in that it inspects the product surface of the mold 9 using a learned model L Mb-j (j = 1, 2,...) instead of the learned model LM. j is identification information for identifying the type of the model. The model of the type identified by the identification information j is also referred to as model j. The inspection device 1b stores a plurality of learned models L Mb-1, L Mb-2,... in the secondary memory 13. Otherwise, it is the same as the inspection device 1b.

[0130] (Learned model L Mb-j) The learned model L Mb-j is machine-learned according to the model j. The input and output of the learned model L Mb-j are the same as the input and output of the learned model LM.

[0131] (Flow of inspection method by inspection device 1b) The flow of the inspection method M1b by the inspection device 1b will be described with reference to FIG. 10. FIG. 10 is a flowchart showing the flow of the inspection method M1b. The inspection method M1b is different from the inspection method M1 shown in FIG. 3 in that it includes an inspection step M12b instead of the inspection step M12 and further includes a selection step M15b. For each of the other steps, it is the same as the inspection method M1.

[0132] In the selection step M15b, the processor 11 acquires the identification information j of the model input by the user and selects the learned model L Mb-j corresponding to the acquired identification information j. The identification information j input by the user identifies the type of the model used for the molding of the mold 9 to be inspected.

[0133] In the inspection step M12b, the processor 11 performs the same processing as in the inspection step M12 using the learned model Lmb-j selected in the selection step M15b.

[0134] (Configuration of the machine learning device 2b) The machine learning device 2b differs from the machine learning device 2 in that it constructs a plurality of learned models Lmb-1, Lmb-2,.... Otherwise, it is the same as the machine learning device 2.

[0135] (Flow of the machine learning method by the machine learning device 2b) The flow of the machine learning method M2b by the machine learning device 2b will be described with reference to FIG. 11. FIG. 11 is a flowchart showing the flow of the machine learning method M2b. The machine learning method M2b executes a learning data set construction step M21b and a learned model construction step M22b for each of a plurality of types of models 1, 2,....

[0136] In the learning data set construction step M21b, the processor 21 executes the same processing as in the learning data set construction step M21 for the model j, and constructs a learning data set Dsb-j.

[0137] Also, in the learned model construction step M22b, the processor 21 constructs a learned model Lmb-j by supervised learning using the learning data set Dsb-j. Then, the processor 21 stores the constructed learned model Lmb-j in the secondary memory 23.

[0138] In this modification example, a learned model Lmb-j is constructed according to the model j. As a result, in this modification example, the learned model Lmb-j is selected according to the model j used for the molding of the mold 9 to be inspected, and the product surface of the mold 9 is inspected using the selected learned model Lmb-j. As a result, in this modification example, the accuracy of detecting defects specific to the model j is improved.

[0139] [Modification Example 3] One embodiment of the present invention described above can be modified by combining Modification Examples 2 and 3.

[0140] Regarding the configuration of the inspection system Sc according to this modification example, it will be described with reference to FIG. 12. FIG. 12 is a block diagram showing the configuration of the inspection system Sc. The inspection system Sc includes an inspection device 1c, a machine learning device 2c, a camera 3, and illumination 4.

[0141] (Configuration of the inspection device 1c) The inspection device 1c is different from the inspection device 1b according to Modification Example 2 in that it inspects the product surface of the mold 9 using a learned model LMc-j instead of the learned model LMb-j (j = 1, 2,...). The inspection device 1c stores a plurality of learned models LMc-1, LMc-2,... in the secondary memory 13. Other points are the same as those of the inspection device 1b.

[0142] (Learned model LMc-j) The learned model LMc-j is machine-learned according to the model j. j is identification information for identifying the type of the model, similar to Modification Example 2. The input of the learned model LMc-j further includes a reference image in addition to the input of the learned model LMb-j in Modification Example 2. That is, the input of the learned model LMc-j is the same as the input of the learned model LMa in Modification Example 1. The output of the learned model LMc-j is the same as the output of the learned model LMb-j in Modification Example 2.

[0143] (Flow of the inspection method by the inspection device 1c) Regarding the inspection method by the inspection device 1c, it will be described with reference to FIG. 10. The inspection method by the inspection device 1c is explained by modifying the acquisition step M11 and the inspection step M12b included in the inspection method M1b according to Modification Example 2 shown in FIG. 10 as follows.

[0144] The acquisition step M11 is modified to acquire the inspection image, sand information, molding information, conveyance information, environment information, and reference image and store them in the primary memory 12. The processor 11 acquires, for example, a reference image specified by the user. The specification of the reference image by the user is realized, for example, by the user inputting the storage location of the reference image in the secondary memory 13 to the inspection apparatus 1c.

[0145] The inspection step M12b is modified to read the inspection image, sand information, molding information, conveyance information, environment information, and reference image from the primary memory 12 and input them to the learned model LMc-j selected in the selection step M15b.

[0146] (Configuration of the machine learning device 2c) The machine learning device 2c differs from the machine learning device 2b according to Modification Example 2 in that a reference image is further used as input data for constructing each of the plurality of learned models LMc-1, LMc-2,.... Other points are the same as those of the machine learning device 2b.

[0147] (Flow of the machine learning method by the machine learning device 2c) The machine learning method by the machine learning device 2c will be described with reference to FIG. 11. The machine learning method M2c is described by modifying each step included in the machine learning method M2b according to Modification Example 2 shown in FIG. 11 as follows.

[0148] The learning dataset construction step M21b is modified to construct a learning dataset DSc-j instead of the learning dataset DSb-j (j = 1, 2, …). The learning dataset DSc-j consists of teacher data Dc-ji (i = 1, 2, …). The processor 21 includes, as input data, (i) an inspection image in which the product surface of the mold 9 is imaged by the camera 3, (ii) information acquired from each device (sand information, molding information, conveyance information, environment information), and (iii) a reference image specified by the user, and stores, in the secondary memory 23, the teacher data Dc-ji including, as correct data, information indicating the inspection result by the operator (information indicating the presence or absence of defects and information indicating the regions of one or more defects). By repeating the above process by the processor 21, the learning dataset DSc-j is constructed.

[0149] The learned model construction step M22 is modified to construct a learned model LMc-j by supervised learning using the learning dataset DSc-j. The processor 21 stores the constructed learned model LMc-j in the secondary memory 23.

[0150] In this modification example, a learned model LMc-j is constructed according to the mold j. The learned model LMc-j includes a reference image as input data. Thereby, this modification example selects the learned model LMc-j corresponding to the mold j used for molding the mold 9 to be inspected, inputs the reference image of the mold j to the learned model LMc-j, and inspects the product surface of the mold 9. As a result, in this modification example, the accuracy of detecting defects peculiar to the mold j is further improved.

[0151] 〔Summary〕 The inspection apparatus according to Aspect 1 includes one or more processors that execute an inspection step of inspecting the appearance of a mold using a learned model constructed by machine learning. The input of the learned model includes an inspection image obtained by imaging the appearance of the mold. The output of the learned model is information indicating the inspection result of the appearance of the mold.

[0152] With the above configuration, it is not necessary to define in advance the features of the appearance of the mold that one wishes to detect, such as defects, and therefore the possibility of judging a mold that is actually normal as abnormal or a mold that is actually abnormal as normal is reduced, thereby improving the accuracy of mold appearance inspection.

[0153] The inspection device according to aspect 2 has the following features in addition to the features of the inspection device according to aspect 1. That is, in the inspection device according to aspect 2, the appearance of the mold is the appearance of the product surface of the mold.

[0154] The above configuration improves the accuracy of inspecting the appearance of the product surface of the mold.

[0155] The inspection device according to aspect 3 has the following features in addition to the features of the inspection device according to aspect 1 or 2. That is, in the inspection device according to aspect 3, the one or more processors further execute a selection step of selecting a trained model according to the type of model used to make the mold, and in the inspection step, inspect the appearance of the mold using the trained model selected in the selection step.

[0156] With the above configuration, inspection results can be obtained according to the type of model used to create the casting mold, further improving inspection accuracy.

[0157] The inspection device according to aspect 4 has the following features in addition to the features of the inspection device according to any one of aspects 1 to 3. That is, in the inspection device according to aspect 4, the input of the trained model further includes a reference image capturing the appearance of a normal mold.

[0158] With the above configuration, the inspection accuracy is further improved by referring to the reference image in addition to the inspection image.

[0159] The inspection apparatus according to aspect 5 has the following features in addition to the features of the inspection apparatus according to any one of aspects 1 to 4. That is, in the inspection apparatus according to aspect 5, the input of the learned model further includes sand information indicating the molding sand constituting the mold, molding information indicating the molding status of the mold, conveyance information indicating the conveyance status of the mold, and part or all of the environmental information.

[0160] With the above configuration, by further referring to the sand information, the molding information, the conveyance information, and part or all of the environmental information in addition to the inspection image, the inspection accuracy is further improved.

[0161] The inspection apparatus according to aspect 6 has the following features in addition to the features of the inspection apparatus according to any one of aspects 1 to 5. That is, in the inspection apparatus according to aspect 6, the output of the learned model indicates at least the presence or absence of defects in the appearance of the mold.

[0162] With the above configuration, the presence or absence of defects in the appearance of the mold can be obtained.

[0163] The inspection apparatus according to aspect 7 has the following features in addition to the features of the inspection apparatus according to aspect 6. That is, in the inspection apparatus according to aspect 7, when the output of the learned model indicates that there are defects in the appearance of the mold, the one or more processors further execute a warning step of outputting warning information based on the information indicating the inspection result.

[0164] With the above configuration, the user can be warned that there are defects in the appearance of the mold. For example, the user can refer to the warning information and skip the subsequent processes using the mold with defects in the appearance.

[0165] The inspection method according to aspect 8 includes an inspection step in which one or more processors inspect the appearance of a mold using a learned model constructed by machine learning. The input of the learned model includes an inspection image obtained by imaging the appearance of the mold, and the output of the learned model is information indicating the inspection result of the appearance of the mold.

[0166] The above configuration provides the same effects as the inspection device according to the first aspect.

[0167] A machine learning device according to a ninth aspect includes one or more processors that execute a construction step of constructing a trained model for inspecting the appearance of a mold by supervised learning using a training dataset. The input of the trained model includes an inspection image of the appearance of the mold, and the output of the trained model is information indicating the inspection result of the appearance of the mold.

[0168] With the above configuration, it is possible to construct a trained model that can inspect the appearance of a mold with high accuracy.

[0169] A machine learning method according to aspect 10 includes a construction step in which one or more processors construct a trained model for inspecting the appearance of a mold by supervised learning using a training dataset. The input of the trained model includes an inspection image of the appearance of the mold, and the output of the trained model is information indicating the inspection result of the appearance of the mold.

[0170] The above configuration provides the same effects as the machine learning device according to aspect 9.

[0171] [Additional Notes] The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the present invention. [Explanation of symbols]

[0172] Sa, Sb, Sc Inspection System 1, 1a, 1b, 1c Inspection equipment 2, 2a, 2b, 2c Machine learning device 3 Camera 4. Lighting 9. Mold 11, 21 processors 12, 22 Primary memory 13, 23 Secondary memory 14, 24 Input / output interface 15, 25 Communication interface 16, 26 Bus M1, M1b Inspection method M11 Acquisition step M12, M12b Inspection steps M13 Judgment step M14 Warning step M15b Selection step M2, M2b Machine learning method M21, M21b Steps for constructing a learning dataset

Claims

1. One or more processors that execute an inspection step and a warning step, and an output device, In the inspection step, the one or more processors inspect the appearance of the mold using a trained model constructed by machine learning, The input of the trained model includes an inspection image obtained by imaging the appearance of the mold, The output of the trained model is information indicating the inspection result of the appearance of the mold, In the warning step, when the output of the trained model indicates that there is a defect in the appearance of the mold, the one or more processors output warning information to the output device based on the information indicating the inspection result, The one or more processors execute the inspection step and the warning step after the molding step and before the core setting step in a casting process including a molding step of molding the mold and a core setting step of setting a core in the mold, An inspection device characterized by the above.

2. The appearance of the mold is the appearance of the product surface of the mold, The inspection device according to claim 1, characterized by the above.

3. The one or more processors further execute a selection step of selecting a trained model according to the type of the model used for molding the mold, In the inspection step, the appearance of the mold is inspected using the trained model selected in the selection step, The inspection device according to claim 1 or 2, characterized by the above.

4. The input of the trained model further includes a reference image obtained by imaging the appearance of a normal mold, The inspection device according to any one of claims 1 to 3, characterized by the above.

5. The input of the trained model further includes some or all of sand information indicating the casting sand constituting the mold, molding information indicating the molding status of the mold, conveyance information indicating the conveyance status of the mold, and environmental information, The inspection device according to any one of claims 1 to 4, characterized by the above.

6. The output of the trained model indicates at least the presence or absence of defects in the appearance of the mold, The inspection device according to any one of claims 1 to 5, characterized by the above.

7. An inspection method including one or more processors executing an inspection step and a warning step, In the inspection step, the one or more processors inspect the appearance of the mold using a trained model constructed by machine learning, ​ The input of the learned model includes an inspection image obtained by imaging the appearance of the mold, the output of the learned model is information indicating the inspection result of the appearance of the mold, in the warning step, when the output of the learned model indicates that there is a defect in the appearance of the mold, the one or more processors output warning information to an output device based on the information indicating the inspection result, the inspection step and the warning step are executed after the molding step and before the core setting step in a casting process including a molding process for molding the mold and a core setting process for setting cores in the mold, characterized by an inspection method.

Images