Device for managing defect of object to be inspected, machine learning method, and method for manufacturing trained model

Abstract

A control device (10) calculates the luminance of the transmitted light in each of a plurality of regions included in an object to be inspected (S) on the basis of information acquired from a camera (201). The control device (10) estimates the shape of a first surface and the shape of a second surface on the basis of information acquired from sensors (301, 401). The control device (10) estimates density information indicating a density distribution of each of the plurality of regions on the basis of the shape of the first surface and the shape of the second surface, and the luminance of the transmitted light in each of the plurality of regions.

Description

Apparatus for managing defects in inspected objects, machine learning method, and method for manufacturing trained model

[0001] The present disclosure relates to an apparatus for managing defects in an inspected object, a machine learning method, and a method for manufacturing a trained model.

[0002] In the quality inspection of manufactured products, it is required to determine defects on the surface and inner layers of the inspected object by non-destructive inspection.To determine whether there are defects, one method is to irradiate the inspected object with light, capture an image of that state with a camera, and determine that there is a defect if there is a point where the brightness of the captured image is high.

[0003] For example, Japanese Patent Laid-Open Publication No. 2014-163694 (Patent Document 1) discloses a defect inspection method in which a sheet-like object to be inspected is irradiated from one side with a first light and from the other side with a second light, imaging data corresponding to the intensities of the reflected light of the first light and the transmitted light of the second light is acquired, and inner layer defects of the object to be inspected are detected based on the acquired intensities of the reflected light and the transmitted light. In this defect inspection method, a brightness ratio obtained by dividing the brightness value of the acquired reflected light or transmitted light by the brightness value when there is no defect is used to determine whether a defect occurring in the object to be inspected is a surface defect or an inner layer defect.

[0004] JP 2014-163694 A

[0005] The defect inspection method described in Patent Document 1 can determine whether or not there are defects in a sheet-like object to be inspected, but cannot estimate the volume of the area where the defect occurs (for example, the volume of the vacuoles caused by the defects) or the density of the area containing the defect (for example, a decrease in the density of some areas due to the inclusion of vacuoles) in a thick object to be inspected.

[0006] For example, in urethane foam resin molded products, which are formed by injecting resin into a mold and then foaming it into a three-dimensional shape through a foaming process, defects such as chips or cavities can occur on the surface or in the inner layer. In urethane foam resin molded products, where the molding results are irregular, it is difficult to control the density of the molded product, and it has not been possible to create an optimal molding process that uniforms the density.

[0007] The present disclosure has been made to solve the above-mentioned problems, and its purpose is to provide a technology that can appropriately manage the density of an object to be inspected, which is related to defects in the object to be inspected.

[0008] The present disclosure provides an apparatus for managing defects in an object to be inspected, comprising an illumination device, an imaging device, a sensor, and a processing device. The illumination device illuminates a first surface of the object to be inspected with transmitted light. The imaging device is disposed opposite the illumination device and captures an image of a second surface of the object to be inspected opposite the first surface. The sensor measures information related to the outer shape of the object to be inspected. The processing device processes information acquired from the sensor and the imaging device. The processing device calculates the luminance of the transmitted light in each of a plurality of regions included in the object to be inspected based on the information acquired from the imaging device. Each of the plurality of regions is a three-dimensional region between a two-dimensional minute first region included in the first surface and a two-dimensional minute second region included in the second surface. The second region is a region into which the transmitted light irradiated onto the first region passes through the object to be inspected. The processing device estimates the shapes of the first surface and the second surface based on the information acquired from the sensor. The processing device estimates density information indicating the density distribution of each of the plurality of regions based on the shapes of the first surface and the second surface and the luminance of the transmitted light in each of the plurality of regions.

[0009] The apparatus for managing defects in an object to be inspected according to the present disclosure includes a data acquisition unit and a model generation unit. The data acquisition unit acquires training data including density information indicating the density distribution of each of a plurality of regions included in the object to be inspected and injection path information of the object to be inspected corresponding to the density information. The model generation unit uses the training data to generate a trained model for inferring the injection path information from the density information. The injection path information includes an injection path along which a robot that generates the object to be inspected by injecting material for the object to be inspected into a mold for molding the object to move while injecting the material for the object to be inspected.

[0010] The apparatus for managing defects in an object to be inspected according to the present disclosure includes a data acquisition unit and an inference unit. The data acquisition unit acquires density information indicating the distribution of densities in each of a plurality of regions included in the object to be inspected. The inference unit outputs injection path information from the density information acquired by the data acquisition unit, using a trained model for inferring injection path information for the object to be inspected from the density information. The injection path information is information including an injection path along which a robot that generates the object by injecting material for the object to be inspected into a mold for molding the object moves while injecting the material for the object to be inspected.

[0011] The machine learning method for managing defects in an object to be inspected according to the present disclosure includes the steps of acquiring training data including density information indicating a density distribution of each of a plurality of regions included in the object to be inspected and injection path information of the object to be inspected corresponding to the density information, and generating a trained model using the training data for inferring the injection path information from the density information. The injection path information is information including an injection path along which a robot that generates the object by injecting material of the object to a mold for molding the object moves while injecting the material of the object to be inspected.

[0012] The manufacturing method of the trained model of the present disclosure includes an acquisition step of acquiring training data including density information indicating the density distribution of each of a plurality of regions included in the object to be inspected and injection path information of the object to be inspected corresponding to the density information, and a generation step of generating a trained model for inferring the injection path information from the density information using the training data. The injection path information is information including an injection path along which a robot that generates the object to be inspected by injecting material of the object to a mold for molding the object moves while injecting the material of the object to be inspected.

[0013] According to the present disclosure, it is possible to appropriately manage the density of an object to be inspected, which is related to defects in the object to be inspected.

[0014] FIG. 1 is a diagram showing a schematic configuration of the entire defect management apparatus according to embodiment 1. FIG. 2 is a diagram showing a cross-sectional shape of an object to be inspected. FIG. 3 is a diagram showing a cross-sectional shape of an object to be inspected. FIG. 4 is a diagram showing a cross-sectional shape of an object to be inspected. FIG. 5 is a diagram showing a cross-sectional shape of an object to be inspected. FIG. 6 is a graph showing a virtual cross-section of an object to be inspected. FIG. 7 is a graph showing a virtual cross-section of an object to be inspected. FIG. 8 is a graph showing brightness and cross-sectional density of an object to be inspected. FIG. 9 is a graph showing corrected cross-sectional density of an object to be inspected. A flowchart showing processing executed by a control device. A flowchart showing defect determination processing. A diagram showing a hardware configuration of a control device. FIG. 10 is a diagram showing a schematic configuration of the entire defect management apparatus according to embodiment 2. FIG. 11 is a diagram showing a process in which an object to be inspected is generated by a robot. A diagram showing how an object to be inspected is measured. A diagram showing how an object to be inspected is measured. A diagram showing how an object to be inspected is measured. A diagram showing how an object to be inspected is measured. A diagram showing the configuration of a learning device. A flowchart showing learning processing of the learning device. A diagram showing the configuration of an inference device. A flowchart showing an inference procedure by the inference device.

[0015] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In the drawings, identical or corresponding parts are designated by the same reference numerals, and their descriptions will not be repeated in principle. However, unless otherwise specified, the dimensions, materials, shapes, relative positions, etc. of the components described in the examples are not intended to limit the scope of the present invention to those.

[0016] 1 is a diagram showing a schematic configuration of the entire defect management device 1 according to embodiment 1. The defect management device 1 is a device that manages defects in an object to be inspected S. The defect management device 1 inspects the object to be inspected S transported by a transport conveyor 501.

[0017] The specimen S is, for example, a urethane foam resin molded product used as a thermal insulator. A urethane foam resin molded product is formed by injecting resin into a mold having the desired shape and then undergoing a foaming process to form a three-dimensional shape. Depending on how the foaming process progresses, defects such as chips or cavities that can become scratches may occur on the surface or inner layer of the finished resin molded product.

[0018] The defect management device 1 according to the first embodiment acquires the external shape of the object S to be inspected and detects the amount of defects present in the inner layers, thereby acquiring the accurate density of the object S to be inspected. The defect management device 1 can determine the presence or absence of defects based on the density of the object S to be inspected, rather than determining the presence or absence of scratches or cavities that may occur.

[0019] The defect management device 1 includes a light source 101 as an irradiation device, a camera 201 as an imaging device, sensors 301, 401, a transport conveyor 501, a control device 10 as a processing device, and a memory unit 660 that stores data output by the control device 10.

[0020] The sensors 301 and 401 measure information relating to the outer shape of the object to be inspected S. The sensors (sensors 301 and 401) include a sensor (also referred to as a "lower sensor") 401 as a first sensor that measures a first surface (lower surface) of the lower part of the object to be inspected S, and a sensor (also referred to as an "upper sensor") 301 as a second sensor that measures a second surface (upper surface) of the upper part of the object to be inspected S. By using the sensors 301 and 401, the outer shape dimensions of the object to be inspected S can be obtained.

[0021] The light source 101 irradiates light (transmitted light) onto the bottom surface of the inspection object S. The camera 201 is disposed opposite the light source 101 and captures an image of the top surface of the inspection object S opposite the bottom surface. The camera 201 captures an image of the transmitted light that has been irradiated from the light source 101 and transmitted through the inspection object S.

[0022] The control device 10 processes the information acquired from the sensors 301, 401 and the camera 201, estimates density information of the inspection object S (described later), and can determine whether the inspection object S has a defect.

[0023] The transport conveyor 501 moves the inspection object S in the X-axis direction. The sensors 301 and 401 continue to measure the shape of the inspection object S as it moves in the X-axis direction while being transported by the transport conveyor 501. The sensors 301 and 401 measure the distance in the Z direction from their respective installation positions to the inspection object S by the amount of the Y-direction component. The sensors 301 and 401 are selected to have an appropriate resolution that affects measurement accuracy within a range that covers the inspection object S.

[0024] The upper shape acquisition unit 611 acquires information from the sensor (upper sensor) 301. The upper shape acquisition unit 611 acquires information on the height z1(y, t) for the Y direction component in the transport time unit t as the measurement result of the upper sensor 301.

[0025] Similarly, the lower shape acquisition unit 621 acquires information from the sensor (lower sensor) 401. The lower shape acquisition unit 621 acquires information on the height z2(y, t) for the Y direction component in the transport time unit t as the measurement result of the lower sensor 401.

[0026] After the measurements by the sensors 301 and 401 are completed, inspection is performed using the light source 101 and the camera 201. If the object to be inspected S has a cavity, the amount of transmitted light from the light source 101 is greater than when there is no cavity, and the brightness of the image captured by the camera 201 is therefore higher. The transmitted light processing unit 631 acquires the image data captured by the camera 201. This makes it possible to detect inner layer defects in the object to be inspected S based on the brightness of the image data.

[0027] 2 to 5 are diagrams showing the cross-sectional shape of the inspection object S. The ideal cross section 11 shown in Fig. 2 is a cross section of the inspection object S when it is ideally formed. Figs. 3 to 5 show cross sections of the inspection object S that are actually produced.

[0028] 3 , the upper part shape acquisition unit 611 acquires the shape of the cross section 12 within the sensor detection range 302 of the upper sensor 301. Specifically, the upper part shape acquisition unit 611 can acquire the shape of the upper surface of the inspection object S. A surface defect 21 exists in the upper part of the cross section 12.

[0029] 4 , the lower shape acquisition unit 621 acquires the shape of the cross section 13 within the sensor detection range 402 of the lower sensor 401. Specifically, the lower shape acquisition unit 621 can acquire the shape of the bottom surface of the inspection object S. A surface defect 21 exists in the upper part of the cross section 13, and a surface defect 22 exists in the lower part of the cross section 13.

[0030] For example, if the cross-sectional shape of the inspection object S is circular, the upper sensor 301 detects the shape of the upper part of the circle, and the lower sensor 401 detects the shape of the lower part of the circle.

[0031] 5, a light source 101 irradiates the lower surface of an object S under inspection with light. A camera 201 captures an image of the upper surface of the object S under inspection, allowing observation of the state of transmitted light that has passed through the object S under inspection. A transmitted light processing unit 631 acquires the image captured by the camera 201. In addition to a surface defect 21 and a surface defect 22, an inner layer defect 23 is present on the cross section 14.

[0032] 6 and 7 are graphs showing a virtual cross section of the object to be inspected S. As shown in Fig. 6, the outer shape of the upper surface of the object to be inspected S is measured as a measurement height z1(y, t) in the cross section 12 shown in Fig. 3. When the ideal cross section 11 is measured, a measurement height zT1 is obtained.

[0033] The upper shape comparison unit 612 calculates the difference (increase) δz1(y,t) [=z1(y,t)−zT1] between the acquired measurement height z1(y,t) and the measurement height zT1 of the ideal cross section 11.

[0034] However, if a surface defect is detected at δz1(y,t), the surface defect is removed to obtain the virtual cross section A1(y,t). For example, in δz1(y,t), "0" is used as a threshold, and defect portions 31 (detected as surface defects 21) that are 0 or less are removed from δz1(y,t) (if they are less than 0, they are corrected to 0). As a result, the surface defect 21 is corrected as the measurement height zT1.

[0035] 7, the outer shape of the lower surface of the object S to be inspected is measured as a measurement height z2(y, t) in the cross section 13 shown in FIG. 4. When the ideal cross section 11 is measured, a measurement height zT2 is obtained.

[0036] The lower shape comparison unit 622 calculates the difference (decrease) δz2(y,t) [=z2(y,t)−zT2] between the acquired measurement height z2(y,t) and the measurement height zT2 of the ideal cross section 11.

[0037] However, if a surface defect is detected in δz1(y,t), the surface defect is removed to obtain the virtual cross section A2(y,t). For example, in δz2(y,t), a defect portion 32 (detected as a surface defect 22) that is equal to or smaller than a threshold value (a negative value in this example) is removed from δz2(y,t).

[0038] 8 is a graph showing the brightness and cross-sectional density of the inspection object S. In the image data captured by the camera 201, the brightness value is higher at a location that includes the inner layer defect 23 than at a location that does not include the inner layer defect 23 because the amount of transmitted light is greater at that location. The brightness value is also higher at locations that include the surface defects 21 and 22.

[0039] The luminance of the ideal cross section 11 where the inner layer state is uniform is luminance LT. The luminance of the cross section 14 of the inspection object S shown in Figure 5 is L(y, t). The luminance value is high due to the surface defect 21, inner layer defect 23, and surface defect 22.

[0040] The defect detection unit 632 calculates the cross-sectional density A3(y,t) in the Y-axis direction. The cross-sectional density A3(y,t) is obtained by correcting and converting the value of L(y,t) based on the correlation with the luminance LT. The cross-sectional density A3(y,t) is calculated based on the inner layer defect or the defect amount due to the inner layer defect (value expressed in L(y,t)) relative to the density of the inspection object S in the ideal cross-section 11 (value based on the luminance LT).

[0041] 9 is a graph showing the corrected cross-sectional density of the inspection object S. The cross-sectional density calculation unit 641 synthesizes and adds the cross-sectional densities in the Y-axis direction to obtain the cross-sectional density A(y,t). Specifically, as shown in FIG. 9, the cross-sectional density A(y,t) is calculated by the formula A(y,t)=A1(y,t)+A2(y,t)+A3(y,t).

[0042] The value A(t) obtained by integrating the cross-sectional density A(y, t) in the Y-axis direction can be derived as the cross-sectional density in a very short time unit. Furthermore, by integrating the value A(t) in the X-axis direction, the total density A of the object S to be inspected can be derived. The output unit 651 outputs information on the cross-sectional density A(t) in the very short time unit, the value of the total density A, etc. as results.

[0043] The cross-sectional density calculation unit 641 can calculate the cross-sectional area and total volume of the object S to be inspected excluding cavities. If there are no cavities, the density of the object S to be inspected is homogeneous. If there are cavities, the density of the object S to be inspected decreases locally. For example, it is possible to calculate the cross-sectional area and total volume of the object S to be inspected excluding cavities based on the distance between the top and bottom surfaces and density information. Furthermore, the total weight of the object S to be inspected can be calculated from the total density A and total volume.

[0044] The process executed by the control device 10 will be described below using a flowchart. Fig. 10 is a flowchart showing the process executed by the control device 10. Hereinafter, a step will also be simply referred to as "S." This process may be started, for example, every time the camera 201 and sensors 301 and 401 detect an inspection object S moving on the transport conveyor 501 (at every sampling time).

[0045] When this process starts, in S11, the control device 10 acquires information (distance to the underside) detected by the lower sensor 401. In S12, the control device 10 estimates the shape of the underside based on the information acquired from the lower sensor 401. As described with reference to FIGS. 4 and 7 , for example, the outer shape of the underside of the object S to be inspected is calculated as the measurement height z2(y, t), and further, a virtual cross section A2(y, t) is obtained.

[0046] In S13, the control device 10 acquires information (distance to the upper surface) detected by the upper sensor 301. In S14, the control device 10 estimates the shape of the upper surface based on the information acquired from the upper sensor 301. As described with reference to FIGS. 3 and 6 , for example, the outer shape of the upper surface of the inspection object S is calculated as the measurement height z1(y, t), and further, a virtual cross section A1(y, t) is obtained.

[0047] In S15, the control device 10 acquires the information detected by the camera 201. In S16, the control device 10 calculates the luminance of the transmitted light in each region based on the information acquired from the camera 201. As described with reference to FIGS. 5 and 8, for example, L(y, t) is calculated as the luminance, and further, the cross-sectional density A3(y, t) is obtained.

[0048] In S17, the control device 10 estimates density information based on the shape of the lower surface, the shape of the upper surface, and the brightness of the transmitted light, and then ends this process. As described above, the lower sensor 401 obtains the measurement height z2(y,t), the upper sensor 301 obtains the measurement height z1(y,t), and the camera 201 obtains the brightness L(y,t). The cross-sectional density A(y,t) is then obtained from the finally calculated virtual cross sections A2(y,t), A1(y,t), and cross-sectional density A3(y,t). The value A(t) is obtained by integrating the cross-sectional density A3(y,t) in the Y-axis direction.

[0049] This process is executed at predetermined intervals (sampling periods) while the object S to be inspected is moving in the X-axis direction. While the object S to be inspected is being transported, the length of the object S to be inspected in the transport direction (X-axis direction) can be calculated based on the timing of the start and end of inspection by the sensor 301, etc. The total density A can be obtained by integrating the value A(t) in the X-axis direction. Furthermore, as described above, the cross-sectional area, total volume, and total weight of the object S to be inspected excluding cavities can be calculated.

[0050] According to the configuration described above, the control device 10 divides the inspection object S into a plurality of regions and estimates the density of each of the plurality of regions. Each of the plurality of regions is a minute region in the Y-axis direction indicated by y(t) in FIG. 6 or 7, and a minute region in the X-axis direction due to the movement of the inspection object S in each sampling period.

[0051] Each of the multiple regions is a region defined by a small width in the Y-axis direction x a small width in the Z-axis direction, and a density is obtained for each of these small regions. Here, the region in contact with the lower surface of the micro region is referred to as a first region, and the region in contact with the upper surface of the micro region is referred to as a second region.

[0052] In other words, each of the plurality of regions is a three-dimensional region between a two-dimensional minute first region included in the lower surface and a two-dimensional minute second region included in the upper surface. The second region is a region where transmitted light irradiated onto the first region passes through the inspection object S and reaches.

[0053] The sensor 401 measures the distance to a first region on the bottom surface. The sensor 301 measures the distance to a second region on the top surface. The sensors 301 and 401 repeat these measurements to measure the shape of the object S to be inspected.

[0054] The control device 10 estimates the shape of the bottom surface based on information acquired from the sensor 401. The control device 10 estimates the shape of the top surface based on information acquired from the sensor 301. Furthermore, the control device 10 calculates the luminance of transmitted light in each of a plurality of regions included in the object S to be inspected based on information acquired from the camera 201. The control device 10 estimates density information indicating the density distribution of each of the plurality of regions based on the shapes of the bottom surface and the top surface and the luminance of transmitted light in each of the plurality of regions.

[0055] In this way, it is possible to determine whether the density of multiple regions is homogeneous. If there is a region where the density is reduced, it can be determined that the object to be inspected has a defect due to vacuoles or the like. In this embodiment, rather than simply determining whether the object has an inner layer, the quantitative density of the object to be inspected can be obtained, allowing the amount of defects to be numerically determined and the quality to be determined. This allows the density of the object to be appropriately managed in relation to defects in the object to be inspected.

[0056] 11 is a flowchart showing the defect determination process. The defect determination process may be started, for example, when the inspection shown in FIG.

[0057] When the defect determination process begins, the control device 10 detects vacuoles in S21 based on the density information obtained in the process shown in Fig. 10. For example, if there are multiple areas where the density drops, it can be determined that there are vacuoles. For example, this is the case in the inner layer 23 shown in Figs. 5 and 9.

[0058] In S22, the control device 10 determines whether or not a vacuole having a predetermined volume or more has been detected. For example, the volume of the vacuole can be calculated based on the amount of decrease in density and the size of the area where the density is decreased.

[0059] If the control device 10 determines that a void (an unacceptably large inner layer defect) of a predetermined volume or more has been detected (YES in S22), it determines that the object being inspected has a defect (S23) and terminates the defect determination process.

[0060] On the other hand, if the control device 10 does not detect any voids of a predetermined volume or more (NO in S22), it determines that the object to be inspected has no defects (S24) and ends the defect determination process. If the defect is an inner layer defect that is acceptable in terms of quality (for example, a minute void), there is no problem.

[0061] In this way, the control device 10 detects vacuoles in the object S to be inspected based on the density information. When vacuoles of a predetermined volume or more are detected, the control device 10 determines that the object S to be inspected has a defect. In this way, it is possible to determine whether or not the object to be inspected has a defect due to vacuoles. This allows the density of the object to be inspected, which is related to defects in the object to be inspected, to be appropriately managed.

[0062] Fig. 12 is a diagram showing the hardware configuration of the control device 10. The corresponding operations of the control device 10 can be configured using digital circuit hardware or software. When the functions of the control device 10 are realized using software, the control device 10 can include, for example, a processor 51 and a memory 52 connected by a bus 53, as shown in Fig. 12, and the processor 51 can execute a program stored in the memory 52.

[0063] 13 is a diagram showing a schematic configuration of the entire defect management device 2 according to embodiment 2. The defect management device 2 is a device that manages defects in the object S to be inspected.

[0064] The defect management device 1 according to the first embodiment includes a light source 101, a camera 201, sensors 301 and 401, a transport conveyor 501, and a control device 10. The defect management device 2 according to the second embodiment further includes a trained model storage unit 661, a learning device 701, an inference device 761, and a robot 801.

[0065] As a result, the defect management device 2 inspects the object to be inspected S transported by the transport conveyor 501, and also generates (manufactures) a learned model using the learning device 701, performs inference using the learned model using the inference device 761, and generates the object to be inspected S using the robot 801.

[0066] 14 is a diagram showing a process in which the inspection object S is produced by the robot 801. The robot 801 produces the inspection object S by injecting the material (base material) of the inspection object S into a mold 821 for molding the inspection object S.

[0067] The robot 801 controls the trajectory (injection path) for injecting the base material of the object S into the shape (mold 821) to be molded, and the amount of base material injected over time, to generate a molding trajectory 811. As a method for injecting the base material, a means such as the robot 801 is assumed. The robot 801 can control the horizontal direction and angular attitude, which affect the quality of the object S.

[0068] 15 to 18 are diagrams showing how the inspection object S is measured. The equipment configuration in Fig. 15 is the same as the equipment configuration shown in Fig. 1. The inspection object S, which is a molded product produced after the base material is injected, is placed on a transport conveyor 501 and inspected by the defect management device 2.

[0069] As shown in Figures 16 to 18, first, the generated inspection object S is transported by a transport conveyor 500, then the inspection object S is inspected on a transport conveyor 501, and after inspection, the inspection object S is transported by a transport conveyor 502.

[0070] The shape of the inspection object S is estimated using sensors 301 and 401 installed between the transport conveyors 500 and 501, and then the luminance of the transmitted light that passes through the inspection object S is measured using a light source 101 and a camera 201 installed between the transport conveyors 501 and 502. Note that the defect management device 1 according to the first embodiment may also be configured using the transport conveyors 500 to 502 in this manner.

[0071] Hereinafter, a learning phase is configured to learn the process of injecting base material into the object S based on the cross-sectional density information acquired by the defect management device 2, and a utilization phase is configured as a molding process to approach the reference cross-sectional density.

[0072] 19 is a configuration diagram of the learning device 701. The learning device 701 includes a data acquisition unit 711 and a model generation unit 721.

[0073] The data acquisition unit 711 acquires, as learning data, data including density information (similar to that in the first embodiment) indicating the density distribution of each of a plurality of regions included in the object S to be inspected, and injection path information of the object S to be inspected corresponding to the density information. The density information is also information indicating the presence or absence of defects (surface / internal layer defects) at each position in the object S to be inspected, and the size of the defects. The injection path information includes the injection path and the injection amount of material (base material) at the injection position on the injection path. The injection path is the path along which the robot 801 moves while injecting the material of the object S to be inspected.

[0074] The model generation unit 721 uses the learning data to generate a trained model for inferring injection route information from the density information of the object to be inspected S. In other words, a trained model is generated that infers injection route information in the input state (injection route information corresponding to the density information) from the density information of the object to be inspected S. When density information (positions and amounts of surface / internal layer defects) is input to the trained model, appropriate injection route information (injection route and injection amount of material at injection positions on the injection route) that deviates little from the reference cross-sectional density is output.

[0075] The learning algorithm used by the model generation unit 721 may be a known algorithm such as supervised learning, unsupervised learning, or reinforcement learning. As an example, a case where reinforcement learning is applied will be described. In reinforcement learning, an agent (subject of action) in a certain environment observes the current state (environmental parameters) and determines the action to be taken. The environment changes dynamically depending on the agent's actions, and the agent is given a reward according to the change in the environment. The agent repeats this process and learns the course of action that will obtain the most reward through a series of actions. Q-learning and TD-learning are known as representative methods of reinforcement learning. For example, in the case of Q-learning, a general update formula for the action value function Q(s, a) is expressed by Equation 1.

[0076]

[0077] In equation 1, st represents the state of the environment at time t, and at represents the action at time t. The state changes to st+1 due to the action at. rt+1 represents the reward obtained due to the change in state, γ represents the discount rate, and α represents the learning coefficient. Note that γ is in the range of 0<γ≦1, and α is in the range of 0<α≦1. The injection route information of the inspection object S becomes the action at, and the density information of the molded product (position and amount of surface / internal layer defects) becomes the state st, and the best action at for the state st at time t is learned.

[0078] The update formula expressed by Equation 1 increases the action value Q if the action value Q of the action a with the highest Q value at time t+1 is greater than the action value Q of the action a executed at time t, and decreases the action value Q in the opposite case. In other words, the action value function Q(s, a) is updated so that the action value Q of the action a at time t approaches the best action value at time t+1. As a result, the best action value in a certain environment is propagated sequentially to the action value in the previous environment.

[0079] As described above, when generating a trained model by reinforcement learning, the model generation unit 721 includes a reward calculation unit 722 and a function update unit 723.

[0080] The reward calculation unit 722 calculates a reward based on the injection route information and density information of the specimen S. The reward calculation unit 722 calculates a reward r based on the deviation value from the reference cross-sectional density. For example, if the deviation value from the reference cross-sectional density decreases, the reward r is increased (for example, a reward of "1" is given), and on the other hand, if the deviation value from the reference cross-sectional density increases, the reward r is decreased (for example, a reward of "-1" is given).

[0081] The function update unit 723 updates the function for determining injection route information in the input state in accordance with the reward calculated by the reward calculation unit 722, and outputs the updated function to the learned model storage unit 661. For example, in the case of Q-learning, the action value function Q(st, at) expressed by Equation 1 is used as the function for calculating injection route information in the input state.

[0082] The learning process described above is repeated. The learned model storage unit 661 stores the action-value function Q(st,at) updated by the function update unit 723, i.e., the learned model.

[0083] Next, the learning process of the learning device 701 will be described with reference to Fig. 20. Fig. 20 is a flowchart showing the learning process of the learning device 701.

[0084] In S101, the data acquisition unit acquires injection route information and density information of the object S to be inspected as learning data.

[0085] In S102, the model generation unit 721 calculates the reward based on the injection route information and density information. Specifically, the reward calculation unit 722 acquires the injection route information and density information, and determines whether to increase the reward (S103) or decrease the reward (S104) based on the deviation value from a predetermined reference cross-sectional density.

[0086] For example, when the difference between the predetermined reference density information of the test object S and the density information is below a predetermined standard (for example, a state in which there are no unacceptably large vacuoles), the model generation unit 721 increases the reward given to the model being trained (behavioral value function Q).

[0087] If the remuneration calculation unit 722 determines that the remuneration should be increased, it increases the remuneration in S103. On the other hand, if the remuneration calculation unit 722 determines that the remuneration should be decreased, it decreases the remuneration in S104.

[0088] In S105, the function update unit 723 updates the action value function Q(st, at) represented by Equation 1 and stored in the trained model storage unit 661 based on the reward calculated by the reward calculation unit 722.

[0089] The learning device repeatedly executes the above steps S101 to S105 and stores the generated action-value function Q(st, at) as a learned model.

[0090] The learning device of this embodiment is configured to store the learned model in a learned model storage unit 661 provided outside the learning device, but the learned model storage unit 661 may also be provided inside the learning device 701.

[0091] Thus, the learning method includes a step of acquiring learning data including density information and injection route information, and a step of generating a trained model for inferring the injection route information from the density information using the learning data. Also, the manufacturing method of the trained model includes an acquisition step of acquiring learning data including density information and injection route information of the test object S corresponding to the density information, and a generation step of generating a trained model for inferring the injection route information from the density information using the learning data.

[0092] 21 is a configuration diagram of an inference device 761. The inference device 761 includes a data acquisition unit 711 and an inference unit 771. The data acquisition unit 711 acquires density information (the same as in the first embodiment) indicating the density distribution of each of a plurality of regions included in the object S to be inspected.

[0093] The inference unit 771 uses a trained model for inferring injection route information of the object S from density information, and outputs injection route information from the density information acquired by the data acquisition unit 711. By inputting the density information acquired by the data acquisition unit 711 into this trained model, it is possible to infer injection route information in the input state (injection route information corresponding to the density information).

[0094] In this embodiment, it has been described that the injection route information in the input state is output using a trained model trained by the model generation unit 721 of the object to be inspected S, but it is also possible to obtain a trained model from another object to be inspected S and output the injection route information in the input state based on this trained model.

[0095] Next, a process for obtaining injection route information in a state where it has been input using the learning device 701 will be described with reference to Fig. 22. Fig. 22 is a flowchart showing the inference procedure performed by the inference device 761.

[0096] In S201, the data acquisition unit 711 acquires density information. In S202, the inference unit 771 inputs the density information into the trained model stored in the trained model storage unit 661 and obtains injection route information in the input state (injection route information corresponding to the density information). The inference unit 771 outputs the obtained injection route information in the input state to the test object S.

[0097] In S203, the inference unit 771 outputs to the robot 801 the injection route information in the input state obtained by the learned model.

[0098] In S204, the robot 801 uses the injection path information in the input state that has been output to control the injection amount at the injection position on the injection path to form the object S. This makes it possible to uniform the cross-sectional density and generate a shape close to the reference cross-sectional density.

[0099] In this embodiment, a case where reinforcement learning is applied to the learning algorithm used by the inference unit has been described, but the present invention is not limited to this. As for the learning algorithm, supervised learning can also be applied in addition to reinforcement learning.

[0100] Furthermore, the learning algorithm used in the model generation unit may be deep learning, which learns to extract the features themselves, or machine learning may be performed according to other known methods, such as neural networks, genetic programming, functional logic programming, and support vector machines.

[0101] The learning device 701 and the inference device 761 may be connected to grasp the state of the inspection object S via a network, for example, and may be devices separate from the molding environment of the inspection object S. The learning device 701 and the inference device 761 may also be built into the inspection object S. Furthermore, the learning device 701 and the inference device 761 may exist on a cloud server.

[0102] The model generation unit 721 may also learn the injection route information in the input state using learning data acquired from multiple test objects S. The model generation unit 721 may acquire learning data from multiple test objects S used in the same area, or may learn the injection route information in the input state using learning data collected from multiple test objects S operating independently in different areas. A learning device that has learned the injection route information in the input state for a certain test object may be applied to another test object, and the injection route information for the other test object may be re-learned and updated.

[0103] The learning device 701 and the inference device 761 can be configured so that the corresponding operations are performed by digital circuit hardware or software, similar to the control device 10. When the functions of the learning device 701 and the inference device 761 are realized by software, the learning device 701 and the inference device 761 can include, for example, a processor 51 and a memory 52 connected by a bus 53, as shown in FIG. 12 , and the processor 51 can execute a program stored in the memory 52.

[0104] As described above, the learning device 701 of the defect management device 2 that manages defects in the object S to be inspected includes a data acquisition unit 711 and a model generation unit 721. The data acquisition unit 711 acquires learning data including density information indicating the density distribution of each of multiple regions included in the object S to be inspected and injection path information for the object S corresponding to the density information. The model generation unit 721 uses the learning data to generate a trained model for inferring the injection path information from the density information. The injection path information includes an injection path along which the robot 801, which generates the object S by injecting the material for the object S into a mold 821 for molding the object S, moves while injecting the material for the object S. The model generation unit 721 increases a reward given to the learning model when a difference between the density information and predetermined reference density information for the object S is equal to or smaller than a predetermined standard.

[0105] The defect management device 2 and the inference device 761 include a data acquisition unit 711 and an inference unit 771. The data acquisition unit 711 acquires density information indicating the density distribution of each of a plurality of regions included in the object to be inspected S. The inference unit 771 outputs injection path information from the density information acquired by the data acquisition unit 711, using a trained model for inferring injection path information of the object to be inspected S from the density information.

[0106] For conventional test objects, such as foamable urethane molded products, where molding results are irregular, it is difficult to control the density of the molding quality, and it is difficult to predict where defects will occur, making it impossible to create an optimal molding process. By configuring as described above, it is possible to produce a test object S with a uniform density using an optimal molding process that does not cause inner layer defects. Furthermore, it is possible to control areas of the test object S where the mass is uneven by increasing or decreasing the amount of resin filled during molding. This allows for appropriate management of the test object's density, which is related to defects in the test object.

[0107] The embodiments disclosed herein should be considered to be illustrative in all respects and not restrictive. Unless there is a contradiction, at least two of the embodiments disclosed herein may be combined. The technical scope of the present disclosure is defined by the claims, not by the description of the above-mentioned embodiments, and is intended to include all modifications within the meaning and scope of the claims.

[0108] 1, 2 Defect management device, 10 Control device, 11 Ideal cross section, 12 to 14 Cross section, 21, 22 Surface defect, 23 Inner layer defect, 31, 32 Measurement height, 51 Processor, 52 Memory, 53 Bus, 101 Light source, 201 Camera, 301 Sensor (upper sensor), 401 Sensor (lower sensor), 500 to 502 Transport conveyor, 611 Upper shape acquisition unit, 612 Upper shape comparison unit, 621 Lower shape acquisition unit, 622 Lower shape comparison unit, 631 Transmitted light processing unit, 632 Defect detection unit, 641 Cross section density calculation unit, 651 Output unit, 660 Memory unit, 661 Learned model memory unit, 701 Learning device, 711 Data acquisition unit, 721 Model generation unit, 722 Reward calculation unit, 723 Function update unit, 761 Inference device, 771 Inference unit, 801 robot, 810 control device, 811 forming trajectory, 821 mold, S inspection object, S1 inspection object after molding, S2 inspection object after defect inspection.

Claims

1. An apparatus for managing defects of an object to be inspected, comprising: an irradiation device that irradiates transmitted light onto a first surface of the object to be inspected; an imaging device that is disposed opposite to the irradiation device and images a second surface of the object to be inspected on the side opposite to the first surface; a sensor that measures information regarding the outer shape of the object to be inspected; a processing device that processes information acquired from the sensor and the imaging device, wherein the processing device calculates the luminance of the transmitted light in each of a plurality of regions included in the object to be inspected based on the information acquired from the imaging device, each of the plurality of regions being a three-dimensional region between a two-dimensional minute first region included in the first surface and a two-dimensional minute second region included in the second surface, the second region being a region where the transmitted light irradiated onto the first region reaches through the object to be inspected, the processing device estimates the shape of the first surface and the shape of the second surface based on the information acquired from the sensor, and estimates volume information corresponding to the volume obtained by excluding void portions in each of the plurality of regions based on the shape of the first surface, the shape of the second surface, and the luminance of the transmitted light in each of the plurality of regions.

2. The sensor includes a first sensor that measures the first surface and a second sensor that measures the second surface, and the processing device estimates the shape of the first surface based on the information acquired from the first sensor, and estimates the shape of the second surface based on the information acquired from the second sensor. The apparatus according to claim 1.

3. The processing device detects voids in the object to be inspected based on the volume information, and determines that the object to be inspected has a defect when voids of a predetermined volume or more are detected. The apparatus according to claim 1 or claim 2.

4. An apparatus for managing defects of an object to be inspected, comprising: a data acquisition unit that acquires learning data including volume information corresponding to the volume obtained by excluding void portions in each of a plurality of regions included in the object to be inspected and injection path information of the object to be inspected; a model generation unit that generates a learned model for inferring the injection path information from the volume information using the learning data, wherein the injection path information is information including an injection path along which a robot that injects the material of the object to be inspected into a mold for forming the object to be inspected moves while injecting the material of the object to be inspected. The apparatus, wherein the volume information is information at each injection position on the injection path.

5. The volume information is information indicating the presence or absence of defects and the size of defects at each position in the object under inspection, The apparatus according to claim 4, wherein the injection path information includes the injection path and the injection amount of the material at the injection position on the injection path.

6. The apparatus according to claim 4 or claim 5, wherein the model generation unit increases the reward given to the learning model when the difference between the reference volume information of the object under inspection determined in advance and the volume information is equal to or less than a predetermined reference.

7. An apparatus for managing defects of an object under inspection, A data acquisition unit that acquires volume information corresponding to the volume obtained by excluding the void portions in each of a plurality of regions included in the object under inspection, An inference unit that outputs the injection path information from the volume information acquired by the data acquisition unit using a learned model for inferring the injection path information of the object under inspection from the volume information, The apparatus, wherein the injection path information includes information on an injection path along which a robot that injects the material of the object under inspection into a mold for molding the object under inspection moves while injecting the material of the object under inspection.

8. A machine learning method for managing defects of an object under inspection, A step of acquiring learning data including volume information corresponding to the volume obtained by excluding the void portions in each of a plurality of regions included in the object under inspection and the injection path information of the object under inspection, A step of generating a learned model for inferring the injection path information from the volume information using the learning data, The injection path information includes information on an injection path along which a robot that injects the material of the object under inspection into a mold for molding the object under inspection moves while injecting the material of the object under inspection, The machine learning method, wherein the volume information is information at each injection position on the injection path.

9. An acquisition step of acquiring learning data including volume information corresponding to the volume obtained by excluding the void portions in each of a plurality of regions included in the object under inspection and the injection path information of the object under inspection, A generation step of generating a learned model for inferring the injection path information from the volume information using the learning data, The injection path information is information including an injection path along which a robot that injects a material of the object to be inspected into a mold for molding the object to be inspected moves while injecting the material of the object to be inspected. The volume information is information at each injection position on the injection path, and a method for manufacturing a learned model.