Appearance inspection support device, appearance inspection support method, and appearance inspection support program
The visual inspection support device enhances inspection quality by analyzing defect trends and projecting warning images, addressing the reliance on individual proficiency and automation challenges.
Patent Information
- Application Number
- JP2020169120
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2020-10-06
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2040-10-06
AI Technical Summary
Visual inspection quality is dependent on individual proficiency and deteriorates with inexperienced workers, making complete automation difficult.
A visual inspection support device that analyzes trends in defect areas, identifies high-tendency regions, and projects warning images based on worker proficiency to guide inspectors.
Improves visual inspection quality by reducing defect overlooks, shortening inspection time, and enhancing worker proficiency, while reducing processing load and production losses.
Smart Images

Figure 0007584268000001 
Figure 0007584268000002 
Figure 0007584268000003
Abstract
Description
[Technical field]
[0001] The present disclosure relates to an appearance inspection support device, an appearance inspection support method, and an appearance inspection support program.
[0002] There are known techniques for supporting visual inspections that check for defects in products. For example, Patent Document 1 discloses a method for supporting visual inspections by using a visual inspection device that irradiates light onto a part to be inspected and inspects for defects based on the light reflected from the part to be inspected. The visual inspection device disclosed in Patent Document 1 judges the presence or absence of defects such as scratches and color unevenness in the inspected product based on an image of the light reflected specularly from the part to be inspected and an image of the light reflected diffusely from the part to be inspected. [Prior art documents] [Patent documents]
[0003] [Patent Document 1] JP 2007-271510 A Summary of the Invention [Problem to be solved by the invention]
[0004] However, it is difficult to completely automate visual inspection, and visual inspection remains important. On the other hand, visual inspection has the problem that the quality of the inspection depends on the individual, and the quality of the inspection is likely to decrease when performed by an inexperienced worker.
[0005] The present disclosure has been made in consideration of the above problems, and aims to improve the quality of visual appearance inspection. [Means for solving the problem]
[0006] In order to achieve the above object, the visual inspection support device disclosed herein supports visual inspection performed on an inspected product. The visual inspection support device includes an input section, an analysis section, and an output section. Defective part data indicating defective parts of defective products in which defects have been found among the inspected products is input to the input section. The analysis section analyzes the tendency of areas including defective parts based on the defective part data of multiple defective products, and identifies areas where the tendency for defects to occur is above a standard as areas with a high tendency for defects to occur. The output section Depending on the level of proficiency of the worker in charge of visual inspection, Output information indicating areas that are highly prone to defects. Effect of the Invention
[0007] According to the present disclosure, a tendency of an area where defects occur is analyzed, Depending on the level of proficiency of the worker in charge of visual inspection, The information indicating the areas where defects are likely to occur is output, which leads to improved quality of the visual inspection. [Brief description of the drawings]
[0008] [Figure 1] FIG. 1 is a diagram showing an overview of a visual inspection support device according to a first embodiment. [Figure 2A] FIG. 1 is a diagram showing an example of a specific region according to the first embodiment; [Figure 2B] FIG. 13 is a diagram showing an example of a warning image displayed on an inspected product according to the first embodiment; [Diagram 3] FIG. 1 is a block diagram showing a configuration of a visual inspection support device according to a first embodiment. [Figure 4] A block diagram showing the functions of a control device according to a first embodiment. [Diagram 5] FIG. 1 is a diagram showing an example of a production schedule storage unit according to the first embodiment. [Figure 6] FIG. 1 is a diagram illustrating an example of a device master storage unit according to the first embodiment. [Figure 7] FIG. 1 is a diagram showing an example of a non-defective product image related data storage unit according to the first embodiment; [Figure 8] FIG. 1 is a diagram showing an example of a defective product image related data storage unit according to the first embodiment; [Figure 9] FIG. 1 is a diagram showing an example of an output preparation data storage unit according to the first embodiment; [Figure 10] FIG. 1 is a flow diagram showing input processing according to the first embodiment; [Figure 11A] FIG. 1 is a diagram showing an example of a defective product image according to the first embodiment; [Figure 11B] FIG. 1 is a diagram showing an example of a non-defective product image according to the first embodiment; [Figure 12] FIG. 1 is a flow diagram showing an analysis process according to the first embodiment. [Figure 13] FIG. 1 is a flow diagram showing an output process according to the first embodiment. [Figure 14] FIG. 11 is a flow diagram showing an analysis process according to the second embodiment. [Figure 15] FIG. 13 is a diagram showing an example of an output preparation data storage unit according to the second embodiment; [Figure 16] A block diagram showing the functions of a control device according to a third embodiment. [Figure 17] FIG. 13 is a diagram showing an example of a schedule storage unit according to the third embodiment. [Figure 18] FIG. 13 is a diagram showing an example of a worker master storage unit according to the third embodiment; [Figure 19] FIG. 11 is a flow diagram showing an input process according to the third embodiment. [Figure 20] FIG. 13 is a diagram showing an example of a defective product image related data storage unit according to the third embodiment; [Figure 21] FIG. 11 is a flow diagram showing an output process according to the third embodiment. [Figure 22] FIG. 13 is a diagram showing an example of an output preparation data storage unit according to the third embodiment; [Figure 23] FIG. 13 is a diagram showing an example of an attention-calling image according to the third embodiment. [Figure 24] FIG. 13 is a diagram showing another example of an attention-calling image according to the third embodiment. [Diagram 25] FIG. 13 is a flow diagram showing an input process according to the fourth embodiment. [Figure 26] FIG. 13 is a diagram showing an example of a defective product image related data storage unit according to the fourth embodiment; [Figure 27] FIG. 13 is a diagram showing an example of an output preparation data storage unit according to the fourth embodiment; [Figure 28] FIG. 13 is a flow diagram showing an output process according to the fourth embodiment. [Figure 29]A block diagram showing the functions of a control device according to a fifth embodiment. [Diagram 30] FIG. 13 is a flow diagram showing a learning process according to the fifth embodiment. [Diagram 31] FIG. 13 is a flow diagram showing an inference process according to the fifth embodiment. [Diagram 32] FIG. 13 is a diagram showing an example of a remedial action data storage unit according to the fifth embodiment. [Diagram 33] FIG. 23 is a diagram showing another example of the improvement data storage unit according to the fifth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0009] Hereinafter, embodiments of an appearance inspection support device, an appearance inspection support method, and an appearance inspection support program according to the present disclosure will be described with reference to the drawings.
[0010] [Embodiment 1] An overview of the visual inspection support device 100 will be described with reference to FIG. 1. As shown in FIG. 1, the visual inspection support device 100 supports visual inspection performed on a product PR produced in a factory. In the visual inspection, an operator P visually judges whether the product PR has defects such as scratches or uneven color. Hereinafter, the product PR that is the subject of the visual inspection is referred to as an "inspected product W," and an inspected product W in which a defect is found in the visual inspection is referred to as a "defective product D." The product PR is, for example, an electrical appliance such as a dehumidifier, a rice cooker, or a vacuum cleaner.
[0011] The visual inspection support device 100 statistically analyzes the tendency of areas including the locations where defects have occurred, identifies areas where the probability of defects occurring is equal to or higher than a standard, and presents the identified areas to the operator P. Hereinafter, the position where the defective part of the defective product D exists will be referred to as the "defect position," and the area including the defective part will be referred to as the "defect area." Additionally, an area that is highly prone to defects occurring, i.e., an area identified as an area with a high probability of defects occurring, may be referred to as the "identified area."
[0012] The visual inspection support device 100 includes a camera 1 that photographs a defective product D and generates image data of the defective product D, a control device 10 that analyzes areas including defect locations based on the image data of multiple defective products D input from the camera 1 and identifies areas where defects are likely to occur, and a projector 2 that projects information indicating the areas identified by the control device 10 as an image onto the inspected product W.
[0013] The camera 1 is installed in a place where it can photograph the defective products D, such as on the ceiling or wall of the defective product storage area SD.
[0014] When an operator P determines that an inspected product W is a defective product D, he moves the defective product D from the appearance inspection area SV where the appearance inspection is carried out to the defective product storage area SD. In the following, it is assumed that the appearance shape of the inspected product W is a rectangular parallelepiped. Also, it is assumed that the appearance inspection is carried out at a different appearance inspection area SV for each of the six sides of the defective product D, and that a different operator P is in charge of the inspection. One camera 1 is placed in each defective product storage area SD. That is, the number of cameras 1 is equivalent to the number of defective product storage areas SD, six in this embodiment.
[0015] Every time the worker P moves the defective product D to the defective product storage area SD, the worker P causes the camera 1 to photograph the defective product D. Hereinafter, an image showing the defective product D will be referred to as a "defective product image."
[0016] The camera 1 captures an image of the defective product D and outputs the captured image data to the control device 10. The data of the defective product image is an example of "defective part data."
[0017] The control device 10 is configured, for example, by a personal computer, a tablet terminal, a PLC (Programmable Logic Controller), or the like.
[0018] The control device 10 analyzes the tendency of defective areas based on the imaging data output from the camera 1, and identifies areas that are highly likely to cause defects.
[0019] An area where defects are likely to occur is an area where the number of defects that occur in a period preset by a user is equal to or greater than a first defect threshold T1. The first defect threshold T1 is preset by a user of the visual inspection support device 100, and may be a value such as "3," "5," or "10." The user may be a production manager, a production engineer, or a defect analyst. The first defect threshold T1 is an example of a standard.
[0020] When the control device 10 identifies an area where a defect is likely to occur as a specific area, the control device 10 generates data of an image indicating the specific area and outputs the data to the projector 2.
[0021] The number of projectors 2 installed corresponds to the number of visual inspection sites SV, six in this embodiment. Based on the input image data, the projector 2 projects an image indicating an area where defects are likely to occur toward the inspected product W, thereby displaying an image on the surface of the inspected product W. Hereinafter, the display of an image on the surface of the inspected product W by the projector 2 projecting an image toward the inspected product W may be simply referred to as "displaying an image." The image displayed on the inspected product W is referred to as a "warning image." The data of the warning image is an example of "information indicating an area where defects are likely to occur."
[0022] The inspected product W is placed in a predetermined position and in a predetermined orientation on the inspection table of each visual inspection station SV. More specifically, the inspected product W is placed in a predetermined position and in a predetermined orientation on the inspection table of the visual inspection station SV in a form in which the surface of the inspected product can display a warning image. The projector 2 is installed in a position where the warning image can be displayed on the surface of the inspected product W that is the inspected product.
[0023] 2A and 2B, the attention drawing image GC displayed on the inspected product W will be described. FIG. 2A is a diagram showing an example of a specific area R1 identified by the control device 10 as an area with a high tendency for defects to occur, and FIG. 2B is a diagram showing an example of an attention drawing image GC displayed on the inspected product W. Note that the X-axis and Y-axis shown in FIGS. 2A and 2B are coordinate axes on the inspection table on which the inspected product W is placed. The X-axis is the left-right direction of the inspected product W as seen by the operator P, and the Y-axis is the front-back direction of the inspected product W as seen by the operator P, and is perpendicular to the X-axis.
[0024] As shown in Fig. 2A, when the specific region R1 is near a corner on the +X side and the -Y side of the product PR, the projector 2 described with reference to Fig. 1 projects an image GC1 surrounding the corner portion on the +X side and the -Y side of the inspected product W as shown in Fig. 2B. As a result, an attention drawing image GC including an image GC1 pointing out the specific region R1 is displayed on the inspected product W. Note that, when there are multiple specific regions R1, the attention drawing image GC includes an image surrounding the multiple specific regions R1.
[0025] Next, the configuration of the control device 10 shown in FIG. 1 will be described in detail with reference to FIG. 3 and FIG.
[0026] 3, the control device 10 further includes an operation unit 11 that accepts operations by a user, a display 12 that displays images, an input unit 13 that receives image data from the multiple cameras 1, an output unit 14 that outputs data of an attention-calling image GC, a processor 15 that executes various processes, a main memory device 16a that is used as a working area for the processor 15, and an auxiliary memory device 16b that stores various data such as programs. The main memory device 16a includes a random access memory (RAM). The auxiliary memory device 16b includes a non-volatile memory such as a flash memory, a hard disk drive (HDD), and a solid state drive (SDD).
[0027] When the operation unit 11 receives an operation by a user, it outputs a signal corresponding to the operation to the processor 15. The operation unit 11 is, for example, a touch sensor, a keyboard, a mouse, or the like.
[0028] The display 12 displays an image based on a signal output from the processor 15. The display 12 is, for example, a liquid crystal display, an organic EL (Organic Electro Luminescence) display, or the like.
[0029] The input unit 13 is connected to each of the multiple cameras 1, and receives as input the imaging data output from each camera 1. The input unit 13 may be any device capable of receiving imaging data, such as a video card or a graphics card. Note that the input unit 13 may be a communication device that performs communication in accordance with a communication standard such as Ethernet or USB (Universal Serial Bus).
[0030] The output unit 14 is connected to each of the multiple projectors 2, and outputs data of the attention calling image GC to each projector 2. The output unit 14 may be any device capable of outputting data of the attention calling image GC to the projector 2, such as a video card or a graphic card. The output unit 14 may be a communication device that performs communication in accordance with a communication standard such as Ethernet or USB.
[0031] The processor 15 is a micro processing unit (MPU), a central processing unit (CPU), etc. The processor 15 reads out an appearance inspection support program stored in the auxiliary storage device 16b into the main storage device 16a and executes it.
[0032] The auxiliary storage device 16b includes the data storage unit 16 shown in Fig. 4. The data storage unit 16 stores data on a plurality of defective products D. The data on the plurality of defective products D is used when analyzing the tendency of the area in which the defect occurs.
[0033] The data storage unit 16 includes an image data storage unit 160 that stores data on defective product images.
[0034] The image data storage unit 160 further stores data of a non-defective product image previously photographed by a user. The non-defective product image data is used when identifying defective areas. After photographing a non-defective product image, the user operates the operation unit 11 to input the non-defective product image data to the control device 10 via the input unit 13, and the non-defective product image data is stored in the image data storage unit 160. The non-defective product images include images of the entire surface for each model.
[0035] The data storage unit 16 also includes a production schedule storage unit 161 that stores data indicating the production schedule of the factory, an equipment master storage unit 162 that stores data related to the equipment equipped in the appearance inspection support device 100, a good product image related data storage unit 163 that stores data related to good product images, a defective product image related data storage unit 164 that stores data related to defective product images captured by the camera 1, and an output preparation data storage unit 165 that stores output preparation data required for output to the projector 2 shown in FIG. 1.
[0036] 5, the production schedule storage unit 161 stores model data 161a indicating the model of the product PR produced in the factory and production schedule data 161b indicating the schedule for producing the product PR of that model in an associated manner. The production schedule data 161b includes production start date and time data 161c indicating the date and time when production starts and production end date and time data 161d indicating the date and time when production ends. The data stored in the production schedule storage unit 161 is registered in advance by the user.
[0037] As shown in FIG. 6, the equipment master storage unit 162 shown in FIG. 4 stores equipment name data 162a indicating the equipment name of the equipment included in the visual inspection support device 100, equipment identification name data 162b indicating the identification name of the equipment, installation location data 162c indicating the location where the equipment is installed, and inspection content data 162d indicating the content of the visual inspection corresponding to the location where the equipment is installed, in association with each other. The equipment identification name data 162b is, for example, identification data such as the MAC address or IP address of the equipment. The installation location data 162c indicates the installation location according to the function of the equipment. The installation location may be, for example, a defective product storage area SD or a visual inspection area SV. The inspection content data 162d indicates the surface to be inspected for the visual inspection at the installation location. The data stored in the equipment master storage unit 162 is registered in advance by a user.
[0038] The non-defective product image related data storage unit 163 shown in Fig. 4 stores non-defective product image identification name data 163a for identifying a non-defective product image, model data 163b for indicating the model of the non-defective product indicated by the non-defective product image, and photographed part data 163c for indicating the photographed part of the non-defective product image in association with each other, as shown in Fig. 7. The data relating to the non-defective product image stored in the non-defective product image related data storage unit 163 is registered in advance in the non-defective product image related data storage unit 163 by a user.
[0039] The defective product image related data storage unit 164 shown in Fig. 4 stores, in association with each other, defective product image identification name data 164a for identifying a defective product image, model data 164b for indicating the model of the defective product D indicated by the defective product image, photographed portion data 164c for indicating the photographed portion of the defective product D, discovery date and time data 164d for indicating the date and time when the defect was discovered, and defective area data 164e for indicating the defective area, as shown in Fig. 8. The defective product image related data is stored in the defective product image related data storage unit 164 by the input processing unit 151 shown in Fig. 4 executing the input process.
[0040] The output preparation data memory unit 165 shown in Figure 4 stores, in association with each other, model data 165a indicating the model of the inspected product W, photographed portion data 165b indicating the portion of the inspected product W where the warning image GC is displayed, and specific area data 165c indicating the specific area R1 pointed to by the warning image GC, as shown in Figure 9.
[0041] The processor 15 shown in FIG. 3 functions as an input processing unit 151 that executes input processing, an analysis unit 152 that executes analysis processing, and an output processing unit 153 that executes output processing, as shown in FIG.
[0042] When the image pickup data is input from the camera 1 via the input unit 13, the input processing unit 151 executes an input process to acquire data necessary for analyzing a defective area and store the data in the data storage unit 16.
[0043] When image data is input from camera 1 and data for analyzing the tendency of defective areas is stored in image data storage unit 160, analysis unit 152 executes an analysis process that analyzes the tendency of the defective areas based on the data stored in image data storage unit 160 and identifies areas that are highly prone to defects.
[0044] The output processing unit 153 generates data of an attention drawing image GC indicating an area identified by the analysis unit 152 that is likely to cause a defect, and executes an output process of outputting the data of the attention drawing image GC to the projector 2.
[0045] The input process, the analysis process, and the output process will be described below with reference to Fig. 10 to Fig. 13. Note that the input process is an example of an input step, the analysis process is an example of an analysis step, and the output process is an example of an output step.
[0046] In the factory, when the product W to be inspected is transported to the appearance inspection site SV that the worker P is in charge of, the worker P places the product W on the inspection table. At this time, the worker P places the product W to be inspected at a predetermined position on the inspection table. More specifically, the worker P places the product W to be inspected at a predetermined position on the inspection table in a predetermined orientation. The worker P places the product W on the inspection table with the inspection area at the appearance inspection site SV, in the first embodiment, one of the six faces, facing up.
[0047] Next, the operator P visually inspects the surface of the inspected product W. At this time, a projector 2 installed at the appearance inspection site SV projects an attention-grabbing image GC onto the inspected product W. This indicates areas on the surface of the inspected product W that are likely to cause defects, drawing the operator P's attention.
[0048] When the worker P finds a defect, he / she moves the inspected product W to the defective product storage area SD as a defective product D. In this embodiment, the worker P places the inspected product W on the work bench of the defective product storage area SD with the surface including the defect of the inspected product W facing upward.
[0049] Next, the worker P operates the operation unit 11 to input an instruction to the control device 10 to have the camera 1 capture an image of the defective product D. In response to the signal from the operation unit 11, the processor 15 causes the camera 1 to capture an image of the surface of the defective product D that includes the defective portion. As a result, captured image data is generated and output to the control device 10.
[0050] Next, the processor 15, functioning as the input processing unit 151, starts the input processing shown in Fig. 10. The input processing unit 151 captures data of the defective product image from the camera 1 via the input unit 13, and stores the captured data of the defective product image in the image data storage unit 160 shown in Fig. 4 (step S102 in Fig. 10).
[0051] Next, the input processing unit 151 acquires data for identifying the defective image (step S104). The data for identifying the defective image is included in the captured image data input from the camera 1, and is, for example, data indicating the file name and storage destination of the defective image.
[0052] Next, the input processing unit 151 identifies the model of the defective product D based on the shooting time included in the imaging data (step S106). In detail, the input processing unit 151 searches the production schedule storage unit 161 shown in Fig. 5 for records whose shooting time data is after the date and time indicated by the production start date and time data 161c and before the date and time indicated by the production end date and time data 161d, using the shooting time data when the defective product image was captured as a search condition. When the input processing unit 151 finds a record that matches the search condition, it acquires the model data 161a of the record to identify the model of the defective product D.
[0053] Next, the input processing unit 151 identifies the photographed surface of the defective product D (step S108 in FIG. 10). Hereinafter, the photographed surface is referred to as the "photographed portion."
[0054] Specifically, the input processing unit 151 uses data indicating the identification name of the camera 1 included in the captured image data as a search criterion to search for a record in which the data indicating the identification name of the camera 1 matches the device identification name data 162b from the device master storage unit 162 shown in Fig. 6. When the input processing unit 151 finds a record that matches the search criterion, it acquires the inspection content data 162d of that record to identify the photographed portion of the defective product D.
[0055] Next, the input processing unit 151 identifies a defective area indicated by the defective image (step S110 in FIG. 10). More specifically, the input processing unit 151 searches for a record whose model data 163b matches the model data 161a acquired in step S106 and whose photographed portion data 163c matches the inspection content data 162d acquired in step S108 from the data stored in the non-defective image related data storage unit 163 shown in FIG. 7. When the input processing unit 151 finds a record that matches the search conditions, it uses the non-defective image identification name data 163a of the found record as a search condition to acquire non-defective image data from the data stored in the image data storage unit 160. When the input processing unit 151 acquires the non-defective image data, it identifies a defective area by comparing the non-defective image with the non-defective image.
[0056] A method for identifying a defective area will be described below with reference to Figures 11A and 11B. Figure 11A is a diagram showing an example of a defective product image GD, and Figure 11B is a diagram showing an example of a non-defective product image GR.
[0057] The input processing unit 151 divides each of the defective image GD and the non-defective image GR into nine sections SC, as shown in Fig. 11A and Fig. 11B. In Fig. 11A and Fig. 11B, for ease of understanding, the boundaries of each section SC are indicated by two-dot chain lines. Note that the number of sections SC to be divided is not limited to nine and is set in advance by the user.
[0058] In the following, the section SC on the -X side and the +Y side will be called "SECTION1", and the sections from "SECTION1" to the +X side will be called "SECTION2" and "SECTION3", respectively. The section SC adjacent to the -Y side of "SECTION1" will be called "SECTION4", and the sections from "SECTION4" to the +X side will be called "SECTION5" and "SECTION6". Furthermore, the section SC adjacent to the -Y side of "SECTION4" will be called "SECTION7", and the sections from "SECTION7" to the +X side will be called "SECTION8" and "SECTION9", respectively.
[0059] When the number of sections SC to be divided is set, for example, data indicating the position of each section SC is calculated by the processor 15 and stored in the auxiliary storage device 16b as coordinate data corresponding to each section SC.
[0060] The input processing unit 151 matches the defective product image GD with the non-defective product image GR for each section SC, and identifies a section SC where the defective product image GD and the non-defective product image GR differ, that is, an area including a defect, as a defective area.
[0061] 11A and 11B, "SECTION 9" of the defective image GD includes an image GS showing a scratch. Therefore, the input processing unit 151 determines that the image of "SECTION 9" of the defective image GD does not match the image of "SECTION 9" of the non-defective image GR, and identifies "SECTION 9" as a defective area. The input processing unit 151 generates data showing the identified defective area.
[0062] When the input processing unit 151 identifies the defective area, it associates the data acquired in steps S104 to S110 in Fig. 10 and stores them in the defective image related data storage unit 164 shown in Fig. 8 (step S112). In detail, the input processing unit 151 stores the data acquired in steps S104 to S110 in the defective image related data storage unit 164 as "defective image identification name data 164a", "model data 164b", "photographed portion data 164c", and "defective area data 164e", respectively. In addition, the input processing unit 151 stores photographing time data indicating the time when the defective image GD was photographed in the defective image related data storage unit 164 as "discovery date and time data 164d".
[0063] With this, the input processing unit 151 ends the input processing.
[0064] Next, the analysis process will be described with reference to Fig. 12. The analysis process is executed, for example, every time the factory closes. Note that the timing at which the analysis process is executed is set in advance by the user.
[0065] At a preset start time, the processor 15, functioning as the analysis unit 152, starts the analysis process shown in FIG.
[0066] The analysis unit 152 counts up data on images of multiple defective products D (defective product images GD) stored in the defective product image related data storage unit 164 shown in Fig. 8 for each model, photographed part, and defective area, and obtains the number of counted records as the number of occurrences of defects (step S202 in Fig. 12). Note that the number of occurrences of defects is the number of occurrences of defects. The analysis unit 152 counts up the number of occurrences of defects based on data on defective product images GD stored during a preset period, such as one day or one week.
[0067] The analysis unit 152 identifies an area where defects are likely to occur based on the number of occurrences of defects (step S204). The analysis unit 152 identifies a defect area where the number of occurrences of defects, which is the result of the counting in step S202, is equal to or greater than a first defect threshold T1 as an area where defects are likely to occur, that is, a specific area R1. Note that the first defect threshold T1 is preset by the user as a value according to the period for which the number of occurrences of defects is counted.
[0068] For example, when the period set by the user is one day and the first defect threshold T1 is set to "3", if the data in the defective product image related data storage unit 164 shown in Fig. 8 is tallied by model, photographed portion, and defective area, the number of records for the model "rice cooker", the photographed portion "top surface", and the defective area "SECTION 9" for the one day "August 26, 2020" is 7, which is more than 3. Therefore, the analysis unit 152 identifies the model "rice cooker" and the defective area "SECTION 9" of the photographed portion "top surface" as areas that are likely to cause defects, that is, the specific area R1.
[0069] When the analysis unit 152 identifies the specific area R1, it stores data necessary for output by the output processing unit 153 in the output preparation data storage unit 165 shown in Fig. 9 (step S206 in Fig. 12). In detail, the analysis unit 152 stores the model data 164b, the photographed portion data 164c, and the defective area data 164e of the record extracted from the defective product image related data storage unit 164 in the output preparation data storage unit 165 as "model data 165a", "photographed portion data 165b", and "specific area data 165c".
[0070] Then, the analysis unit 152 ends the analysis process.
[0071] The processor 15 executes output processing as the output processing unit 153, for example, periodically, during the period in which the visual appearance inspection is being performed.
[0072] 13, in order to identify the model of the inspected product W, the output processing unit 153 identifies the model of the production target produced in the last n hours (step S302). In detail, the output processing unit 153 acquires data indicating the current time from a timer. The output processing unit 153 searches the data stored in the production schedule storage unit 161 shown in FIG. 5 using the acquired current time data as a search condition. When the output processing unit 153 finds a record that matches the search condition, it identifies the model indicated by the model data 161a of that record as the model that was produced in the factory in the last n hours and is the target of visual inspection.
[0073] Next, the output processing unit 153 specifies, for each projector 2, the contents of the appearance inspection to be performed at the appearance inspection site SV where the projector 2 is installed (step S304). In detail, the output processing unit 153 searches the device master storage unit 162 shown in FIG. 6 using data indicating the identification name of each projector 2 as a search condition. When the output processing unit 153 finds a record that matches the search condition, it specifies the inspection contents indicated by the inspection content data 162d of that record as the appearance inspection to be performed at the appearance inspection site SV where the projector 2 is installed. Note that the data indicating the identification name of the projector 2 is input by the user via the operation unit 11, for example.
[0074] Next, the output processing unit 153 searches for a record in which the model data 161a acquired in step S302 matches the model data 165a and the examination content data 162d acquired in step S304 matches the photographed portion data 165b from among the data stored in the output preparation data storage unit 165 shown in FIG. 9. When the output processing unit 153 finds a record that matches the search conditions, it acquires the specific area data 165c of the record. The output processing unit 153 generates data of an attention drawing image GC that indicates the specific area R1 indicated by the acquired specific area data 165c. The output processing unit 153 outputs the data of the attention drawing image GC to the corresponding projector 2 via the output unit 14 (step S306), and ends the output process. Note that in the output process, the output processing unit 153 generates data of an attention drawing image GC for each projector 2 and outputs it to the corresponding projector 2.
[0075] When data of the attention warning image GC is input from the output unit 14, the projector 2 displays the attention warning image GC pointing to the specific area R1 on the surface of the inspected product W, as shown in Figure 2B, based on the input data of the attention warning image GC.
[0076] The worker P performs a visual appearance inspection while being aware that defects tend to occur in the area indicated by the attention-calling image GC. Note that the output processing unit 153 executes output processing every time n hours have passed or every time the manufactured product PR is changed, and responds to the change in model.
[0077] As described above, according to the visual inspection support device 100 of the first embodiment, the attention drawing image GC indicating the area where defects are likely to occur is displayed on the surface of the inspected product W. Therefore, the operator P can easily grasp the area where defects are likely to occur. This makes it possible to prevent defects from being overlooked even when an operator P with low proficiency is in charge of the visual inspection. As a result, the quality of the visual inspection can be maintained. In other words, the quality of the visual inspection can be improved.
[0078] Furthermore, since the operator P can easily grasp the specific region R1, the time required for the appearance inspection can be shortened, and the appearance inspection can be made more efficient.
[0079] In addition, since the worker P can easily grasp the specific area R1, the worker P can quickly become proficient.
[0080] If the worker P overlooks a defect, the inspected product W will be returned, which is called "rework," and there is a risk of production loss, production time loss, and the like. However, the appearance inspection support device 100 allows the worker P to easily grasp the specific region R1, which prevents the defect from being overlooked. This prevents rework from occurring, and reduces production loss, production time loss, and the like.
[0081] Furthermore, every time the worker P moves a defective product D to the defective product storage area SD, the worker P causes the camera 1 to photograph the defective product D. That is, the worker P causes the camera 1 to photograph all defective products D. As a result, the appearance inspection support device 100 can collect data on all defective products D in which defects have occurred and statistically analyze it. Therefore, the reliability of the analysis results can be improved compared to a case in which data on all defects that have occurred cannot be analyzed due to, for example, a manual analysis of the tendency of defects.
[0082] Furthermore, the visual inspection support device 100 only needs to identify the specific area R1 for the inspected product W where defects exist. Therefore, the processing burden on the visual inspection support device 100 can be reduced compared to a case where visual inspection is fully automated. By reducing the processing burden on the visual inspection support device 100, it is possible to speed up the visual inspection judgment. Alternatively, the quality of identifying the specific area R1 can be improved.
[0083] [Modification of the first embodiment] In the first embodiment, the worker P photographs the defective product D by operating the operation unit 11, but the person who photographs the defective product D is not limited to the worker P and may be, for example, a production manager, an analyst, etc.
[0084] In addition, in the first embodiment, the defective product D is photographed every time it is placed in the defective product storage area SD, but the defective product D does not have to be photographed every time it is placed in the defective product storage area SD, and for example, the defective product D may be photographed collectively in half-day or daily units. In this case, data indicating the date and time when the defect occurred is input to the control device 10 by the photographer based on the contents recorded in a document such as a defect discovery sheet.
[0085] Furthermore, the timing of photographing the defective product D is not limited to being determined by a person such as a worker P, a production manager, or an analyst, but may be determined by the camera 1. For example, the camera 1 photographs the defective product D when it detects that the defective product D has been placed in the defective product storage area SD using image recognition technology. Alternatively, the camera 1 may photograph the defective product D when it receives a signal indicating that the defective product D has been detected from a sensor that detects the presence or absence of the defective product D installed in the defective product storage area SD. Note that the sensor may be a reflective sensor, a transmissive sensor, or the like, as long as it is capable of detecting the presence or absence of the defective product D.
[0086] In addition, the defective areas were identified by dividing the defective image GD and the good image GR into nine sections SC each and comparing them, but they may also be identified by dividing the defective image GD and the good image GR into parts, such as the handle portion, the lid portion, etc., and comparing them.
[0087] Furthermore, the defective area may be identified using a machine learning technique. For example, the input processing unit 151 generates a trained model that is trained using teacher data in which data indicating the defective area is added to data of the defective image GD. The input processing unit 151 may input the data of the defective image GD as an input parameter to the generated trained model, and acquire data indicating the defective area that is output as an output parameter, thereby identifying the defective area.
[0088] Furthermore, the data stored in the image data storage unit 160 can be changed as appropriate. For example, the image data storage unit 160 can omit the output preparation data storage unit 165. In this case, the analysis unit 152 may use a flag to distinguish the specific region R1 from other defective regions. The flag is further stored in the defective product image related data storage unit 164 in association with the specific region R1. The output processing unit 153 may generate image data indicating the specific region R1 based on the flag stored in the defective product image related data storage unit 164 and output the image data to the projector 2.
[0089] [Embodiment 2] 14 and 15, the visual inspection support device 100 according to the second embodiment will be described. The basic configuration and basic operation of the visual inspection support device 100 according to the second embodiment are similar to those of the visual inspection support device 100 according to the first embodiment. However, it differs from the first embodiment in that the weight of a defect is set based on the number of defects found, and that the user can specify the weight of the specific region R1 to be output to the projector 2. The following description will focus on the differences from the first embodiment.
[0090] In the analysis process, the analysis unit 152 sets a weight for each specific region R1 based on the number of defects occurring in a period set by the user. The weight is a value that evaluates the importance of the specific region R1, and the higher the weight value, the higher the importance of the specific region R1.
[0091] As shown in FIG. 14, the analysis unit 152 aggregates data related to defective product images GD stored in the defective product image related data storage unit 164 shown in FIG. 8 by model, photographed part, and defective area (step S202), and sets a weight for the specific area R1 based on the aggregated number of records, i.e., the number of defects for each model, photographed part, and defective area (step S402).
[0092] For example, the analysis unit 152 sets the weight of a specific region R1 where the number of defect occurrences in a period preset by the user is between 3 and 5 to "1," and sets the weight of a specific region R1 where the number of defect occurrences is 5 or more to "2." Note that the weight is not limited to 2 levels, and can be preset by the user in 5 levels, 10 levels, or the like. Also, a weight equal to or greater than a preset weight reference value, for example "2," is referred to as "high," and a weight less than the weight reference value, for example "1," is referred to as "low."
[0093] In the example shown in FIG. 8, the number of records for the model "rice cooker", the photographed portion "top surface", and the defective area "SECTION 9" for the one day "August 26, 2020" is 7. Therefore, the analysis unit 152 sets the weight of the specific area R1 "SECTION 9" to "2". After setting the weight, the analysis unit 152 generates data indicating the set weight.
[0094] The analysis unit 152 further stores the data indicating the generated weights as "weight data 165d" in the output preparation data storage unit 165 shown in FIG. 15 (step S206 in FIG. 14), and ends the analysis process.
[0095] The output preparation data storage unit 165 shown in FIG. 15 stores weight data 165d in addition to model data 165a, photographed portion data 165b, and specific area data 165c in association with each other.
[0096] In the output process, the output processing unit 153 generates data of an attention drawing image GC indicating a specific region R1 of a weight designated by a user as an output target, and outputs the data to the projector 2 via the output unit 14. The user can designate a weight by operating the operation unit 11. When the output processing unit 153 receives data indicating a weight designated from the operation unit 11, the output processing unit 153 generates data indicating the weight. The generated data indicating the weight is stored in, for example, the auxiliary storage device 16b. The output processing unit 153 generates data of the attention drawing image GC by referring to the data indicating the weight stored in the auxiliary storage device 16b.
[0097] Specifically, in step S306 of the output process shown in Fig. 13, the output processing unit 153 searches for a record including data indicating the specified weighting, in addition to the data acquired in steps S302 and S304, from among the data stored in the output preparation data storage unit 165 shown in Fig. 15. Specifically, the output processing unit 153 searches for a record in which the model data 161a acquired in step S302 matches the model data 165a, the examination content data 162d acquired in step S304 matches the photographed portion data 165b, and the weighting data 165d indicating the specified weighting, from among the data stored in the output preparation data storage unit 165.
[0098] When the output processing unit 153 finds a record that matches the search conditions, it generates data of an attention drawing image GC that indicates the specific area R1 indicated by the specific area data 165c of the record. The output processing unit 153 outputs the data of the attention drawing image GC to the projector 2 via the output unit 14 (step S306 in FIG. 13), and ends the output process.
[0099] For example, when the weight "2" is specified, as shown in Fig. 2B, an attention drawing image GC pointing to "SECTION 9" is displayed on the inspected product W. Through the above process, an attention drawing image GC pointing to the specific region R1 to which the weight specified by the user is set is displayed on the inspected product W.
[0100] As described above, according to the visual inspection support device 100 of the second embodiment, it is possible to display the attention drawing image GC to which a specified weight is set on the inspected product W. Therefore, it is possible to improve the convenience for the operator P and to improve the quality of the visual inspection.
[0101] [Modification of the second embodiment] In addition, the output processing unit 153 may change attributes such as color, brightness, thickness, etc. of the image contained in the attention warning image GC according to the weight, so as to allow the worker P to easily understand the weight, i.e., the importance, of the specific region R1.
[0102] [Embodiment 3] A visual inspection support device 100 according to a third embodiment will be described with reference to Fig. 16 to Fig. 24. The third embodiment differs from the first and second embodiments in that an attention-calling image GC displayed on the inspected product W changes depending on the proficiency level of an operator P in charge of the visual inspection. The following description will focus on the differences from the first and second embodiments.
[0103] 16 further includes data necessary for identifying the proficiency of the worker P in charge of the visual inspection. The data storage unit 16 further includes a work schedule storage unit 166 that stores data related to the work schedule of the worker P in charge of the visual inspection, and a worker master storage unit 167 that stores data related to the proficiency of the worker P.
[0104] 17 stores data 166a of a person in charge of visual inspection, data 166b of the schedule of the visual inspection for which the worker P is in charge, and inspection content data 166e of the visual inspection for which the worker P is in charge, in association with each other. The schedule data 166b includes data 166c of the start of the visual inspection and data 166d of the end of the visual inspection. The data on the schedule is registered in advance in the schedule storage unit 166 by, for example, a user.
[0105] The worker master storage unit 167 shown in FIG. 18 stores worker data 167a indicating a worker P in charge of visual inspection and proficiency data 167b indicating the proficiency of the worker P in association with each other. In the proficiency data 167b, a value evaluated on a three-level scale ("1" to "3") is set based on the worker P's years of service, experience, technical ability, and the like. The higher the proficiency value, the higher the proficiency of the worker P. The evaluation of the proficiency is not limited to three levels, and may be preset by the user to five levels, ten levels, or the like. Furthermore, a proficiency level equal to or higher than a predetermined proficiency standard value, for example, "2" or higher, is referred to as "high," and a proficiency level lower than the proficiency standard value, for example, "1," is referred to as "low."
[0106] 19, when the input processing unit 151 identifies a defective area (step S110), it identifies the worker P who discovered the defect (step S502). The input processing unit 151 searches for a record matching the search conditions from among the data stored in the duty schedule storage unit 166 shown in FIG. 17, using the inspection content data 162d acquired in step S108 and the photography time data as search conditions.
[0107] In detail, the input processing unit 151 searches the data stored in the assigned schedule storage unit 166 for records in which the inspection content data 162d acquired in step S108 of Fig. 19 matches the inspection content data 166e and the photography time data is after the assigned start date and time data 166c and before the assigned end date and time data 166d. When the input processing unit 151 finds a record that matches the search conditions, it acquires the person in charge data 166a of that record to identify the worker P who discovered the defect.
[0108] When the input processing unit 151 identifies the worker P who discovered the defect, it associates the data acquired in steps S104 to S502 and stores them in the defective product image related data storage unit 164 as shown in Fig. 20 (step S112 in Fig. 19). In detail, the input processing unit 151 further stores the person in charge data 166a acquired in step S502 as "worker data 164f" in the defective product image related data storage unit 164, and ends the input process.
[0109] The defective product image related data storage unit 164 further stores, in association with each other, defective product image identification name data 164a, model data 164b, photographed part data 164c, discovery date and time data 164d, defective area data 164e, and worker data 164f indicating the worker P who discovered the defect.
[0110] In the output process, the output processing unit 153 identifies the worker P in charge of the visual inspection, and outputs data of an attention-calling image GC according to the proficiency of the worker P to the projector 2 via the output unit 14. That is, the output processing unit 153 outputs data of the attention-calling image GC according to the proficiency of the worker P in charge of the visual inspection.
[0111] 21, when the output processing unit 153 specifies the contents of the appearance inspection (step S304), it specifies the worker P in charge of the appearance inspection at the appearance inspection site SV where the projector 2 to be outputted is installed (step S602). The output processing unit 153 searches for data stored in the in-charge schedule storage unit 166 shown in FIG. 17 using the inspection contents data 162d acquired in step S304 and data indicating the current time as search conditions.
[0112] In detail, the output processing unit 153 searches the data stored in the in-charge schedule storage unit 166 for a record in which the inspection content data 162d acquired in step S304 matches the inspection content data 166e, and the shooting time data is after the in-charge start date and time data 166c and before the in-charge end date and time data 166d. When the output processing unit 153 finds a record that matches the search conditions, it acquires the in-charge data 166a of that record, thereby identifying the worker P in charge of the appearance inspection at the appearance inspection site SV where the projector 2 to be outputted is installed.
[0113] Next, the output processing unit 153 identifies the proficiency level of the worker P identified in step S602 in Fig. 21 (step S604). The output processing unit 153 searches for data stored in the worker master storage unit 167 shown in Fig. 18 using the person in charge data 166a acquired in step S602 as a search condition.
[0114] In detail, the output processing unit 153 searches for a record in which the person in charge data 166a acquired in step S602 matches the worker data 167a from among the data stored in the worker master storage unit 167. When the output processing unit 153 finds a record that matches the search conditions, it acquires the skill data 167b of that record to identify the skill of worker P.
[0115] When the output processing unit 153 identifies the proficiency level of the worker P, it obtains specific area data 165c associated with the identified proficiency level of the worker P from the data stored in the output preparation data storage unit 165 shown in FIG. 22.
[0116] The output preparation data storage unit 165 stores the model data 165a, the photographed portion data 165b, the specific area data 165c, and the proficiency data 165e in association with each other. The proficiency data 165e is registered by the user based on the data stored in the defective product image related data storage unit 164 shown in FIG.
[0117] When the output processing unit 153 acquires the specific area data 165c associated with the specified proficiency level of the worker P, the output processing unit 153 generates data of an attention drawing image GC indicating the specific area R1 indicated by the acquired specific area data 165c. The output processing unit 153 outputs the generated data of the attention drawing image GC to the projector 2 via the output unit 14 (step S306 in FIG. 21), and ends the output process.
[0118] When the data of the attention drawing image GC is input from the output processing unit 153, each projector 2 projects the attention drawing image GC onto the surface of the inspection product W based on the input data of the attention drawing image GC.
[0119] 22, if the model of the product to be produced in the most recent n hours is a "rice cooker," the content of the visual inspection performed at the visual inspection site SV where the projector 2 that outputs the data of the attention image GC is installed is a "top surface," and the proficiency level of the worker P in charge of the visual inspection is "1," then, as shown in Fig. 23, an attention image GC including an image GC2 pointing to the specific region R1 "SECTION 1" and an image GC1 pointing to the specific region R1 "SECTION 9" is projected by the projector 2 toward the inspected product W and displayed on the surface of the inspected product W. On the other hand, if the proficiency level of the worker P is "3," an attention image GC including an image GC2 pointing to the specific region R1 "SECTION 1" is displayed on the surface of the inspected product W, as shown in Fig. 24.
[0120] For example, a highly skilled worker P may feel cumbersome if many images pointing to the specific region R1 are displayed on the surface of the inspected product W. On the other hand, a less skilled worker P may overlook defects if images pointing to all of the specific regions R1 are not displayed on the surface of the inspected product W. According to the appearance inspection support device 100 of the third embodiment, the attention drawing image GC displayed on the surface of the inspected product W can be changed according to the proficiency of the worker P. This can improve the convenience of the worker P and can improve the quality of the appearance inspection.
[0121] Furthermore, according to the third embodiment, the number of defective areas found by each worker P can be tallied, so that the user can objectively grasp the level of proficiency of the worker P.
[0122] [Modification of the third embodiment] The order in which the input processes are performed can be changed as appropriate. The process of step S502 shown in Fig. 19 may be performed after step S108 and before step S112, and may be performed between step S108 and step S110, for example.
[0123] [Embodiment 4] A visual inspection support device 100 according to a fourth embodiment will be described with reference to Fig. 25 to Fig. 28. The fourth embodiment differs from the first to third embodiments in that the tendency of defective areas for each time period is analyzed, and the attention calling image GC changes depending on the time period during which the visual inspection is performed. The following description will focus on the differences from the first to third embodiments.
[0124] 25, when the input processing unit 151 identifies a defective area (step S110), it identifies the time period in which the defect was found (step S702). Based on the shooting time data, the input processing unit 151 determines to which of the five time periods "first time period," "second time period," "third time period," "fourth time period," or "others" the time period in which the defect was found belongs.
[0125] The time ranges of the five time periods "first time period", "second time period", "third time period", "fourth time period", and "others" are set in advance by the user. For example, the user sets the "first time period" as "9:00 to 10:00" after the start of work, the "second time period" as "11:00 to 12:00" before lunch, the "third time period" as "13:00 to 14:00" after lunch, the "fourth time period" as "16:00 to 17:00" before the end of work, and the time range other than "first time period" to "fourth time period" as "others". When the input processing unit 151 identifies the time period in which the defective part was found, it generates data indicating the time period.
[0126] Next, the input processing unit 151 associates and stores the data acquired in steps S104 to S702 (step S112). In detail, the input processing unit 151 further stores the data indicating the time period acquired in step S702 as "time period data 164g" in the defective product image related data storage unit 164 shown in FIG. 26, and ends the input process.
[0127] The defective product image related data storage unit 164 shown in FIG. 26 stores defective product image identification name data 164a, model data 164b, photographed portion data 164c, discovery date and time data 164d, defective area data 164e, and in addition, time period data 164g indicating the time period when the defect was discovered, in association with each other.
[0128] In the analysis process, the analysis unit 152 analyzes the number of defects occurring in a defective area for each time period in which defects occurred during a period set by the user. The analysis unit 152 identifies a defective area in which the number of defects occurring for each time period is equal to or greater than a second defect threshold T2 as an area that is likely to have defects occurring in that time period. The second defect threshold T2 may be set to the same value as the first defect threshold T1, or may be set to a different value. The second defect threshold T2 is an example of a criterion.
[0129] More specifically, in step S202 of the analysis process shown in FIG. 12, the analysis unit 152 collects data related to the defective product images GD stored in the defective product image related data storage unit 164 by model, photographed part, defective area, and time period.
[0130] Next, the analysis unit 152 identifies areas that are likely to cause defects for each time period based on the number of records that have been collected, i.e., the number of defects that have occurred (step S204). For example, when the second defect threshold T2 is set to "3", if the data in the defective product image related data storage unit 164 shown in FIG. 26 is collected for one day on "August 26, 2020" by model, photographed part, defective area, and time period, the number of records for the model "rice cooker", the photographed part "top surface", the time period "fourth time period", and the defective area "SECTION 9" is four, which is more than three. Therefore, the analysis unit 152 identifies the model "rice cooker" and the defective area "SECTION 9" of the photographed part "top surface" as areas that are likely to cause defects, i.e., the specific area R1, in the time period "fourth time period".
[0131] When the analysis unit 152 identifies the specific area R1 for each time period, it further stores the data indicating the time period used for the aggregation as "time period data 165f" in the output preparation data storage unit 165 shown in FIG. 27 (step S206 in FIG. 12), and terminates the analysis process.
[0132] The output preparation data storage unit 165 shown in FIG. 27 stores, in addition to the model data 165a, the photographed portion data 165b, and the specific area data 165c, time period data 165f in association with each other.
[0133] In the output process, the output processing unit 153 generates data of an attention drawing image GC including an image indicating a specific area R1 of the time period during which the appearance inspection is performed, and outputs the generated data to the projector 2 via the output unit .
[0134] In the output process shown in FIG. 28, when the output processing unit 153 identifies the content of the appearance inspection (step S304), it identifies to which of the five time periods the most recent hour belongs based on the current time data (step S802).
[0135] Next, the output processing unit 153 acquires the specific area data 165c associated with the specified time period from the data stored in the output preparation data storage unit 165 shown in FIG.
[0136] When the output processing unit 153 acquires the specific area data 165c associated with the identified time period, the output processing unit 153 generates data of an attention drawing image GC indicating the specific area R1 indicated by the acquired specific area data 165c. The output processing unit 153 outputs the data of the attention drawing image GC to the projector 2 via the output unit 14 (step S306 in FIG. 28), and ends the output process.
[0137] When the data of the attention drawing image GC is input from the output processing unit 153, the projector 2 projects the attention drawing image GC on the surface of the inspected product W based on the input data of the attention drawing image GC. Through the above process, the attention drawing image GC according to the time period during which the appearance inspection is performed is displayed on the surface of the inspected product W.
[0138] In general, when the time period changes, the tendency of defects to occur changes. For example, immediately after the start of work, the concentration of workers in each process is high, and immediately before the end of work, the concentration of workers tends to be lower than immediately after the start of work. Therefore, immediately before the end of work, defects caused by carelessness of workers, such as defects caused by contacting parts with a conveyor, tend to occur more frequently than immediately after the start of work. On the other hand, immediately after the start of work, defects caused by problems in the production line or problems in product design tend to occur more than defects caused by carelessness of workers. According to the appearance inspection support device 100 of the fourth embodiment, the attention drawing image GC displayed on the surface of the inspected product W changes depending on the time period during which the appearance inspection is performed. Therefore, the efficiency of the appearance inspection can be improved.
[0139] [Modification of the fourth embodiment] The order in which the input processes are performed can be changed as appropriate. The process of step S702 shown in Fig. 25 only needs to be performed before step S112, and may be performed, for example, between step S108 and step S110. Similarly, the order in which the output processes are performed can be changed as appropriate. The process of step S802 shown in Fig. 28 only needs to be performed before step S306, and may be performed, for example, between step S302 and step S304.
[0140] [Embodiment 5] 29 to 33, a visual inspection support device 100 according to a fifth embodiment will be described. The fifth embodiment differs from the first to fourth embodiments in that the visual inspection support device 100 presents improvement target areas of the inspected product W based on the number of defects and items related to improvement measures already implemented for defects that have occurred in the past. The following describes the fifth embodiment, focusing on the differences from the first to fourth embodiments.
[0141] 29, the visual inspection support apparatus 100 further includes an improvement measure presentation unit 154 that presents an area to be improved based on the number of defects and items related to implemented improvement measures. The improvement measure presentation unit 154 is realized by the processor 15 shown in FIG. 3 executing a control program and using the operation unit 11, the display 12, the input unit 13, the output unit 14, the main storage device 16a, and the auxiliary storage device 16b as resources.
[0142] The data storage unit 16 further includes a trained model storage unit 168 and an improvement data storage unit 169.
[0143] The trained model storage unit 168 stores trained models. The trained models are generated by, for example, a "supervised learning method" that uses a known machine learning library and trains a set of data in which training data is labeled with a correct answer. Note that the trained models may be trained and generated by other learning methods such as an "unsupervised learning method," a "reinforcement learning method," or a "semi-supervised learning method."
[0144] The learning algorithm may be a known learning algorithm such as a neural network. The neural network has an input layer composed of a plurality of nodes to which different input parameters are input, an intermediate layer to which signals output from each node of the input layer are input, and an output layer to which signals output from the intermediate layer are input and which outputs output parameters. The intermediate layer of the neural network may be composed of one or more layers.
[0145] As the learning data, data on implemented improvement measures is used, and as the correct label, data indicating the area to be improved and data indicating the priority improvement points are used.
[0146] The items related to the implemented improvement measures include data indicating the model, data indicating the surface of the inspected product W to be inspected, data indicating the defective area, data indicating the number of defects, data indicating a rework coefficient set based on the rework time required to remedy the defects, and data indicating a replacement coefficient set based on the cost of a substitute product required to remedy the defects. The data indicating the model, data indicating the surface of the inspected product W to be inspected, data indicating the defective area, data indicating the number of defects, data indicating the rework coefficient, and data indicating the replacement coefficient are associated with each other. The number of defects is calculated by tallying up the model, the surface of the inspected product W to be inspected, and the defective area.
[0147] The improvement priority points used for the correct answer label are set for each model, the surface to be inspected of the inspected product W, and the defective area. The improvement priority points are calculated by multiplying the number of defects by a rework time coefficient and an alternative coefficient. The improvement priority points are set in descending order of the value obtained by multiplying the number of defects by a rework time coefficient and an alternative coefficient. Alternatively, the improvement priority points may be set according to the range of the value obtained by multiplication, for example. The improvement measure points may be set to "1" when the value obtained by multiplication is equal to or greater than 1 and less than 10, and "2" when the value obtained by multiplication is equal to or greater than 10 and less than 50.
[0148] The improvement measure presentation unit 154 generates a trained model and presents improvement measures to the user using the generated trained model.
[0149] As shown in FIG. 29, the improvement measure presentation unit 154 includes a learning unit 154a that generates a trained model, and an inference unit 154b that infers an improvement measure using the trained model.
[0150] The learning process by the learning unit 154a will be described below. The learning process shown in Fig. 30 starts when the operation unit 11 accepts an operation by a user to instruct the execution of the learning process.
[0151] The learning unit 154a receives data on the implemented improvement measures to which the correct answer labels are added as training data (step S902). The training data is input to the learning unit 154a via the input unit 13.
[0152] The learning unit 154a repeatedly learns using the teacher data (step S904) and analyzes the characteristics, rules, etc. of the data included in the teacher data. The learning unit 154a stores the trained model in the trained model storage unit 168 (step S906) and ends the learning process.
[0153] The inference unit 154b executes the inference process shown in Fig. 31 by using the trained model. The inference process is executed at a timing preset by a user. The user sets the timing for executing the inference process to, for example, every week or every month.
[0154] The inference unit 154b inputs data on the implemented improvement measures as input parameters to the trained model (step S1002). Data input via the operation unit 11, the input unit 13, etc. is used as the input parameters.
[0155] The inference unit 154b uses the learned model to perform inference based on the input parameters (step S1004), outputs data indicating the area to be improved and data indicating the improvement priority points as output parameters (step S1006), and terminates the inference process.
[0156] The improvement measure presentation unit 154 acquires the output parameters output from the inference unit 154b as improvement measure data. Upon acquiring the improvement measure data, the improvement measure presentation unit 154 stores the improvement measure data in an improvement measure data storage unit 169 shown in FIG.
[0157] The improvement measure data storage unit 169 stores priority order data 169a indicating the priority order, which is the order in which improvement is required, model data 169b indicating the model, part data 169c indicating the surface to be inspected of the inspected product W, and improvement target area data 169d indicating the improvement target area, in association with each other. The improvement measure presentation unit 154 stores data indicating the improvement target area included in the output parameters as the improvement target area data 169d. The improvement measure presentation unit 154 sets a priority order based on data indicating an improvement priority point included in the output parameters output by the inference unit 154b, and stores the set data as priority order data 169a. The improvement measure presentation unit 154 sets a priority order, for example, in descending order of the value of the improvement priority point.
[0158] When the improvement measure presenting unit 154 acquires the improvement measure data, it presents the improvement target areas, which have been prioritized based on the acquired improvement measure data, as improvement measures to the user.
[0159] The improvement measure presentation unit 154 may display the improvement measures on the display 12 using an application such as a spreadsheet software based on the improvement measure data. The improvement measure data can be used as training material for new employees. In addition, the improvement measure data can be used as reference material when a similar defect occurs.
[0160] As described above, according to the visual inspection support device 100 of the fifth embodiment, the improvement measure presenting unit 154 presents an improvement measure to a user by using a trained model trained with data including rework time for defects that have occurred in the past, the cost of a substitute, and the like. That is, the improvement measure presenting unit 154 presents to a user an improvement measure that takes into consideration losses such as rework time for defects that have occurred in the past and the cost of a substitute. The user can carry out efficient improvement activities by implementing the improvement measure presented by the improvement measure presenting unit 154. As a result, the occurrence of defects is reduced, and the workload of visual inspection is reduced. Therefore, the efficiency of visual inspection can be improved, and the quality of visual inspection can be improved.
[0161] The improvement measure presenting unit 154 presents the prioritized improvement target areas to the user, so that the user can efficiently carry out improvement activities.
[0162] Also, due to time constraints, some factories are unable to analyze all defects that have occurred, and the reliability of the results of an analysis based on some of the defects that have occurred is low, and improvement measures based on unreliable analysis results may not be suited to the actual situation. Improvement measures that are not suited to the actual situation often deviate from the improvement measures desired by the workers in charge of each process, and improvement measures that deviate from the workers' requests cannot be used to carry out efficient improvement activities. However, the visual inspection support device 100 collects and analyzes data on all defects that have occurred, and therefore can obtain highly reliable analysis results. Therefore, improvement measures suited to the actual situation can be presented to the user, and efficient improvement activities can be carried out.
[0163] Furthermore, if a time lag occurs between the occurrence of a defect and the implementation of an improvement activity, there is a possibility that the improvement measures implemented in the improvement activity may not be suited to the actual situation, but the appearance inspection support device 100 according to the fifth embodiment can suppress the occurrence of a time lag between the occurrence of a defect and the implementation of an improvement activity. This makes it possible to present the user with an improvement measure suited to the actual situation, and to implement efficient improvement activities.
[0164] [Modification of the fifth embodiment] 33, the remediation data storage unit 169 may further store, in association with each other, number of defects data 169e indicating the number of defects that occurred during a period set by a user, rework time data 169f indicating rework time required to remedy defects that occurred in the past, and replacement cost data 169g indicating the cost of a replacement product required to remedy defects that occurred in the past. The rework time data 169f is calculated based on a rework coefficient, and the replacement cost data 169g is calculated based on a replacement coefficient.
[0165] The improvement measure presentation unit 154 may further present the number of defects, rework time, and replacement cost to the user based on the data stored in the improvement measure data storage unit 169. This allows the user to quantitatively and objectively grasp the items related to the defects that have occurred, such as the rework time and replacement cost.
[0166] Although the improvement measure presenting unit 154 presents the prioritized improvement target areas as improvement measures to the user, the priorities may not be assigned. In this case, the teacher data learned by the learning unit 154a can omit the improvement priority points. Furthermore, the inference unit 154b outputs data indicating the improvement target areas as output parameters.
[0167] The input parameters input to the learning unit 154a may further include data indicating the degree of difficulty of improvement. The degree of difficulty of improvement is preset by a user, such as three levels or five levels, based on the cause of the defect (scratches, color unevenness) and the area of the defect. For example, the degree of difficulty of improvement of a stain on the inspected product W is set to "1" because the defect can be improved by a relatively easy task such as wiping off the stain. The degree of difficulty of improvement of a dent on the inspected product W is set to "2" because the defect can be improved by a task of correcting the dent location, which requires more time than wiping off the stain. The degree of difficulty of improvement of a scratch on the inspected product W is set to "3" because the defect can be improved by replacing a part, which requires more time than wiping off the stain and correcting the dent location. By further including the degree of difficulty of improvement as an input parameter input to the learning unit 154a, more efficient improvement activities can be carried out.
[0168] In the fifth embodiment, the learning unit 154a uses a neural network as a learning algorithm. However, the learning unit 154a may use other known methods, such as genetic programming, functional logic programming, or a support vector machine, as a learning algorithm.
[0169] In the fifth embodiment, the learning unit 154a and the inference unit 154b are provided in the control device 10, but the learning unit 154a and the inference unit 154b may be provided in a device different from the control device 10. The improvement measure presentation unit 154 acquires output parameters output from the inference unit 154b provided in a device different from the control device 10 as improvement measure data.
[0170] The improvement measure data storage unit 169 may be stored in a device that can be shared among multiple departments. For example, in the work of each process in a factory, by devising a way to hold a workpiece, it is often possible to prevent the workpiece from colliding with factory equipment such as a conveyor or a workbench, thereby reducing the occurrence of defects. Therefore, by storing the improvement measure data storage unit 169 in a form that can be shared among multiple departments, effective improvement measures can be deployed throughout the factory, and improvement activities can be carried out efficiently.
[0171] In addition, the improvement measure presentation unit 154 may use data on implemented improvement measures of not only the same model but also similar models. That is, the improvement measure presentation unit 154 can also use data on implemented improvement measures of similar models as learning data.
[0172] The present disclosure is not limited to the above-described exemplary embodiments, and can be embodied in other embodiments by making appropriate modifications.
[0173] For example, the first to fifth embodiments can be combined as appropriate. In particular, the attention drawing image GC displayed on the inspected product W may change according to the weight described in the second embodiment and the proficiency level of the operator P in charge of the visual inspection described with reference to the third embodiment. For example, the attention drawing image GC may include an image pointing to the specific region R1 to which all weights "1" to "3" are set when the proficiency level of the operator P in charge of the visual inspection is low at "1," and may include an image pointing to the specific region R1 to which the high weight "3" is set when the proficiency level of the operator P in charge of the visual inspection is high at "3."
[0174] In the present disclosure, the tendency of defective areas is analyzed as the tendency of defects, but the tendency analyzed by the visual inspection support device 100 is not limited to defective areas, and may be causes of defects such as scratches, color unevenness, etc. Also, although the tendency of defective areas is identified as the tendency of defective areas, areas with a low tendency of defects may be identified.
[0175] In the present disclosure, the defective part data is data of the defective product image GD, but the defective part data may be data related to the defective part of the defective product D, and is not limited to the data of the defective product image GD. The defective part data may be, for example, text data, audio data, etc.
[0176] In the present disclosure, the warning image GC is projected onto the inspected product W by the projector 2, but as long as the worker P can identify the area on the inspected product W, the warning image GC may be displayed, for example, on a monitor installed at the appearance inspection site SV.
[0177] In the present disclosure, data indicating a warning image GC pointing to the specific area R1 is output as information indicating an area where defects are highly likely to occur, but the information on the area where defects are highly likely to occur is not limited to data indicating the warning image GC and may be audio data explaining the specific area R1.
[0178] Furthermore, the images GC1 and GC2 pointing out the specific region R1 may be any shape as long as the operator P can recognize the specific region R1, and are not limited to a circle. The images GC1 and GC2 pointing out the specific region R1 may be a rectangle, an ellipse, an arrow, or the like. Alternatively, the images GC1 and GC2 may be solid images in a color different from the external color of the inspected product W. Alternatively, the images may be images of text such as "SECTION 9" or "Pay attention to SECTION 9."
[0179] In the present disclosure, the data storage unit 16 is stored in the auxiliary storage device 16b of the control device 10, but the data storage unit 16 may also be stored in a memory area of a device other than the control device 10, such as a shared server within a company, a cloud server, or the like.
[0180] In this disclosure, the product PR has been an electrical appliance, but the product PR is not limited to an electrical appliance and may be any product that requires visual inspection. [Explanation of symbols]
[0181] 1...camera, 2...projector, 10...control device, 11...operation unit, 12...display, 13...input unit, 14...output unit, 15...processor, 16...data storage unit, 16a...main storage unit, 16b...auxiliary storage unit, 100...visual inspection support device, 151...input processing unit, 152...analysis unit, 153...output processing unit, 154...improvement measure presentation unit, 154a...learning unit, 154b...inference unit, 160...image data storage unit, 161...production schedule storage unit, 161a...model data, 161b...production schedule data, 161c...production development Start date and time data, 161d...production end date and time data, 162...equipment master storage unit, 162a...equipment name data, 162b...equipment identification name data, 162c...installation location data, 162d...inspection content data, 163...good product image related data storage unit, 163a...good product image identification name data, 163b...model data, 163c...photographed part data, 164...defective product image related data storage unit, 164a...defective product image identification name data, 164b...model data, 164c...photographed part data, 164d...discovery date and time data, 164e...defective area data, 164f...worker data, 164g...time period data, 165...output preparation data storage unit, 165a...model data, 165b...photographed portion data, 165c...specific area data, 165d...weight data, 165e...proficiency data, 165f...time period data, 166...responsibility schedule storage unit, 166a...worker data, 166b...responsibility schedule data, 166c...responsibility start date and time data, 166d...responsibility end date and time data, 166e...inspection content data, 167...worker master storage unit, 167a...worker data, 167b ...proficiency data, 168...learned model memory section, 169...improvement measure data memory section, 169a...priority order data, 169b...model data, 169c...partial data, 169d...improvement target area data, 169e...number data, 169f...rework time data, 169g...replacement cost data, D...defective product, GC...warning image, GD...defective product image, GR...good product image, P...worker, PR...product, R1...specific area, SC...section, SD...defective product storage area, SV...visual inspection area, T1...first defect threshold, T2...second defect threshold, W...inspected product.
Claims
1. An appearance inspection support device that supports an appearance inspection performed on an inspected product, an input unit for inputting defect location data indicating defect locations of defective products in which defects are found among the products to be inspected; an analysis unit that analyzes a tendency of an area including the defect portion based on the defect portion data of the plurality of the defective products, and identifies an area where the tendency of the defect to occur is equal to or higher than a standard as an area where the defect is likely to occur; an output unit that outputs information indicating an area where the defect is likely to occur according to a level of proficiency of an operator in charge of the visual inspection; An appearance inspection support device comprising:
2. 2. The visual inspection support device according to claim 1, wherein the analysis unit acquires a number of defect occurrences indicating the number of occurrences of the defect locations, and identifies an area where the number of defect occurrences is equal to or greater than a first defect threshold as an area where the defect is highly likely to occur.
3. the analysis unit sets a weight for each area in which the defect is likely to occur based on the number of defect occurrences; The visual inspection support device according to claim 2 , wherein the output unit outputs information indicating an area having the weight specified as an output target and having a high tendency for the defect to occur.
4. The analysis unit includes: Analyzing the number of occurrences of the defect at the defect location for each time when the defect occurred and for each area including the defect location; Identifying an area including a defect portion where the time when the defect occurred and the number of occurrences of the defect for each area including the defect portion are equal to or greater than a second defect threshold value as an area where the defect is likely to occur; 4. The visual inspection support device according to claim 2, wherein the output unit outputs information indicating an area where the defect is likely to occur corresponding to a time when the visual inspection is performed.
5. 5. The visual inspection support device according to claim 2, further comprising an improvement measure presenting unit that presents an improvement target area of the inspected product based on the number of defects and information related to improvement measures that have been implemented.
6. The improvement measure suggestion unit is A learning unit that generates a trained model that has learned data related to the implemented improvement measures; an inference unit that inputs data on the implemented improvement measures into the trained model and outputs data indicating the improvement target area and data indicating improvement priority points; Equipped with The improvement measure suggestion unit is setting a priority order for improvement for each improvement target area based on the data indicating the improvement priority points output by the inference unit; The visual inspection support device according to claim 5 , further comprising: a display unit that displays the priority order and the improvement target area.
7. a projector to which information indicating an area where the defect is likely to occur is input from the output unit, 7. The visual inspection support device according to claim 1, wherein the projector projects an image onto the inspected product based on information indicating an area where the defect is likely to occur input from the output unit.
8. Further, a camera is provided for photographing the defective product and generating image data of the defective product.
8. The visual inspection support device according to claim 1, wherein the camera outputs image data of the defective product to the input section as the defect location data.
9. an input step of inputting defect location data indicating defect locations of defective products in which defects are found among the products to be inspected; an analysis step of analyzing a tendency of an area including the defect portion based on the defect portion data of the plurality of the defective products, and identifying an area in which the tendency of the defect to occur is equal to or higher than a reference value as an area in which the defect is highly likely to occur; an output step of outputting information indicating an area where the defect is likely to occur according to the level of proficiency of an operator in charge of the visual inspection; A visual inspection support method comprising:
10. 10. A visual inspection support program that causes a computer to execute the visual inspection support method according to claim 9.
Citation Information
Patent Citations
Visual inspection method and visual inspection device
JP2007271510A
Flaw estimating system and method
JP2009198219A
Visual inspection system
JP2010139461A
Method and apparatus for inspecting defect of steel plate surface
JP2010266430A
Substrate inspection system, display device, and display method
JP2012145427A
