Inspection device and inspection method

Abstract

To provide an inspection device that can improve the convenience of workers in kitting operations. [Solution] The inspection device comprises a plate for placing an object to be inspected, a camera for photographing the object to be inspected placed on the plate, and a control device. The control device stores setting information including type information and quantity information for each of the multiple parts that make up the target product, and determines whether a first condition is met, that the type of the object to be inspected is the same as the type of a first part among the multiple parts, and that the number of objects to be inspected is the same as the required number of first parts needed to make up the target product, based on the captured image taken by the camera and the setting information, and displays the result of the determination on a display.

Description

Technical Field

[0001] The present disclosure relates to an inspection apparatus and an inspection method.

Background Art

[0002] In an operation of collecting (for example, kitting) parts necessary for product production from a parts storage area (for example, a warehouse) where various parts are stored, it is important to quickly and accurately kit the necessary parts in the necessary quantities. When kitting parts for manufacturing a product (for example, an elevator), the following operations are performed as an example.

[0003] An operator kits parts (for example, screws, nuts, doors, wires) necessary for elevator manufacturing from a parts storage area existing in the factory, and transports these parts to the work site using a trolley. At the work site, it is sequentially visually inspected whether each collected part is the correct part, and if there is no problem with the inspection, production (for example, welding, etc.) of the product is performed. If the collected part is the wrong part, the operator needs to kit the correct part from the parts storage area again. The above visual inspection is performed using, for example, a photographed image of a camera.

[0004] Japanese Patent Application Laid-Open No. 11-37946 (Patent Document 1) discloses a key and cylinder inspection method. In this inspection method, a key or a cylinder is photographed by a camera, a reference point is determined from this photographed image, a measurement line is generated from this reference point, and the coordinates of the notch pattern and production data are compared.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] As mentioned above, workers need to accurately and quickly determine whether the various parts collected from the parts storage area are the correct parts, and proceed smoothly with the kitting work (i.e., the work of collecting the parts necessary for product production). However, the technology described in Patent Document 1 is an inspection method specifically for keys or cylinders and does not provide a solution to this problem.

[0007] One objective in certain aspects of this disclosure is to provide an inspection device and an inspection method that can improve the convenience of workers in kitting operations. [Means for solving the problem]

[0008] An inspection apparatus according to one embodiment comprises a plate for placing an object to be inspected, a photographing device for photographing the object to be inspected placed on the plate, and a control device. The control device includes a storage unit for storing setting information including type information and quantity information for each of a plurality of parts constituting a target product, a first determination unit for determining whether a first condition is met, based on the photographed image taken by the photographing device and the setting information, that the type of the object to be inspected is the same as the type of a first part among the plurality of parts, and that the number of objects to be inspected is the same as the required number of first parts necessary to constitute the target product, and a display control unit for displaying the determination result of the first determination unit on a display.

[0009] In another embodiment, an inspection method using an inspection device is provided. The inspection device comprises a plate on which an object to be inspected is placed, and a photographic device for photographing the object to be inspected placed on the plate. The inspection method includes the steps of determining, based on a photographed image taken by the photographic device and setting information including type information and quantity information for each of a plurality of parts constituting the target product, whether the type of the object to be inspected is the same as the type of a first part among the plurality of parts, and whether the number of objects to be inspected is the same as the required number of first parts required to constitute the target product, and displaying the determination result from the determination step on a display. [Effects of the Invention]

[0010] According to this disclosure, it is possible to improve the convenience of workers in kitting operations. [Brief explanation of the drawing]

[0011] [Figure 1] This is a perspective view showing an example of the configuration of an inspection device. [Figure 2] This is a side view taken from arrow II in Figure 1. [Figure 3] This figure shows an example of the hardware configuration of a control device. [Figure 4] This flowchart shows an example of the processing procedure for the inspection device. [Figure 5] This figure shows an example of a display screen. [Figure 6] This flowchart shows the subsequent processing of X in Figure 4. [Figure 7] This figure shows another example of the display screen. [Figure 8] This is a flowchart showing the continuation of the process for Z in Figure 4. [Figure 9] This figure shows yet another example of the display screen. [Figure 10] This figure shows yet another example of the display screen. [Figure 11] This figure shows an example of parts to be kitted. [Figure 12] This figure shows an example of an object to be inspected. [Figure 13] This is a flowchart showing the subsequent processing of Y in Figure 4. [Figure 14] This figure shows yet another example of the display screen. [Figure 15] This is a block diagram showing an example of the functional configuration of an inspection device. [Modes for carrying out the invention]

[0012] Hereinafter, this embodiment will be described while referring to the drawings. In the following description, the same parts are denoted by the same reference numerals. Their names and functions are also the same. Therefore, detailed descriptions thereof will not be repeated.

[0013] <Overall Configuration> FIG. 1 is a perspective view showing an example of the configuration of an inspection apparatus. FIG. 2 is a side view seen from arrow II in FIG. 1. Referring to FIGS. 1 and 2, the inspection apparatus 100 includes a control device 10, a plate 11, a top plate 12, a base 13, and a column member 14 that form a housing, a transmitted illumination 21 and reflected illuminations 22a and 22b as lighting devices, and cameras 23a and 23b as imaging devices.

[0014] The plate 11 is a transparent plate for placing the inspection object 50. The plate 11 may be any transparent plate having a strength capable of supporting the components constituting the product, and is, for example, made of an acrylic plate or the like. The top plate 12 has a role of preventing light from directly entering the cameras 23a and 23b, and is provided so as not to affect the identification process using the images captured by the cameras 23a and 23b (hereinafter, also simply referred to as "captured images"). The top plate 12 may be any plate that blocks light from the outside, and is, for example, made of a plastic plate having light-shielding properties.

[0015] The transmitted illumination 21 is installed below the lower surface of the plate 11 and irradiates light toward the lower surface of the plate 11. The reflected illuminations 22a and 22b are installed above the upper surface of the plate 11 and irradiate light toward the upper surface of the plate 11.

[0016] The cameras 23a and 23b are respectively arranged at different positions. Specifically, the camera 23a is installed on the upper surface of the housing (for example, the top plate 12) and acquires a captured image of the inspection object 50 placed on the plate 11 taken from above. The camera 23b is installed on the side surface of the housing and acquires a captured image of the inspection object 50 placed on the plate 11 taken from the side.

[0017] The control device 10 includes a main unit 140 having a processor, memory, etc., and a display 160. Typically, the control device 10 has a structure that conforms to a general-purpose computer architecture, and the processor executes pre-installed programs to realize various processes described later. The control device 10 is, for example, a desktop PC (Personal Computer). However, the control device 10 may be any device capable of performing the functions and processes described below, and may be other devices (for example, a laptop PC or a tablet terminal).

[0018] The processor controls the operation of the transmitted illumination 21, reflected illumination 22a, 22b, and cameras 23a, 23b based on the configuration information stored in memory. The processor performs image processing on the images output from cameras 23a, 23b, and processes such as measuring the shape of the object to be inspected 50 placed on the board 11 by the operator, and displays the processing results on the display 160. For example, the processor determines whether the object to be inspected 50 is suitable as a part to be kitted (hereinafter also simply referred to as "part to be kitted"), and displays the result of that determination on the display 160.

[0019] If the object to be inspected 50 is suitable as a kitting target part, the display 160 displays completion information indicating that the kitting of the kitting target part is complete. After confirming the completion information, the worker removes the object to be inspected 50 from the board 11 and places another object to be inspected on the board 11. On the other hand, if the above completion information is not displayed on the display 160, the worker determines that the object to be inspected 50 is not a kitting target part, removes the object to be inspected 50, and places another object to be inspected on the board 11. This process is repeated until the kitting of each component that makes up the product is complete.

[0020] <Control device hardware configuration> Figure 3 shows an example of the hardware configuration of the control device. Referring to Figure 3, the control device 10 is mainly composed of a microcomputer. Specifically, the control device 10 includes a processor 150, a main memory 152, a secondary memory 154, a communication interface 156, an input interface 158, and a display 160. These components are connected to each other so as to be able to communicate via an internal bus 162. The main unit 140 in Figure 1 includes the processor 150, the main memory 152, the secondary memory 154, the communication interface 156, and the input interface 158.

[0021] The processor 150 is typically a processing unit such as a CPU (Central Processing Unit) or MPU (Multi-Processing Unit), which reads various programs, including the OS (Operating System), installed in the secondary storage device 154, and executes them while expanding them into the main memory device 152.

[0022] The main memory 152 is typically a volatile storage medium such as DRAM (Dynamic Random Access Memory) and holds the code for various programs, including the OS executed by the processor 150, as well as various work data necessary for the execution of those programs. The secondary memory 154 is a non-volatile storage medium such as a hard disk or SSD (Solid State Drive) and holds various programs, including the OS, as well as various settings and other data.

[0023] The communication interface 156 mediates data transmission between the processor 150 and external devices. Communication via the communication interface 156 is carried out using the Internet, a wide area network (WAN), a local area network (LAN), a wired network, a wireless network, or a combination thereof.

[0024] The input interface 158 receives operational input to the control device 10. The input interface 158 can be implemented by, for example, a keyboard, buttons, a mouse, a touch panel, or the like.

[0025] The display 160 is, for example, a liquid crystal display, an organic EL (Electro-Luminescence) display, etc. The display 160 may be configured integrally with the control device 10, or it may be configured separately from the control device 10.

[0026] Unlike the example in Figure 1, it is also possible to configure at least a portion of the control device 10 using FPGA (Field Programmable Gate Array) and ASIC (Application Specific Integrated Circuit) circuits.

[0027] <Processing Procedure> Figure 4 is a flowchart illustrating an example of the processing procedure of the inspection device. Each process in Figure 4 is typically performed by the processor 150 of the inspection device 100. For the sake of clarity, let's assume that the components necessary to constitute a certain product are two parts A, three parts B, and one part C.

[0028] It is assumed that the product in question consists of two parts A, three parts B, and one part C, and that the inspection should be conducted in the order of parts A, B, and C, as predetermined in the work procedure. Therefore, the worker will attempt to inspect "two parts A," "three parts B," and "one part C" according to this work procedure.

[0029] Referring to Figure 4, the processor 150 reads the product configuration information (step S10). Specifically, the configuration information includes information on the type and quantity of each component required for the product, information on the order of each component, information defining the inspection method (e.g., measurement method) for each component, the shooting conditions for the cameras 23a and 23b used to inspect each component, and the lighting conditions for the transmitted illumination 21 and reflected illumination 22a and 22b used to inspect each component.

[0030] Type information indicates the type (i.e., variety) of a component, such as component A, component B, component C, etc. Components of the same type are all identical in shape, color, etc., and are interchangeable. Quantity information indicates the required number of components needed for a product. In this example, the processor 150, by referring to the type information and quantity information, determines that 2 parts A, 3 parts B, and 1 part C are needed to manufacture the product.

[0031] The sequence information indicates the order in which each component is inspected. In this embodiment, the inspection order corresponds to the kitting order of each component that the worker is to kit. In this example, the processor 150, by referring to the sequence information, determines that the inspection order (i.e., the kitting order) of each component is component A, component B, and component C.

[0032] The information defining the inspection method specifies an inspection algorithm suitable for measuring the shape of each part. For example, inspection methods include measuring the width of a part, measuring the diameter of holes in a part, and measuring the angle of a part. The specific inspection method to be used is predetermined for each type of part.

[0033] The imaging conditions include imaging equipment information indicating the camera to be used when inspecting each component, as well as information such as the camera's exposure time and shooting interval. Specifically, the imaging equipment information indicates which of cameras 23a and 23b will be used. The imaging conditions, for example, might be that camera 23a is used with an exposure time of 100ms and images are taken every 500ms. Note that inspection may be performed using both cameras 23a and 23b.

[0034] The lighting conditions include lighting equipment information indicating the lighting used when inspecting each component, as well as information such as the timing of lighting equipment activation and lighting intensity. Specifically, the lighting equipment information indicates which of the transmitted lighting 21 and reflected lighting 22a and 22b will be used. For example, transmitted lighting 21 is used when the inspection is performed based on information regarding the external dimensions of the object to be inspected 50. Reflected lighting 22a and 22b are used when the inspection is performed based on information regarding the surface of the object to be inspected 50.

[0035] Typically, the imaging and lighting equipment used is determined by which parts of the component have different dimensions. For example, when measuring the dimensions of a thin, plate-like component viewed from above, transmitted illumination 21 and camera 23a are used. When measuring the dimensions of a thick component viewed from the side, reflected illumination 22a, 22b and camera 23b are used. When measuring the diameter of a hole on the surface of a thick component, reflected illumination 22a, 22b and camera 23a are used.

[0036] The processor 150 reads the configuration information and determines that part A is the part to be inspected first. Based on the inspection algorithm, shooting conditions, and lighting conditions suitable for measuring the shape of part A, it starts inspecting the object placed on the board 11 by the operator. Here, the camera used to inspect part A is assumed to be camera 23a.

[0037] Next, the processor 150 starts taking images using the camera 23a (step S12). At this time, the display 160 shows a display screen as shown in Figure 5.

[0038] Figure 5 shows an example of a display screen. Referring to Figure 5, the display screen 510 includes a sample image of the part currently being inspected (i.e., the part to be kitted) (for example, part A), the required number of parts A (for example, 2), the image captured by camera 23a, and information indicating whether or not the kitting of the part to be kitted has been completed.

[0039] The image captured by camera 23a shows that no objects to be inspected have yet been placed on board 11. Also, the display "Kitting complete: No" indicates that the kitting of part A is not yet complete. The worker then uses the sample image of part A and the required quantity displayed on screen 510 as a reference and begins placing the two objects that they have identified as part A onto board 11.

[0040] Referring again to Figure 4, the processor 150 processes the captured image using a known method to determine whether or not it has recognized the worker's hand from the captured image (step S14). For example, the processor 150 recognizes the worker's hand by comparing the captured image with a pre-prepared reference image of a hand. Here, the image processing may include, for example, binarization, pattern matching, or blob feature extraction. Note that the image processing may also be of a different type.

[0041] Figure 6 is a flowchart showing the continuation of the processing for X in Figure 4. Referring to Figures 4 and 6, if the operator's hand is recognized (YES in step S14), the processor 150 processes the captured image using a known method to determine whether or not a pre-registered hand gesture has been recognized from the captured image (step S20). For example, the processor 150 extracts characteristic body points (fingertips, wrists, etc.) from the captured image and recognizes the gesture based on the movement of these characteristic points. Examples of pre-registered gestures include sweeping the hand from left to right, covering the lower right of the screen, and clenching the hand. There are no restrictions on the hand shape used for the gesture or the number of gestures that can be registered. At this time, the display 160 displays a screen as shown in Figure 7.

[0042] Figure 7 shows another example of the display screen. Referring to Figure 7, it can be seen that the captured image on display screen 520 includes the worker's hands. The other display contents on display screen 520 are the same as those on display screen 510.

[0043] Referring again to Figure 6, if the gesture is not recognized (NO in step S20), the processor 150 proceeds to the process in step S14. An example of a case where the hand is recognized but the gesture is not is when the operator's hand is visible when placing the object to be inspected on the board 11.

[0044] On the other hand, if a gesture is recognized (YES in step S20), the processor 150 executes a pre-configured process according to the recognized gesture (step S22) and proceeds to the process in step S14. For example, pre-configured processes include closing a pop-up error message displayed on the display screen of the display 160, temporarily suspending the inspection (or kitting work), and changing the part to be inspected to another part. This allows the worker to proceed with the work without using the input interface 158 (e.g., keyboard, mouse, touch panel, etc.). This is advantageous because workers often wear gloves or have dirty hands during kitting work, as they can give instructions to the control device 10 without directly operating a mouse, touch panel, etc.

[0045] Referring again to Figure 4, if the worker's hand is not recognized (NO in step S14), in step S16, the processor 150 determines whether the kitted flag is "True". The kitted flag indicates whether the kitting of the part currently being kitted (i.e., the part being inspected) (for example, part A) has been completed.

[0046] The kitting flag is a variable that takes either the value "True" or "False". Specifically, a kitting flag of "True" indicates that the parts to be kitted have been kitted (i.e., the parts have been properly assembled). In this case, the worker determines that the parts to be kitted (e.g., part A) have been properly kitted, and therefore it is safe to remove the object to be inspected from the board 11 and proceed to inspect the next part (e.g., part B).

[0047] On the other hand, a "Kitted" flag of "False" indicates that the kitting of the component to be kitted is not yet complete. In this case, the worker understands that the component to be kitted has not been properly kitted. Therefore, the worker needs to place the object to be inspected, which they have determined to be a component to be kitted, onto board 11. Note that in the initial state before work begins, the "Kitted" flag is set to "False".

[0048] First, we will explain the process executed in the route when the kitting flag is "False" (NO in step S16), and then we will explain the process executed in the route when the kitting flag is "True" (YES in step S16).

[0049] If the kitting flag is "False" (NO in step S16), the processor 150 determines whether or not it has recognized an object to be inspected on the board 11 based on the captured image (step S18). For example, the processor 150 processes the captured image using a known method to determine whether or not it has recognized any object (e.g., an object to be inspected) from the captured image. If no object to be inspected is recognized (i.e., nothing is placed on the board 11), the processor 150 proceeds to the process in step S14.

[0050] Figure 8 is a flowchart showing the continuation of the process from Z in Figure 4. Referring to Figures 4 and 8, if the object to be inspected is recognized (YES in step S18), the processor 150 measures the shape of the recognized object to be inspected (step S40). Specifically, the measurement of the shape includes measuring various dimensions that define the external shape (e.g., height, width, depth, etc.), angles, hole positions, hole diameters, distances between specified locations, etc. The values ​​to be measured, the measurement method, etc. are predetermined in the setting information. There may be one object to be inspected or there may be multiple objects. If there are multiple objects to be inspected, the processor 150 measures the shape of each object to be inspected.

[0051] Next, the processor 150 counts the number of inspection objects that have a shape matching the shape of the kitting target part (for example, part A) (step S42). In other words, the processor 150 counts the number of inspection objects of the same type as the kitting target part. The information indicating the shape of the kitting target part is predetermined in the setting information.

[0052] The processor 150 determines whether the counted number matches the required number of parts to be kitted (step S44). If the parts to be kitted are "part A", the required number is "2", so the processor 150 determines whether the number of inspection targets of the same type as part A is "2".

[0053] If the counted number matches the required number (YES in step S44), the processor 150 sets the kitted flag to “True” (step S46) and proceeds to step S14. This indicates that the kitting of the component to be kitted has been successfully completed. At this time, the display 160 displays a screen as shown in Figure 9.

[0054] Figure 9 shows yet another example of the display screen. Referring to Figure 9, the captured image displayed on display screen 530 includes two objects to be inspected. Furthermore, the information "Kitting Completed" is displayed. This allows the operator to understand that the kitting of the currently kitting target part has been completed. Note that the other display contents on display screen 530 are the same as the display contents on display screen 510.

[0055] Referring again to Figure 8, if the counted number does not match the required number (NO in step S44), the processor 150 proceeds to step S14. At this time, the display 160 shows a screen as shown in Figure 10.

[0056] Figure 10 shows yet another example of the display screen. Referring to Figure 10, the captured image displayed on display screen 540 contains two objects to be inspected. In the captured image, the upper object to be inspected, enclosed by a dotted line, is of a different type from the kitting target part. On the other hand, the lower object to be inspected, not enclosed by a dotted line, is of the same type as the kitting target part. As a result, the operator understands that the upper object to be inspected is not the kitting target part. In this case, the kitting completed flag is not set to "True" (i.e., the kitting completed flag remains "False"), and the information "Kitting Complete: No" is displayed, indicating that the kitting of the kitting target part is not yet complete.

[0057] Here, we will further explain the flow of steps S40 to S46 in Figure 8 using a specific example. Figure 11 shows an example of a part to be kitted. Figure 12 shows an example of an object to be inspected.

[0058] Referring to Figure 11, the kitting target part 600 has a shape characterized by a length of 100 mm in the part shown in Figure 11 and an angle of 30° in the part shown in Figure 11. It is assumed that two such kitting target parts 600 need to be kitted.

[0059] Referring to Figure 12, the captured image 800 taken by camera 23a includes the objects 710 and 720 to be inspected. The processor 150 performs image analysis on the captured image 800 to determine the length L1 and angle θ1 of the object 710 and the length L2 and angle θ2 of the object 720. This corresponds to the process in step S40 in Figure 8.

[0060] First, let's consider the case where length L1 is 100 mm, angle θ1 is 30 degrees, length L2 is 100 mm, and angle θ2 is 40 degrees. In this case, the dimensions of the object to be inspected 710 (i.e., length L1 and angle θ1) are the same as the dimensions of the kitting target part 600. Therefore, the processor 150 counts the object to be inspected 710 as the same type of part as the kitting target part 600. On the other hand, the dimensions of the object to be inspected 720 (i.e., length L2 and angle θ2) are different from the dimensions of the kitting target part 600. Therefore, the processor 150 does not count the object to be inspected 720 as the same type of part as the kitting target part 600. Therefore, the number of items counted is "1". This corresponds to the processing in step S42.

[0061] The processor 150 determines that the counted number (i.e., "1") does not match the required number (i.e., "2"). This corresponds to the NO processing in step S44 in Figure 8.

[0062] Next, let's consider the case where length L1 is 100 mm, angle θ1 is 30 degrees, length L2 is 100 mm, and angle θ2 is 30 degrees. In this case, the dimensions of the objects to be inspected 710 and 720 are the same as the dimensions of the kitting target part 600. Therefore, the processor 150 counts the objects to be inspected 710 and 720 as parts of the same type as the kitting target part 600. Thus, the number of items counted is "2". This corresponds to the processing in step S42 in Figure 8.

[0063] The processor 150 determines that the counted number (i.e., "2") matches the required number (i.e., "2"). This corresponds to the YES processing in step S44 in Figure 8. Then, the processor 150 sets the kitted flag to "True" (corresponding to step S46 in Figure 8).

[0064] Next, in Figure 4, we will explain the process executed in the route when the kitting flag is "True" (YES in step S16).

[0065] Figure 13 is a flowchart showing the continuation of the process for Y in Figure 4. Referring to Figure 13, the processor 150 determines whether or not it has recognized the object to be inspected on the board 11 based on the captured image (step S30). If the object to be inspected is recognized (YES in step S30), the processor 150 proceeds to the process in step S14. In this case, it is determined that the object to be inspected has been recognized after determining that the worker's hand is not recognized (NO in step S14) and determining that the kitting flag is "True" (YES in step S16). This state indicates that the worker's hand is not recognized and the kitting of the kitting target parts is complete, but the object to be inspected placed on the board 11 has not been removed. At this time, the display screen 530 shown in Figure 9 above is displayed on the display 160.

[0066] On the other hand, if the object to be inspected is not recognized (NO in step S30), the processor 150 sets the kitted flag to "False" (step S32). In this case, the determination that the object to be inspected is not recognized is made after determining that the worker's hand is not recognized (NO in step S14) and determining that the kitting flag is "True" (YES in step S16). This state indicates that the worker's hand is not recognized and the object to be inspected has already been removed from the board 11 because the kitting of the kitted parts has been completed. Therefore, the processor 150 sets the kitted flag to its initial state of "False".

[0067] The processor 150 determines, based on the sequence information, whether or not there is a component that should be inspected next (i.e., a component to be kitted) (step S34). If the component does not exist (NO in step S34), the processor 150 terminates the process. This indicates that all components required for the product have been inspected and successfully kitted.

[0068] On the other hand, if the part in question exists (YES in step S34), the processor 150 performs a transition process to the next part to be inspected (step S36). Specifically, based on the settings information it has read, the processor 150 determines that the next part to be inspected is part B, and sets an inspection algorithm, shooting conditions, and lighting conditions suitable for measuring the shape of part B. At this time, the display 160 displays a screen as shown in Figure 14.

[0069] Figure 14 shows yet another example of the display screen. Referring to Figure 14, the display screen 550 shows a sample image of the part to be kitted (for example, part B), and the required number of parts to be kitted, "3," is displayed. In addition, the information "Kitting complete: No" is displayed, indicating that the kitting of the part to be kitted is not yet complete. By checking the display screen 550, the operator can understand the current shape and required number of parts to be kitted.

[0070] After the processor 150 performs the process of transitioning to the next part to be inspected, it continues the process of step S14. This allows the prepared inspection object to be inspected for the new kitting target part (for example, part B). The same process is followed for part C.

[0071] According to the process described above, the objects to be inspected placed on the board 11 are recognized in real time, and the processing results are displayed on the display 160. Therefore, the operator can determine in real time whether the objects to be inspected placed on the board 11 are of the same type as the parts to be kitted, and whether the number of objects to be inspected matches the required number. This prevents errors such as kitting the wrong parts.

[0072] Furthermore, because appropriate shooting conditions, lighting conditions, and inspection algorithms are set for each component, the accuracy of determining whether the inspected object matches the kitting target component can be improved. In addition, operators can perform inspections using hand gestures during the process, without using input devices such as mice or touch panels.

[0073] Furthermore, by using the kitted flag, the switching of parts to be kitted can be automated. Specifically, after the operator confirms that the kitting of the current part to be kitted is complete, they remove the object to be inspected from the board 11, leaving it empty. At this point, a sample image of the next part to be kitted is displayed, and the settings (e.g., shooting conditions, lighting conditions, inspection algorithm) are automatically switched to those suitable for that part.

[0074] Thus, the inspection device 100 according to this embodiment can improve the convenience of the worker during kitting work.

[0075] <Functional Configuration> Figure 15 is a block diagram showing an example of the functional configuration of an inspection device. Referring to Figure 15, the inspection device 100 includes a first determination unit 204, a display control unit 206, a second determination unit 208, a selection unit 210, a recognition unit 212, an imaging control unit 214, and an illumination control unit 216. These functions are realized, for example, by a processor 150 included in the inspection device 100. The inspection device 100 further includes a storage unit 202. The storage unit 202 is realized by at least one of a main memory device 152 and a secondary memory device 154.

[0076] The memory unit 202 stores setting information for multiple components (for example, components A to C) that make up the target product. The setting information includes type information and quantity information for each of the multiple components, sequence information indicating the inspection order of the multiple components, and information indicating the shooting conditions and lighting conditions for each of the multiple components.

[0077] The first determination unit 204 determines, based on the captured images and setting information taken by the imaging equipment (e.g., cameras 23a, 23b), whether the first condition is met, which is that the type of object to be inspected is the same as the type of the first part (e.g., part A) among the multiple parts, and the number of objects to be inspected is the same as the required number of first parts (e.g., 2) needed to constitute the target product.

[0078] The display control unit 206 displays a sample image of the first component, the required quantity, an image taken by the camera, and information indicating whether or not the kitting of the first component has been completed on the display 160. Typically, the display control unit 206 displays the above-described display screens 510 to 550 on the display 160.

[0079] In certain situations, the display control unit 206 displays the determination result of the first determination unit 204 on the display 160. Specifically, if the first condition is met, the display control unit 206 displays information on the display 160 indicating that the inspection of the first component has been completed (for example, information indicating that kitting is complete) (for example, displaying display screen 530). If the first condition is not met, the display control unit 206 displays information on the display 160 indicating that the inspection has not been completed (for example, information indicating that kitting is not complete) (for example, displaying display screen 540).

[0080] After the first condition is met, the second determination unit 208 determines whether the captured image includes at least one of the object to be inspected and the worker's hand. If neither the object to be inspected nor the worker's hand is included in the captured image, the selection unit 210 selects a second part (for example, part B) from among the multiple parts to be inspected after the first part, based on the sequence information. The display control unit 206 displays a sample image of the selected second part (for example, the sample image shown in Figure 14) on the display 160.

[0081] The recognition unit 212 recognizes the operator's hand gestures included in the captured image. The display control unit 206 executes display processing corresponding to the gesture. For example, the display processing may include closing the pop-up error message displayed on the display screen, or changing the sample image and required quantity of the current kitting target part to those of other kitting target parts.

[0082] The shooting control unit 214 controls cameras 23a and 23b for each of the multiple components based on the shooting conditions of that component. The lighting control unit 216 controls transmitted illumination 21 and reflected illumination 22a and 22b for each of the multiple components based on the lighting conditions of that component.

[0083] Other embodiments. (1) In the above-described embodiment, a configuration for recognizing hand gestures and objects to be inspected was explained using rule-based image processing, but the system is not limited to this configuration. For example, these may be recognized using AI (Artificial Intelligence) methods such as neural networks. In this case, the hand gestures and the parts to be kitted are trained in advance to create a trained model. Then, inference is performed using the trained model to identify the hand gestures and the parts to be kitted. By using AI methods, it is possible to handle more complex hand gestures and parts to be kitted.

[0084] (2) The configurations illustrated above as embodiments are examples of the configuration of the present invention, and can be combined with other known technologies, and can be modified, such as by omitting parts, without departing from the spirit of the present invention. Furthermore, in the embodiments described above, processes and configurations described in other embodiments may be appropriately adopted and implemented.

[0085] <Note> The various aspects of this disclosure are summarized below as an appendix.

[0086] (Note 1) An inspection apparatus comprising a board for placing an object to be inspected, a photographing device for photographing the object to be inspected placed on the board, and a control device, wherein the control device includes a storage unit for storing setting information including type information and quantity information for each of a plurality of parts constituting a target product, a first determination unit for determining whether a first condition is met, based on a photographed image taken by the photographing device and the setting information, that the type of the object to be inspected is the same as the type of a first part among the plurality of parts, and the number of objects to be inspected is the same as the required number of the first part necessary to constitute the target product, and a display control unit for displaying the determination result of the first determination unit on a display.

[0087] (Note 2) The inspection apparatus as described in Appendix 1, wherein the display control unit displays a sample image of the first component on the display.

[0088] (Note 3) If the first condition is met, the display control unit displays information on the display indicating that the inspection of the first component has been completed, as described in Appendix 1 or Appendix 2 of the inspection apparatus.

[0089] (Note 4) The inspection apparatus according to any one of Appendix 1 to Appendix 3, wherein the setting information further includes sequence information indicating the inspection order of the plurality of parts, the control device further includes a second determination unit that determines whether at least one of the object to be inspected and the worker's hand is included in the captured image after the first condition is met, and a selection unit that, if neither the object to be inspected nor the worker's hand is included in the captured image, selects a second part to be inspected after the first part from among the plurality of parts based on the sequence information, and the display control unit displays a sample image of the selected second part on the display.

[0090] (Note 5) The inspection apparatus according to any one of the appendices 1 to 4, wherein the control device further includes a recognition unit for recognizing the hand gestures of an operator included in the captured image, and the display control unit performs display processing corresponding to the gestures.

[0091] (Note 6) The inspection apparatus according to any one of the appendices 1 to 5, wherein the plate is a transparent plate, and further comprises a first lighting device installed above the upper surface of the transparent plate and irradiating light toward the upper surface of the transparent plate, and a second lighting device installed below the lower surface of the transparent plate and irradiating light toward the lower surface of the transparent plate, and the imaging device includes a first camera and a second camera positioned separately from the first camera.

[0092] (Note 7) The inspection apparatus as described in Appendix 6, wherein the setting information further includes information indicating shooting conditions and lighting conditions for each of the plurality of components, and the control device further includes a shooting control unit that controls the first camera and the second camera based on the shooting conditions for each of the plurality of components, and a lighting control unit that controls the first lighting device and the second lighting device based on the lighting conditions for each of the plurality of components.

[0093] (Note 8) An inspection method using an inspection device, wherein the inspection device comprises a plate for placing an object to be inspected and a photographing device for photographing the object to be inspected placed on the plate, and the inspection method includes the steps of determining whether the type of the object to be inspected is the same as the type of a first component among the plurality of components, and whether the number of the objects to be inspected is the same as the required number of the first component needed to constitute the target product, based on a photographed image taken by the photographing device and setting information including type information and quantity information of each of a plurality of components constituting the target product, and displaying the determination result from the determination step on a display.

[0094] The embodiments disclosed herein should be considered in all respects to be illustrative and not restrictive. The scope of the present invention is indicated by the claims rather than by the foregoing description, and all modifications within the meaning and scope equivalent to the claims are intended to be included. [Explanation of symbols]

[0095] 10 Control device, 11 Board, 12 Top plate, 13 Unit, 14 Column member, 21 Transmitted illumination, 22a, 22b Reflected illumination, 23a, 23b Camera, 50, 710, 720 Object to be inspected, 100 Inspection device, 140 Main unit, 150 Processor, 152 Main memory, 154 Secondary memory, 156 Communication interface, 158 Input interface, 160 Display, 162 Internal bus, 202 Storage unit, 204 First judgment unit, 206 Display control unit, 208 Second judgment unit, 210 Selection unit, 212 Recognition unit, 214 Shooting control unit, 216 Lighting control unit, 510, 520, 530, 540, 550 Display screen, 600 Kitting target parts, 800 Captured image.

Claims

1. A board for placing the object to be inspected, A photographic device for photographing the object to be inspected placed on the aforementioned plate, Equipped with a control device, The control device is A storage unit that stores setting information including type information and quantity information for each of the multiple components that make up the target product, A first determination unit determines, based on the captured image taken by the aforementioned photographic device and the setting information, whether the first condition is met, that the type of the object to be inspected is the same as the type of the first part among the plurality of parts, and that the number of objects to be inspected is the same as the required number of the first part necessary to constitute the target product. An inspection apparatus including a display control unit that displays the determination result of the first determination unit on a display.

2. The inspection apparatus according to claim 1, wherein the display control unit displays a sample image of the first component on the display.

3. If the first condition is met, the display control unit displays information on the display indicating that the inspection of the first component has been completed, according to claim 1 or claim 2.

4. The setting information further includes sequence information indicating the inspection order of the plurality of parts, The control device is After the first condition is met, a second determination unit determines whether the captured image includes at least one of the object to be inspected and the worker's hand, If the captured image does not include either the object to be inspected or the worker's hand, the system further includes a selection unit that, based on the sequence information, selects a second part from the plurality of parts to be inspected after the first part. The inspection apparatus according to claim 1 or claim 2, wherein the display control unit displays a sample image of the selected second component on the display.

5. The control device further includes a recognition unit for recognizing the hand gestures of the worker included in the captured image, The inspection apparatus according to claim 1 or 2, wherein the display control unit performs display processing corresponding to the gesture.

6. The aforementioned plate is a transparent plate, A first lighting device is installed above the upper surface of the transparent plate and irradiates light toward the upper surface of the transparent plate, The system further comprises a second lighting device installed below the lower surface of the transparent plate and irradiating light toward the lower surface of the transparent plate, The inspection apparatus according to claim 1 or 2, wherein the imaging equipment includes a first camera and a second camera positioned separately from the first camera.

7. The setting information further includes information indicating the shooting conditions and lighting conditions for each of the plurality of components, The control device is For each of the plurality of components, a shooting control unit controls the first camera and the second camera based on the shooting conditions of the component, The inspection apparatus according to claim 6, further comprising a lighting control unit that controls the first lighting device and the second lighting device based on the lighting conditions of each of the plurality of components.

8. An inspection method using an inspection device, The inspection apparatus comprises a plate for placing an object to be inspected and a photographic device for photographing the object to be inspected placed on the plate. The aforementioned inspection method is A step of determining whether the type of the object to be inspected is the same as the type of the first component among the multiple components, and whether the number of the object to be inspected is the same as the required number of the first component needed to constitute the target product, based on the image captured by the aforementioned imaging device and setting information including type information and quantity information of each of the multiple components constituting the target product, An inspection method comprising the step of displaying the result of the determination step on a display.

Images