Workpiece inspection camera, workpiece inspection lighting apparatus, system for assisting determination of arrangement of same, method for controlling system, and program for controlling system

Abstract

The present invention assists a camera, a lighting apparatus, and the arrangement of same suitable for inspecting a workpiece by evaluating an image of the workpiece imaged in a virtual space. This system (10) is provided with a storage unit, an input unit, and a processor. On the basis of one or more selection criteria pertaining to arrangement, the processor: selects one or more candidate cameras as a candidate for an inspection camera from among a plurality of cameras; selects one or more candidate lighting apparatuses as a candidate lighting apparatus for an inspection lighting apparatus from among a plurality of lighting apparatuses; selects a camera from among the one or more candidate cameras; selects a lighting apparatus from among the one or more candidate lighting apparatuses; images the workpiece in the virtual space a plurality of times while changing at least a portion of the position of the selected camera, the orientation of the selected camera, the position of the selected lighting apparatus, and the orientation of the selected lighting apparatus; analyzes the plurality of images obtained by the imaging; and outputs, on the basis of the analyzed result, evaluation information about each of a plurality of combinations of the selected cameras, the position and orientation of the selected camera, the selected lighting apparatus, and the position and orientation of the selected lighting apparatus.

Description

Workpiece inspection camera, inspection lighting, and system for assisting in determining their placement, system control method, and system control program

[0001] The present disclosure relates to a system for assisting in determining a camera for inspecting a workpiece, an inspection light, and the placement of these.

[0002] Cameras and lighting are used to inspect workpieces on production lines and the like. There are a wide variety of types of cameras and lighting. Furthermore, the appropriate camera and lighting positions and orientations vary depending on the surrounding environment, the shape or size of the workpiece, and other factors. Therefore, the designer of the production line or the like must select in advance the appropriate camera and lighting combination for workpiece inspection. Furthermore, the designer of the production line must adjust the positions and orientations of the selected cameras and lighting. These tasks require a large amount of man-hours. Therefore, technology to make these tasks more efficient is desired. One such technology is the ability to move a camera within a virtual space.

[0003] Regarding technology for moving a camera within a virtual space, for example, International Publication No. 2021 / 070226 (Patent Document 1) discloses a virtual camera control device including: "a gaze point determination unit that determines an arbitrary point of a traveling object or a viewing object, both of which are virtual 3D objects, placed within a virtual 3D space as a gaze point; and a virtual camera traveling unit that moves the virtual camera while maintaining the direction in which a virtual camera placed within the virtual 3D space takes pictures from the virtual 3D space in a direction from the virtual camera toward the gaze point determined by the gaze point determination unit and while maintaining a constant distance from the virtual camera to the traveling object" (see paragraph

[0007] ).

[0004] International Publication No. 2021 / 070226

[0005] The technology disclosed in Patent Document 1 may be able to move a camera within a virtual space. However, the technology disclosed in Patent Document 1 does not allow evaluation of images of workpieces captured using a camera and lighting within the virtual space. Therefore, even if a designer of a production line or the like uses the technology disclosed in Patent Document 1, he or she is unable to determine cameras, lighting, and their placement suitable for inspecting workpieces. Therefore, there is a need for a technology that supports the determination of cameras, lighting, and their placement suitable for inspecting workpieces by evaluating images of workpieces captured within a virtual space.

[0006] The present disclosure has been made in consideration of the above-described background, and in one aspect, the purpose is to provide a technology that assists in determining cameras, lighting, and their placement suitable for inspecting a workpiece by evaluating images of the workpiece captured in a virtual space.

[0007] According to one embodiment, a system for assisting in determining an inspection camera, an inspection light, and their placement for a workpiece is provided. The system includes a memory unit that stores information about a plurality of cameras and a plurality of lights, an input unit that accepts input of one or more selection criteria related to placement, and a processor. The processor selects one or more candidate cameras as candidates for the inspection camera from among the plurality of cameras based on the one or more selection criteria, selects one or more candidate lights as candidates for the inspection light from among the plurality of lights based on the one or more selection criteria, selects a camera from among the one or more candidate cameras, selects a light from among the one or more candidate lights, captures images of the workpiece multiple times in a virtual space while changing at least a portion of the positions of the selected cameras, the orientations of the selected cameras, the positions of the selected lights, and the orientations of the selected lights, analyzes the multiple images obtained by capturing the images, and outputs evaluation information for each of the multiple selected cameras, the positions and orientations of the selected cameras, the selected lights, and the combinations of the positions and orientations of the selected lights based on the analysis results.

[0008] According to the above disclosure, by using the system, a user (such as a line designer) can significantly reduce the man-hours required to select cameras and lighting and determine their positions and orientations. In other words, a user (such as a line designer) can significantly reduce the man-hours required to design an inspection jig.

[0009] In one aspect, the processor repeatedly captures images of the workpiece multiple times, the number of times being equal to the number of combinations of one or more candidate cameras and one or more candidate lighting, and analyzes the multiple images obtained by capturing the images.

[0010] According to the above disclosure, the system may perform an evaluation for all combinations of one or more candidate cameras and one or more candidate lighting.

[0011] In one aspect, the one or more selection criteria include some or all of the following: the use of the inspection camera and inspection lighting, environmental resistance settings, budget, delivery time, imaging time of the inspection camera, imaging distance, monochrome / color, hue, saturation, brightness of the inspection lighting, and shape information of the workpiece.

[0012] According to the above disclosure, the system may select one or more candidate cameras and one or more candidate lights that meet the user's needs based on one or more selection criteria.

[0013] In one aspect, the processor limits a set range of some of the positions and orientations of the selected camera based on selection criteria for the inspection camera included in the one or more selection criteria.

[0014] According to the above disclosure, the system may limit the setting range of a part of the position and orientation of the selected camera based on the selection criteria for the inspection camera, thereby enabling the system to output evaluation information more quickly.

[0015] In one aspect, the processor limits a range of settings for a portion of the position and orientation of the selected illumination based on selection criteria for the test illumination included in the one or more selection criteria.

[0016] According to the above disclosure, the system may limit the setting range of a part of the position and orientation of the selected lighting based on the selection criteria for the inspection lighting, thereby enabling the system to output evaluation information more quickly.

[0017] In one aspect, the evaluation information includes the model number, position, and orientation of the camera used to image the workpiece, the model number, position, and orientation of the lighting used to image the workpiece, an evaluation value of the analysis results, and the captured image.

[0018] According to the above disclosure, the system can present to the user the model number, position, and orientation of the camera used to image the workpiece, the model number, position, and orientation of the lighting used to image the workpiece, an evaluation value of the analysis results, and the captured image in association with each other.

[0019] In one aspect, capturing images of the workpieces in the virtual space includes capturing images of each of the plurality of workpieces, and the evaluation information includes comprehensive evaluation information when capturing images of each of the plurality of workpieces for each combination of the plurality of selected cameras, the positions and orientations of the selected cameras, the selected lighting, and the positions and orientations of the selected lighting.

[0020] According to the above disclosure, the system can present the user with comprehensive evaluation information of cameras and lighting for use with multiple inspection jigs.

[0021] In one aspect, the system further includes a communication unit that communicates with the shop server. The input unit receives order information for at least one of the camera and the lighting included in the evaluation information from the user. The communication unit transmits the order information to the shop server based on the input of the order information.

[0022] According to the above disclosure, the system can transmit order information to the shop server based on receiving an order input from the user, thereby allowing the user to easily obtain the camera and lighting.

[0023] According to another embodiment, there is provided a control method for a system for assisting in determining an inspection camera, an inspection light, and their placement for a workpiece. The control method includes accepting input of one or more selection criteria related to placement, selecting one or more candidate cameras as candidates for the inspection camera from among a plurality of cameras based on the one or more selection criteria, selecting one or more candidate lights as candidates for the inspection light from among a plurality of lights based on the one or more selection criteria, selecting a camera from among the one or more candidate cameras, selecting a light from among the one or more candidate lights, capturing images of the workpiece in a virtual space multiple times while changing at least a portion of the positions of the selected cameras, the orientations of the selected cameras, the positions of the selected lights, and the orientations of the selected lights, analyzing the multiple images obtained by capturing images, and outputting evaluation information for each of the multiple selected cameras, the positions and orientations of the selected cameras, the selected lights, and the combinations of the positions and orientations of the selected lights based on the analysis results.

[0024] According to the above disclosure, by using the system, a user (such as a line designer) can significantly reduce the man-hours required to select cameras and lighting and determine their positions and orientations. In other words, a user (such as a line designer) can significantly reduce the man-hours required to design an inspection jig.

[0025] According to yet another embodiment, there is provided a control program for a system for assisting in determining an inspection camera, an inspection light, and their placement for a workpiece. The control program causes the system to execute the following operations: accept input of one or more selection criteria related to placement, select one or more candidate cameras as candidates for the inspection camera from among a plurality of cameras based on the one or more selection criteria, select one or more candidate lights as candidates for the inspection light from among a plurality of lights based on the one or more selection criteria, select a camera from among the one or more candidate cameras, select a light from among the one or more candidate lights, capture images of the workpiece multiple times in a virtual space while changing at least a portion of the positions of the selected cameras, the orientations of the selected cameras, the positions of the selected lights, and the orientations of the selected lights, analyze the multiple images obtained by capturing the images, and output evaluation information for each of the multiple selected cameras, the positions and orientations of the selected cameras, the selected lights, and combinations of the positions and orientations of the selected lights based on the analysis results.

[0026] According to the above disclosure, by using the system, a user (such as a line designer) can significantly reduce the man-hours required to select cameras and lighting and determine their positions and orientations. In other words, a user (such as a line designer) can significantly reduce the man-hours required to design an inspection jig.

[0027] According to one embodiment, evaluation of images of a workpiece captured in virtual space can assist in determining the appropriate cameras, lighting, and their placement for inspecting the workpiece.

[0028] The above and other objects, features, aspects and advantages of the present disclosure will become apparent from the following detailed description of the disclosure taken in conjunction with the accompanying drawings.

[0029] 5 is a diagram showing an example of an outline of the operation of system 10 according to the present embodiment. FIG. 6 is a diagram showing a first example of the configuration of system 10. FIG. 7 is a diagram showing a second example of the configuration of system 10. FIG. 8 is a diagram showing an example of a series of setting screens for one or more selection criteria. FIG. 9 is a diagram showing an example of camera and lighting setting items that can be set on each screen shown in FIG. 4. FIG. 10 is a diagram showing an example of a setting screen for the camera position and orientation. FIG. 11 is a diagram showing an example of a setting screen for other lighting items. FIG. 12 is a diagram showing an example of a setting screen for an inspection tool. FIG. 13 is a diagram showing an example of a selection screen for one or more candidate cameras. FIG. 14 is a diagram showing an example of a selection screen for one or more candidate lighting. FIG. 15 is a diagram showing an example of a display screen for evaluation information for each imaging setting. FIG. 16 is a diagram showing an example of the overall processing procedure of system 10. FIG. 17 is a diagram showing an example of details of the process of accepting various settings of system 10.

[0030] Hereinafter, embodiments of the technical concept of the present disclosure will be described with reference to the drawings. In the following description, identical components are assigned the same reference numerals. Their names and functions are also the same. Therefore, detailed descriptions thereof will not be repeated. Furthermore, each embodiment, each modification, each software configuration, each hardware configuration, each function, each process, etc. may be selectively combined as appropriate.

[0031] <A. Application example>

[0032] (a. System Overview)

[0033] FIG. 1 is a diagram showing an example of an outline of the operation of a system 10 according to the present embodiment. Many factory lines are equipped with a product inspection jig. The inspection jig includes an inspection camera and an inspection light. The inspection jig uses the inspection camera to capture an image of the workpiece illuminated by the inspection light. The captured image is inspected by an inspection tool such as software. As an example, the inspection tool may detect scratches on the surface of the workpiece.

[0034] There is a wide variety of inspection cameras and inspection lighting available. As a result, line designers must find the optimal combination from the vast number of combinations available. Furthermore, the inspection results obtained vary depending on the position and orientation of the inspection camera and inspection lighting. Therefore, line designers must also consider the position and orientation (design of the inspection jig) for each camera and lighting combination. Selecting inspection cameras and inspection lighting, and determining the position and orientation of the inspection camera and inspection lighting, was an extremely burdensome task for designers.

[0035] Therefore, system 10 according to the present embodiment provides a function for selecting an inspection camera and inspection lighting simply by receiving one or more selection criteria from a user, and also provides a function for proposing the position and orientation of the inspection camera and inspection lighting.

[0036] More specifically, first, in step S100, the system 10 selects one or more candidate cameras from among a plurality of cameras registered in the system 10 based on one or more received selection criteria. Similarly, the system 10 selects one or more candidate lights from among a plurality of lights registered in the system 10 based on one or more received selection criteria. Selecting one or more candidate cameras means selecting one or more model numbers of the candidate cameras. Selecting one or more candidate lights means selecting one or more model numbers of the candidate lights.

[0037] Next, the system 10 captures an image of the workpiece 20 in the virtual space using a camera 30 selected from one or more candidate cameras and a light 40 selected from one or more candidate lights. At this time, the system 10 captures an image of the workpiece 20 in the virtual space multiple times while changing at least one of the position of the camera 30, the attitude of the camera 30, the position of the light 40, and the attitude of the light 40.

[0038] Hereinafter, the combination of the camera model number, camera position, camera attitude, lighting model number, lighting position, and lighting attitude may be referred to as the "imaging setting." The system 10 repeatedly captures images of the workpiece while changing some of the imaging setting items. That is, as described above, the system 10 repeatedly captures images of the workpiece while changing at least one of the position of the camera 30, the attitude of the camera 30, the position of the lighting 40, and the attitude of the lighting 40. The system 10 can capture images of the workpiece 20 any number of times while changing the positions and attitudes of the camera 30 and the lighting 40. Selecting a camera 30 from one or more candidate cameras means selecting the model number of the camera 30. Selecting a lighting 40 from one or more candidate lighting means selecting the model number of the lighting 40.

[0039] Next, in step S110, the system 10 generates evaluation information for the captured image. The evaluation information is linked to a combination of the model number of the camera 30, the position of the camera 30, the attitude of the camera 30, the model number of the lighting 40, the position of the lighting 40, and the attitude of the lighting 40. As an example, the system 10 may inspect the image of the workpiece 20 using an existing inspection tool (software, etc.). The evaluation information may include an evaluation of how accurately the existing inspection tool was able to identify anomalies or features of the workpiece.

[0040] Finally, in step S120, the system 10 outputs the evaluation information. The same number of pieces of evaluation information as the number of imaging settings are generated. As an example, suppose there are 10 candidate cameras and 10 candidate lights. Also, suppose there are 100 combinations of the position and orientation of each camera. Also, suppose there are 100 combinations of the position and orientation of each light. In this case, the number of imaging settings and pieces of evaluation information is "(number of candidate cameras * number of combinations of positions and orientations) * (number of candidate lights * number of combinations of positions and orientations) = (10 * 100) * (10 * 100) = 1,000,000."

[0041] As described above, the evaluation information is linked to a combination of the model number of the camera 30, the position of the camera 30, the attitude of the camera 30, the model number of the lighting 40, the position of the lighting 40, and the attitude of the lighting 40. Therefore, the line designer can obtain the optimal combination of inspection camera and inspection lighting simply by selecting the camera 30 and lighting 40 that are linked to the evaluation information with the best performance. Furthermore, the line designer can design an inspection jig based on the position and attitude of the camera 30 and the position and attitude of the lighting 40 that are linked to the evaluation information with the best performance.

[0042] As described above, system 10 selects candidate inspection cameras and inspection lighting devices based on one or more selection criteria. System 10 also captures images of the workpiece in a virtual space while varying the position and orientation of each combination of inspection camera and inspection lighting device. System 10 also inspects the images of the workpiece using an existing inspection tool. System 10 then generates evaluation information for the inspection results. A production line designer can select an inspection camera and inspection lighting device, and determine their position and orientation (design of the inspection jig), simply by inputting one or more selection criteria into system 10 and referring to the evaluation information.

[0043] (b. Terminology)

[0044] In this specification, a "system" may refer to any information processing device, such as one or more personal computers, workstations, server devices, tablets, or smartphones. A system may also refer to a combination of these. According to an embodiment, the system may be connected to input / output devices, such as a display and a keyboard, and used by a user. According to another embodiment, the system may provide various functions to a user as a service or web application via a network. In this case, the user may use the system's functions via a browser or client software installed on their own terminal. Furthermore, according to another embodiment, the system may be provided in a cloud environment. Furthermore, when a system is realized by a single device, the system may be interpreted as a device.

[0045] In this specification, a "line" refers to production equipment that is used in a factory or the like and is made up of robots, conveyor belts, sensors, etc. In one aspect, the line includes a line for manufacturing products or parts. In another aspect, the line includes a line for inspecting products or parts. In another aspect, the line includes a line for transporting or sorting products or parts. In yet another aspect, the line includes a combination of these.

[0046] In this specification, an "inspection jig" is a jig for inspecting a workpiece. The inspection jig includes an inspection camera that captures an image of the workpiece and an inspection light that illuminates the workpiece. The inspection jig also includes a frame for fixing the inspection camera and the inspection light. One or more inspection jigs may be installed on a production line.

[0047] In this specification, the term "inspection camera" refers to a camera used in an inspection jig. In addition, in this specification, the term "inspection camera" includes not only a camera in real space but also a camera used by system 10 in virtual space. As an example, the inspection camera may be a CCD (Charge Coupled Device) camera or a CMOS (Complementary Metal Oxide Semiconductor) camera. The inspection camera outputs the captured image to an inspection device.

[0048] In this specification, "inspection lighting" refers to lighting used in an inspection jig. In addition, in this specification, inspection lighting includes not only lighting in real space but also lighting used by the system 10 in virtual space. One example of inspection lighting is lighting including an LED (Light-emitting Diode).

[0049] In this specification, "one or more candidate cameras" refers to a set of inspection cameras selected by the system 10 based on one or more selection criteria. In practice, the system 10 selects one or more candidate camera model numbers based on one or more selection criteria. As an example, assume that 10,000 types of cameras are registered in the system 10. Also assume that the system 10 receives input of one or more selection criteria from a user. At that time, assume that the number of cameras that satisfy one or more selection criteria is 200. In this case, the system 10 selects the 200 types of cameras as one or more candidate cameras. The system 10 then evaluates the 200 types of cameras by capturing images of a workpiece in a virtual space.

[0050] In this specification, "one or more candidate illuminants" refers to a set of inspection illuminants selected by the system 10 based on one or more selection criteria. In practice, the system 10 selects one or more model numbers of candidate illuminants based on one or more selection criteria. As an example, assume that 10,000 types of illuminants are registered in the system 10. Also assume that the system 10 receives input of one or more selection criteria from a user. At that time, assume that the number of illuminants that satisfy one or more selection criteria is 100. In this case, the system 10 selects the 100 types of illuminants as one or more candidate illuminants. Then, the system 10 evaluates the 100 types of illuminants by capturing images of a workpiece in a virtual space.

[0051] In this specification, a "camera selected from one or more candidate cameras" refers to a camera selected from one or more candidate cameras for capturing images in a virtual space. In other words, the selected camera is a camera of a certain model number selected from a set of one or more candidate camera model numbers.

[0052] In this specification, "lighting selected from one or more candidate lighting" refers to lighting of a model number selected from one or more candidate lighting for imaging in a virtual space. In other words, the selected lighting is lighting of a certain model number selected from a set of one or more candidate lighting model numbers.

[0053] As an example, the one or more candidate cameras include camera A, camera B, and camera C. Furthermore, the one or more candidate lights include light X and light Y. In this case, the system 10 first selects camera A from among the one or more candidate cameras. Similarly, the system 10 selects light X from among the one or more candidate lights. Then, the system 10 images the workpiece in the virtual space using a combination of camera A and light X while changing their positions and orientations. After completing the imaging using camera A and light X, the system 10 next selects light Y from among the one or more candidate lights. Then, the system 10 images the workpiece in the virtual space using the combination of camera A and light Y while changing their positions and orientations. After completing the imaging using camera A and light Y, the system 10 further selects camera B from among the one or more candidate cameras. Similarly, the system 10 selects light X from among the one or more candidate lights. Then, the system 10 images the workpiece in the virtual space using the combination of camera B and light X while changing their positions and orientations. In this way, the system 10 captures an image of the workpiece while changing the combination of a camera selected from one or more candidate cameras and lighting selected from one or more candidate lighting.

[0054] In this specification, a "virtual space" is a three-dimensional (3D) space. As an example, in a virtual space, physical calculations can be used to reproduce the movements of objects, light sources, and the like that mimic those in real space. In this specification, "capturing" may refer to taking a still image. Furthermore, capturing an image may refer to taking a video.

[0055] As used herein, "one or more selection criteria" may include any criteria for selecting an inspection camera and an inspection lighting. For example, the one or more selection criteria may include information about the workpiece to be inspected, the inspection content (application), the inspection tool, the budget, the delivery date, environmental resistance information, and color / monochrome settings. The one or more selection criteria may also include the imaging distance, the imaging time, the range of inspection camera positions, and the range of inspection camera orientations. The one or more selection criteria may also include the distance from the inspection lighting to the workpiece, the range of inspection lighting positions, and the range of inspection lighting orientations. Furthermore, the one or more selection criteria may also include any settings of the inspection lighting, such as the hue, saturation, and brightness.

[0056] In this specification, "position" refers to coordinates in three-dimensional space (or virtual space). As an example, the positions of the inspection camera and inspection lighting may be expressed as X, Y, and Z coordinate values. In this specification, "orientation" refers to the angle of each rotation axis in three-dimensional space (or virtual space). As an example, the orientation of the inspection camera and inspection lighting may be expressed as yaw, pitch, and roll values. Position and orientation may be collectively referred to as "configuration."

[0057] In this specification, the term "imaging settings" refers to a combination of the camera model, camera position, camera attitude, lighting model, lighting position, and lighting attitude. In addition, in some aspects, the imaging settings may also include lighting settings (hue, color, brightness, etc.).

[0058] <B. System Configuration>

[0059] FIG. 2 is a diagram illustrating a first example of the configuration of system 10. In the example of FIG. 2, system 10 is realized as a single device 200. In one aspect, system 10 can realize the processing disclosed in this specification by executing a program on the hardware illustrated in FIG. 2. Device 200 includes a processor 201, a RAM (Random Access Memory) 202, a storage unit 203, an external device connection unit 204, a communication unit 205, an input unit 206, an output unit 207, and a bus 208. Each component of device 200 is configured to be able to communicate with each other via bus 208.

[0060] The processor 201 may execute programs for implementing various functions of the device 200. The processor 201 may be configured, for example, with at least one integrated circuit. According to an embodiment, the integrated circuit may include at least one central processing unit (CPU), at least one graphics processing unit (GPU), at least one field programmable gate array (FPGA), at least one application specific integrated circuit (ASIC), at least one artificial intelligence (AI) chip, or a combination thereof.

[0061] RAM 202 stores programs executed by processor 201 and data referenced by processor 201. In one aspect, RAM 202 may be implemented by a dynamic random access memory (DRAM), a static random access memory (SRAM), or the like.

[0062] Storage unit 203 is a non-volatile memory that stores programs executed by processor 201 and data referenced by processor 201. Processor 201 executes programs read from storage unit 203 to RAM 202 and references data read from storage unit 203 to RAM 202. In one aspect, storage unit 203 can be realized by a hard disk drive (HDD), a solid state drive (SSD), an erasable programmable read only memory (EPROM), an electrically erasable programmable read only memory (EEPROM), a flash memory, or the like.

[0063] The storage unit 203 stores a virtual space generation unit 220, an arrangement unit 230, an imaging unit 240, an image analysis unit 250, and an evaluation unit 260. The virtual space generation unit 220, the arrangement unit 230, the imaging unit 240, the image analysis unit 250, and the evaluation unit 260 can be realized as programs.

[0064] The virtual space generation unit 220 generates a virtual space. The virtual space generation unit 220 may also generate a camera, lighting, a workpiece, and any other object to be placed in the virtual space. The virtual space generation unit 220 may also perform physical calculations in the virtual space.

[0065] The placement unit 230 determines the placement of each object in the virtual space. As an example, the placement unit 230 determines the placement (position and orientation) of a camera and a light in the virtual space. The placement unit 230 executes a process of determining the position and orientation of the camera 30 and the light 40 in FIG. 1 . In a certain aspect, the placement unit 230 may output a placement instruction for the camera 30 and the light 40 to the virtual space generation unit 220. In this case, the virtual space generation unit 220 places the camera 30 and the light 40 in the virtual space based on the instruction.

[0066] The imaging unit 240 captures an image of the workpiece using a camera in the virtual space. In a certain aspect, the imaging unit 240 may output an imaging instruction to the virtual space generation unit 220. In this case, the virtual space generation unit 220 captures an image of the workpiece using a camera in the virtual space and returns the image to the imaging unit 240. The imaging unit 240 outputs the obtained image to the image analysis unit 250. The imaging unit 240 captures an image of the workpiece each time the placement unit 230 changes the position and orientation of the camera and lighting in the virtual space.

[0067] The image analysis unit 250 analyzes one or more obtained images using an arbitrary inspection tool, and outputs, for each image, the results of determining whether the workpiece has scratches, the results of extracting the feature quantities of the workpiece, and the like.

[0068] The evaluation unit 260 outputs evaluation information. The evaluation information includes a plurality of evaluation values. The evaluation values ​​are output values ​​of the image analysis unit 250. Each evaluation value is linked to at least the captured image, the model number, position, and orientation of the camera, and the model number, position, and orientation of the lighting.

[0069] External device connection unit 204 can be connected to any external device such as a printer, a scanner, an external HDD, etc. In one aspect, external device connection unit 204 can be realized by a USB (Universal Serial Bus) terminal or the like.

[0070] Communication unit 205 is connected to other devices via a wired network or a wireless network. In one aspect, communication unit 205 may be implemented by a wired local area network (LAN) port, a Wi-Fi (registered trademark) (Wireless Fidelity) module, or the like. In another aspect, communication unit 205 may transmit and receive data using a communication protocol such as TCP / IP (Transmission Control Protocol / Internet Protocol) or UDP (User Datagram Protocol). Communication unit 205 may transmit various screens shown in FIGS. 4 to 12 to other devices.

[0071] The input unit 206 may be connected to any input device 216, such as a keyboard, a mouse, a touchpad, or a gamepad. In one aspect, the input unit 206 may be realized by a USB terminal, a PS / 2 terminal, a Bluetooth (registered trademark) module, or the like. In one aspect, the input unit 206 may be integrated with the input device 216. As an example, the input unit 206 may receive input from a user of one or more selection criteria regarding the placement of an inspection camera and an inspection light for a workpiece.

[0072] Output unit 207 may be connected to any output device 217, such as a cathode ray tube display, a liquid crystal display, or an organic electroluminescence (EL) display. In one aspect, output unit 207 may be implemented by a USB terminal, a D-sub terminal, a DVI (Digital Visual Interface) terminal, an HDMI (High-Definition Multimedia Interface) terminal, a DisplayPort terminal, or the like. In one aspect, output unit 207 may be integrated with output device 217. The various screens shown in FIGS. 4 to 12 may be displayed on output device 217.

[0073] 3 is a diagram showing a second example of the configuration of the system 10. In the example of FIG. 3, the system 10 is realized as two devices 300 and 310. The device 300 has the same hardware configuration as the device 200. The storage unit 203 of the device 300 stores a placement unit 230, an image analysis unit 250, and an evaluation unit 260. The device 310 stores a virtual space generation unit 220 and an imaging unit 240. The device 310 may have the same hardware configuration as the device 300.

[0074] In the example of FIG. 3 , device 300 transmits position and orientation information of the camera and lighting to device 310. Device 300 may transmit one or more pieces of position and orientation information to device 310. Device 310 places the camera and lighting in the virtual space based on the received position and orientation information of the camera and lighting. Device 310 then uses the camera to capture an image of the workpiece and transmits one or more images to device 300. Device 300 analyzes the one or more images and generates evaluation information.

[0075] As described above, the system 10 is a system that assists in determining an inspection camera, an inspection light, and their placement for a workpiece. The system 10 includes a memory unit 203 that stores information about multiple cameras and multiple lights. The system 10 also includes an input unit 206 that accepts input of one or more selection criteria related to placement. The system 10 also includes a processor 201. The processor 201 selects one or more candidate cameras as candidates for the inspection camera from among the multiple cameras based on the one or more selection criteria. The processor 201 also selects one or more candidate lights as candidates for the inspection light from among the multiple lights based on the one or more selection criteria. The processor 201 also selects a camera from among the one or more candidate cameras. The processor 201 also selects a light from among the one or more candidate lights. The processor 201 also captures images of the workpiece multiple times in a virtual space while changing at least a portion of the position of the selected camera, the orientation of the selected camera, the position of the selected light, and the orientation of the selected light. The processor 201 also analyzes the multiple images obtained by capturing the images. Furthermore, based on the analysis results, the processor 201 outputs evaluation information for each of the multiple selected cameras, the positions and orientations of the selected cameras, the selected lighting, and the combinations of the positions and orientations of the selected lighting.

[0076] Furthermore, the processor 201 repeatedly captures images of the workpiece multiple times, and analyzes the multiple images obtained by capturing the images, for each combination of one or more candidate cameras and one or more candidate lighting devices.

[0077] C. Camera and Lighting Settings

[0078] Next, one or more selection criteria for selecting one or more candidate cameras and one or more candidate lights will be described with reference to FIGS. 4 to 9 . The system 10 may acquire one or more selection criteria via the screens shown in FIGS. 4 to 9 . In one aspect, the various screens shown in FIGS. 4 to 9 may be displayed on the output device 217. In this case, the system 10 may function as a standalone device. In another aspect, the various screens shown in FIGS. 4 to 9 may be transmitted to another device via the communication unit 205. In this case, the system 10 may function as a system that provides a web application or the like.

[0079] 4 shows an example of a series of setting screens for one or more selection criteria. Screens 410, 420, 430, 440, 450, 460, and 470 may be provided as a user interface (UI) of an application executed by system 10. System 10 accepts input of one or more selection criteria related to the placement of the inspection camera and inspection lighting via these screens.

[0080] In one aspect, the screens 410, 420, 430, 440, 450, 460, and 470 may be provided as screens that transition in sequence. In another aspect, the screens 410, 420, 430, 440, 450, 460, and 470 may be provided as a single screen.

[0081] Screen 410 is a setting screen for the camera and lighting use or inspection details. A user can input any camera and lighting use or inspection details into system 10 via screen 410. The camera and lighting use or inspection details are part of one or more selection criteria. Examples of camera and lighting use or inspection details include "inspecting scratches on the edge or metal surface of a workpiece," "inspecting irregularly shaped workpieces," "inspecting the shape of a workpiece that should not be affected by the condition of the workpiece surface," and "inspecting highly reflective workpieces."

[0082] Screen 420 is a camera color and monochrome setting screen. A user can input the camera color and monochrome settings into system 10 via screen 420. The camera color and monochrome settings are part of one or more selection criteria. The camera color and monochrome settings can include, by way of example, "color," "color NIR (near infrared)," "monochrome," "monochrome NIR," and "monochrome SWIR (short wave infrared)." Alternatively, the camera color and monochrome settings do not have to be specified.

[0083] Screen 430 is an environmental resistance setting screen. A user can input environmental resistance settings for the camera and lighting into system 10 via screen 440. The environmental resistance settings are part of one or more selection criteria. Examples of environmental resistance settings may include temperature range, waterproof performance, dustproof performance, shock resistance, and vibration resistance. Also, the environmental resistance settings do not need to be specified.

[0084] Screen 440 is a setting screen for the imaging distance (the distance of the camera from the workpiece). A user can input the imaging distance setting to system 10 via screen 440. The imaging distance setting is part of one or more selection criteria. For example, the imaging distance setting can be configured to be set in millimeters or centimeters. Alternatively, the imaging distance setting does not have to be specified.

[0085] Screen 450 is a setting screen for setting a budget for cameras and lighting. A user can input a budget setting to system 10 via screen 450. The budget setting is part of one or more selection criteria. As an example, the budget setting can be configured to be set in units of 1 yen. Alternatively, the budget setting need not be specified.

[0086] Screen 460 is a setting screen for the delivery date of the camera and the lighting. The user can input the delivery date setting to the system 10 via screen 460. The setting of the estimated delivery date is part of one or more selection criteria. As an example, the delivery date setting can be configured to be set in units of one day. Alternatively, the delivery date setting need not be specified.

[0087] Screen 470 is a setting screen for the shape of the workpiece. A user can input the setting of the shape of the workpiece to system 10 via screen 470. The setting of the shape of the workpiece is part of one or more selection criteria. As an example, the setting of the shape of the workpiece can be configured to be set by reading CAD (Computer Aided Design) data of the workpiece.

[0088] In one aspect, the system 10 may output only a portion of the screens 410, 420, 430, 440, 450, 460, and 470. In this case, the system 10 acquires, as one or more selection criteria, items input via the output portion of the screen.

[0089] In another aspect, the system 10 may provide any setting screen related to the camera and lighting other than the screen shown in FIG. 4 . Furthermore, the system 10 may acquire any setting as part of one or more selection criteria via these any setting screens. As one example, a setting screen for the number of pixels of the camera may be provided as a UI for an application executed by the system 10. As another example, a setting screen for the type of lighting (such as a ring light) may be provided as a UI for an application executed by the system 10.

[0090] FIG. 5 is a diagram showing an example of camera and lighting setting items that can be set on each screen shown in FIG. 4 . Setting item 510 is an item common to cameras and lighting. Setting item 510 includes, for example, various uses (inspection contents). Setting item 520 is an item for cameras. Setting item 520 may include, for example, model number, type, number of pixels, color / monochrome, imaging time, environmental resistance, price, etc. Setting item 530 is an item for lighting. Setting item 530 may include, for example, model number, type, environmental resistance, price, etc.

[0091] FIG. 6 is a diagram illustrating an example of a camera position and orientation setting screen. Screen 600 may be provided as a UI of an application executed by system 10. Screen 600 is a camera position and orientation setting screen. A user may input settings for the camera position range and orientation range to system 10 via screen 600. The settings for the camera position range and orientation range are part of one or more selection criteria. System 10 moves the camera in virtual space so that the camera is within the camera position range and orientation range acquired via screen 600. In other words, system 10 may limit the setting ranges for some or all of the position and orientation of the selected camera based on the selection criteria (camera position range and orientation range) related to the inspection camera included in the one or more selection criteria.

[0092] Screen 600 includes an input item 610 for setting the camera position and an input item 620 for setting the camera attitude. The input item 610 for setting the camera position includes the ranges for each of the X, Y, and Z axes, and the placement interval of the camera. In the example of FIG. 6 , the camera position ranges from 0 to 200 mm in the X axis direction, and the placement interval is 10 mm. The input item 620 for setting the camera attitude includes the ranges for each of the yaw, pitch, and roll angles, and the placement interval of the camera. In the example of FIG. 6 , the camera yaw ranges from -10 to 10°, and the placement interval is 1°. Screen 600 also includes a preview 630 for confirming the input camera position and orientation. Preview 630 displays the camera and workpiece in the virtual space. In one aspect, preview 630 may display the input camera position range and the input camera orientation range. In another aspect, preview 630 may display the camera, lighting, and workpiece in the virtual space.

[0093] In one aspect, screen 600 may be included in the transition of each screen shown in Fig. 4. In another aspect, screen 600 may be provided as a single screen together with screens 410, 420, 430, 440, 450, 460, and 470.

[0094] As described above, the processor 201 (system 10) may limit the setting ranges of some or all of the positions and orientations of the selected camera based on the selection criteria (position ranges and orientation ranges of the camera) for the inspection camera included in one or more selection criteria.

[0095] FIG. 7 is a diagram illustrating an example of a setting screen for the position and orientation of a light. Screen 700 may be provided as a UI of an application executed by system 10. Screen 700 is a setting screen for the position and orientation of a light. A user may input settings for the position range and orientation range of the light to system 10 via screen 700. The settings for the position range and orientation range of the light are part of one or more selection criteria. System 10 moves the light in the virtual space so that the light is within the position range and orientation range of the light acquired via screen 700. In other words, system 10 may limit the setting ranges for some or all of the position and orientation of the selected light based on the selection criteria (the position range and orientation range of the light) related to the test light included in the one or more selection criteria.

[0096] The screen 700 includes an input item 710 for setting the position of the light and an input item 720 for setting the attitude of the light. The input item 710 for setting the position of the light includes ranges in the X, Y, and Z axes and the placement interval of the light. In the example of FIG. 7 , the position of the light is in the range of "0 to 200 mm" in the X axis direction, and the placement interval is "10 mm." The input item 720 for setting the attitude of the light includes ranges of the yaw, pitch, and roll angles and the placement interval of the light. In the example of FIG. 7 , the yaw range of the light is "-10 to 10°" and the placement interval is "1°." The screen 700 also includes a preview 730 for confirming the input position and attitude of the light. The preview 730 displays the light and workpiece in the virtual space. In one aspect, the preview 730 may display the input position range of the light and the input attitude range of the light. In another aspect, the preview 730 may display the camera, light, and workpiece in the virtual space.

[0097] In one aspect, screen 700 may be included in the transition of each screen shown in Fig. 4 together with screen 600. In another aspect, screen 700 may be provided as a single screen together with screens 410, 420, 430, 440, 450, 460, 470, and 600.

[0098] As described above, the processor 201 (system 10) may limit the setting ranges of some or all of the positions and orientations of the selected illumination based on the selection criteria (the range of positions and the range of orientations of the illumination) for the test illumination included in one or more selection criteria.

[0099] 8 is a diagram showing an example of a setting screen for other lighting items. Screen 800 may be provided as a UI of an application executed by system 10. Screen 800 is a setting screen for other lighting items. A user may input any other lighting item settings, such as hue, saturation, brightness, and intensity, to system 10 via screen 800. The setting for other lighting items is part of one or more selection criteria.

[0100] Screen 800 includes lighting hue, saturation, brightness, and intensity inputs 810. In one aspect, screen 800 may include lighting hue, saturation, and brightness graphical inputs 820.

[0101] In one aspect, screen 800 may be included in the transition of each screen shown in Fig. 4 together with screens 600 and 700. In another aspect, screen 800 may be provided as a single screen together with screens 410, 420, 430, 440, 450, 460, 470, 600, and 700.

[0102] FIG. 9 is a diagram showing an example of an inspection tool setting screen. Screen 900 may be provided as a UI of an application executed by system 10. Screen 900 is an inspection tool setting screen. A user may input settings of a tool to be used for inspecting an image of a workpiece to system 10 via screen 900. The settings of a tool to be used for inspecting an image of a workpiece are part of one or more selection criteria. Screen 900 includes setting items for one or more tools.

[0103] In one aspect, screen 900 may be included in the transition of each screen shown in Fig. 4 together with screens 600, 700, and 800. In another aspect, screen 900 may be provided as a single screen together with screens 410, 420, 430, 440, 450, 460, 470, 600, 700, and 800.

[0104] In one aspect, the system 10 may output only a portion of the screens 410, 420, 430, 440, 450, 460, 470, 600, 700, 800, and 900. In this case, the system 10 acquires, as one or more selection criteria, items input via the portion of the screens that have been output.

[0105] As described above, the one or more selection criteria may include some or all of the following: the purpose of the inspection camera and inspection lighting, environmental resistance settings, budget, delivery time, imaging time of the inspection camera, imaging distance, monochrome / color, hue, saturation, brightness of the inspection lighting, and shape information of the workpiece.

[0106] As described above, the system 10 acquires one or more selection criteria via the screens shown in FIGS. 4 to 9. Furthermore, a plurality of cameras and lighting fixtures are registered in the system 10 in advance. The system 10 selects one or more candidate cameras and one or more candidate lighting fixtures from among the plurality of cameras and lighting fixtures based on the one or more selection criteria. In other words, the system 10 selects cameras and lighting fixtures that satisfy the one or more selection criteria.

[0107] D. Selecting and Evaluating Camera and Lighting Combinations

[0108] Next, one or more candidate cameras and one or more candidate lights selected based on one or more selection criteria will be described with reference to FIGS. 10 and 11 . Furthermore, evaluation information for each imaging setting will be described with reference to FIG. 12 . In one aspect, the various screens shown in FIGS. 10 to 12 may be displayed on the output device 217. In this case, the system 10 may function as a standalone device. In another aspect, the various screens shown in FIGS. 10 to 12 may be transmitted to another device via the communication unit 205. In this case, the system 10 may function as a system that provides a web application or the like.

[0109] 10 is a diagram showing an example of a selection screen for one or more candidate cameras. Screen 1000 may be provided as a UI for an application executed by system 10. Screen 1000 is a selection screen for one or more candidate cameras. Screen 1000 displays information about some or all of the cameras registered in system 10.

[0110] Screen 1000 includes a check item 1010 and a specification item 1020 for each camera. Check item 1010 indicates whether each camera has been selected as one or more candidate cameras to be evaluated. The specification item 1020 for each camera includes optional specifications for each camera. The specifications for each camera include optional specifications such as the model number, appearance (photo), camera type, lens mount type, number of pixels, color / monochrome, and capture time. In some aspects, the specification item 1020 for each camera may include other specifications not shown in FIG. 10 .

[0111] Screen 1000 is output after system 10 accepts input of one or more selection criteria. More specifically, system 10 accepts input of one or more selection criteria via each of the screens shown in FIGS. 4 to 9. Next, system 10 determines one or more candidate cameras based on the one or more selection criteria. Then, system 10 outputs screen 1000. In screen 1000, check items 1010 in rows corresponding to one or more candidate cameras are checked.

[0112] Screen 1000 is configured to allow editing of check items 1010. The user can reduce the number of candidate cameras by one or more by unchecking check items 1010. Conversely, the user can increase the number of candidate cameras by one or more by checking check items 1010. In some aspects, the user may select one or more candidate cameras by directly operating screen 1000 without using each of the screens shown in FIGS. 4 to 9. System 10 determines the selection of one or more candidate cameras by pressing a decision button (not shown) or the like on screen 1000.

[0113] 11 is a diagram showing an example of a selection screen for one or more candidate lights. Screen 1100 may be provided as a UI of an application executed by system 10. Screen 1100 is a selection screen for one or more candidate lights. Screen 1100 displays information about some or all of the lights registered in system 10.

[0114] The screen 1100 includes a check item 1110 and a specification item 1120 for each lighting fixture. The check item 1110 indicates whether each lighting fixture has been selected as one or more candidate lighting fixtures to be evaluated. The specification item 1120 for each lighting fixture includes optional specifications for each lighting fixture. The specifications for each lighting fixture include optional specifications such as the model number, lighting color, power, external dimensions, and type of lighting controller. In some aspects, the specification item 1120 for each lighting fixture may include other specifications not shown in FIG. 11 .

[0115] The screen 1100 is output after the system 10 receives input of one or more selection criteria. More specifically, the system 10 receives input of one or more selection criteria via each of the screens shown in FIGS. 4 to 9. Next, the system 10 determines one or more candidate illuminants based on the one or more selection criteria. The system 10 then outputs the screen 1100. In the screen 1100, check boxes 1110 in rows corresponding to one or more candidate illuminants are checked.

[0116] The screen 1100 is configured to allow editing of check items 1110. The user can remove one or more candidate illuminants by unchecking the check items 1110. Conversely, the user can add one or more candidate illuminants by checking the check items 1110. In some aspects, the user may select one or more candidate illuminants by directly operating the screen 1100 without using each of the screens shown in FIGS. 4 to 9. The system 10 determines the selection of one or more candidate illuminants by pressing a decision button (not shown) or the like on the screen 1100.

[0117] In one aspect, the screens provided by system 10 may transition in the order of the screens in Figures 4 to 9, screen 1000, and screen 1100. In another aspect, the screens provided by system 10 may transition in the order of the screens in Figures 4 to 9, screen 1100, and screen 1000. In another aspect, screen 1000 and screen 1100 may be displayed simultaneously.

[0118] FIG. 12 is a diagram illustrating an example of a display screen displaying evaluation information for each imaging setting. As described above, the system 10 determines a combination of one camera (a camera with a certain model number) selected from one or more candidate cameras and one lighting device (a lighting device with a certain model number) selected from one or more candidate lighting devices in a brute-force manner. For each combination of the camera model number and lighting device model number, the system 10 images the workpiece in a virtual space while changing the possible positions and orientations of the camera and lighting device in a brute-force manner. That is, the system 10 images the workpiece for each imaging setting. The system 10 then analyzes each of the multiple images obtained using an inspection tool and outputs an evaluation value. Each of the multiple evaluation values ​​obtained is associated with the imaging setting, i.e., the camera (model number), camera position, camera orientation, lighting device (model number), lighting position, and lighting orientation.

[0119] Hereinafter, for the sake of distinction, the value of the tool's analysis result will be referred to as an "evaluation value," and the information included in each row of the list displayed on screen 1200 will be referred to as "evaluation information." The "evaluation information" includes at least the evaluation value, camera (model number), camera position, camera attitude, lighting (model number), lighting position, and lighting attitude. The evaluation information may further include lighting settings, captured images, 3D images showing the placement of the camera and lighting, and the total price of the camera and lighting.

[0120] The system 10 displays these multiple pieces of evaluation information on a screen 1200. The screen 1200 includes a list of evaluation information. In the example of Fig. 12, the screen 1200 includes five pieces of evaluation information (rows). However, this is merely an example, and the screen 1200 may include any number of pieces of evaluation information.

[0121] Screen 1200 includes the following items: evaluation value 1210, camera model number 1220, camera position and orientation 1230, lighting model number 1240, lighting position and orientation 1250, lighting settings 1260, captured image 1270, 3D image 1280, total price 1290, and order check item 1295.

[0122] The evaluation value 1210 is an item of evaluation values ​​by one or more inspection tools. The evaluation value 1210 may include evaluation values ​​by each of a plurality of inspection tools. The evaluation information (rows of the screen 1200) may be configured to be sortable by the evaluation value by each of the plurality of inspection tools.

[0123] The camera model number 1220 is an item for the model number of the camera. The camera position and orientation 1230 is an item for the position and orientation of the camera. The camera position includes the coordinate values ​​of the camera in the virtual space. The camera orientation includes the values ​​of the yaw, pitch, and roll of the camera.

[0124] Lighting model number 1240 is an item for the model number of the lighting. Lighting position and orientation 1250 is an item for the position and orientation of the lighting. The lighting position includes the coordinate values ​​of the lighting in the virtual space. The lighting orientation includes the yaw, pitch, and roll values ​​of the lighting. Lighting settings 1260 include optional settings for each lighting (brightness, which LEDs to light, etc.). The lighting settings that can be set may differ depending on the lighting model.

[0125] The captured image 1270 is an item of an image of a work captured in virtual space. The 3D image 1280 is an item of an image of the arrangement of the camera and lighting in virtual space. The total price 1290 is an item of the total price of the camera and lighting.

[0126] The order check item 1295 is an item for determining whether to order cameras and lights with the model numbers in the corresponding row. The system 10 may order cameras and lights for rows in which the order check item 1295 is checked from an external shop server. In one aspect, the order check item 1295 may be configured to allow the ordering of cameras and lights to be set individually. The order check item 1295 may also be configured to allow the number of cameras and lights to be ordered to be set individually. The order check item 1295 may also be configured to allow the number of cameras and lights to be ordered to be set individually. The system 10 may transmit order information (one or more product model numbers, order quantity, etc.) to the shop server based on the input contents of the order check item 1295.

[0127] By referring to screen 1200, the user can easily understand the combination of the camera model number, camera position and orientation, lighting model number, and lighting position and orientation that has a high evaluation value (high inspection accuracy). The user can also order the camera and lighting that have a high evaluation value (high inspection accuracy) from the shop via screen 1200. Furthermore, the user can easily design an inspection jig based on the camera position and orientation and lighting position and orientation that have a high evaluation value (high inspection accuracy).

[0128] In some aspects, capturing images of workpieces in a virtual space may include capturing images of each of multiple workpieces. As an example, assume that a line under design includes multiple inspection jigs. In this case, the system 10 captures each of the multiple workpieces using the same imaging settings. The same imaging settings refer to the same camera model number, the same camera position and orientation, the same lighting model number, and the same lighting position and orientation. The system 10 then uses an inspection tool to output an evaluation value for each of the captured images of the multiple workpieces. Next, the system 10 calculates an overall evaluation value for each of the captured images of the multiple workpieces. The "overall evaluation value" may be the average, median, or integrated value of each of the five captured images of the workpieces. The overall evaluation value is associated with the settings at the time of capture, i.e., the camera (model number), camera position, camera orientation, lighting (model number), lighting position, and lighting orientation. Finally, the system 10 outputs a screen 1200 including the overall evaluation value instead of the evaluation value. In this case, each row of the screen 1200 can be referred to as "overall evaluation information" for the analysis results of the captured images of each of the multiple workpieces. Alternatively, the comprehensive evaluation information may be simply called evaluation information.

[0129] As described above, the evaluation information includes the model number, position, and orientation of the camera used to image the workpiece, the model number, position, and orientation of the lighting used to image the workpiece, an evaluation value of the analysis results, and the captured image.

[0130] As described above, capturing images of the workpieces in the virtual space may include capturing images of each of the plurality of workpieces. In this case, the evaluation information includes comprehensive evaluation information obtained when capturing images of each of the plurality of workpieces for each combination of the plurality of selected cameras, the positions and orientations of the selected cameras, the selected lighting, and the positions and orientations of the selected lighting.

[0131] As described above, the system 10 includes a communication unit 205 that communicates with the shop server. The input unit 206 accepts input of order information for at least one of the camera and the lighting included in the evaluation information from the user. The communication unit 205 transmits the order information to the shop server based on the input of the order information. In some aspects, the communication unit 205 may transmit the order information to the shop's email address, etc.

[0132] <E. System Processing Procedure>

[0133] Next, a processing procedure from when system 10 receives input of one or more selection criteria to when it outputs evaluation information will be described with reference to Figures 13 and 14. In one aspect, processor 201 may load a program for performing the processing of Figures 13 and 14 from storage unit 203 into RAM 202 and execute the program. In another aspect, some or all of the processing may be realized as a combination of circuit elements configured to perform the processing.

[0134] 13 is a diagram showing an example of the overall processing procedure of the system 10. By executing the processing of FIG. 13, the system 10 can generate evaluation information for each imaging setting. Furthermore, the system 10 can transmit order information to an external shop server based on receiving an input of a product order from a user.

[0135] In step S1305, the system 10 accepts various settings. The processing of this step corresponds to the processing described with reference to FIGS. 4 to 9. The various settings are part of one or more selection criteria. In one aspect, the system 10 may accept the various settings via the input device 216. In another aspect, the system 10 may receive the various settings from another device via the communication unit 205.

[0136] In step S1310, the system 10 selects a camera. More specifically, the system 10 selects one or more candidate cameras based on one or more selection criteria. Furthermore, the system 10 selects one camera (model number) from the one or more candidate cameras.

[0137] In step S1315, the system 10 determines the settings of the camera selected in step S1310. The settings here are the position and orientation of the camera. When the system 10 repeatedly executes this step, it changes at least one of the position and orientation of the camera.

[0138] In step S1320, the system 10 selects a lighting fixture. More specifically, the system 10 selects one or more candidate lighting fixtures based on one or more selection criteria. Furthermore, the system 10 selects one lighting fixture (model number) from the one or more candidate lighting fixtures.

[0139] In step S1325, the system 10 determines the settings of the light selected in step S1320. The settings here refer to the position and orientation of the light. When the system 10 repeatedly executes this step, it changes at least one of the position and orientation of the light. In one aspect, the settings of the light may include color, hue, and brightness. In this case, when the system 10 repeatedly executes this step, it changes at least one of the position, orientation, color, hue, and brightness of the light.

[0140] In step S1330, the system 10 captures an image of the workpiece. More specifically, the system 10 captures an image of the workpiece in the virtual space based on the image capture settings determined in the processes of steps S1310 to S1325.

[0141] In step S1335, the system 10 analyzes the image. More specifically, the system 10 analyzes the image of the workpiece using any one or more inspection tools.

[0142] In step S1340, the system 10 evaluates the analysis results. More specifically, the system 10 outputs an evaluation value, which is a value of the analysis result of each inspection tool. The system 10 also associates the evaluation value with the imaging setting to generate evaluation information for each imaging setting.

[0143] In step S1345, the system 10 determines whether all lighting settings have been selected. More specifically, the system 10 determines whether the captured images of the workpiece have been evaluated using all of the combinations of positions and orientations to be verified for the lighting (model number) selected in step S1320. If the system 10 determines that all lighting settings have been selected (YES in step S1345), the system 10 transfers control to step S1350. If not (NO in step S1345), the system 10 transfers control to step S1325.

[0144] In step S1350, the system 10 determines whether all the lighting fixtures have been selected. More specifically, the system 10 determines whether the captured image of the workpiece has been evaluated using all of one or more candidate lighting fixtures. If the system 10 determines that all the lighting fixtures have been selected (YES in step S1350), the system 10 transfers control to step S1355. If not (NO in step S1350), the system 10 transfers control to step S1320.

[0145] In step S1355, the system 10 determines whether all camera settings have been selected. More specifically, the system 10 determines whether the captured images of the workpiece have been evaluated using all of the combinations of positions and orientations to be verified for the camera (model number) selected in step S1315. If the system 10 determines that all camera settings have been selected (YES in step S1355), the system 10 transfers control to step S1360. If not (NO in step S1355), the system 10 transfers control to step S1315.

[0146] In step S1360, the system 10 determines whether all cameras have been selected. More specifically, the system 10 determines whether the captured images of the workpiece have been evaluated using all of the one or more candidate cameras. If the system 10 determines that all cameras have been selected (YES in step S1360), the system 10 transfers control to step S1365. If not (NO in step S1360), the system 10 transfers control to step S1310.

[0147] In step S1365, the system 10 displays the evaluation information. The processing of this step corresponds to the processing of outputting the screen 1200. In step S1370, the system 10 proposes an inspection jig configuration. In one aspect, the system 10 may propose an inspection jig configuration associated with the best evaluation value. In another aspect, the system 10 may propose multiple inspection jig configurations in descending order of evaluation value. The inspection jig configuration includes at least the camera model number, camera position, camera attitude, lighting model number, lighting position, and lighting attitude. The system 10 does not need to perform the processing of this step (proposal processing).

[0148] In step S1375, the system 10 accepts an order for the camera and the lighting. For example, the system 10 accepts the order for the camera and the lighting via the screen 1200. In one aspect, the system 10 may accept an order for only the camera. In another aspect, the system 10 may accept an order for only the lighting.

[0149] In step S1380, the system 10 places an order for the camera and the lighting. More specifically, the system 10 transmits order information to the shop server. At that time, the system 10 may include part of the imaging settings (the position and orientation of each of the camera and the orientation) in the order information. In this case, the distributor may print information such as the mounting position of the camera on the jig based on the imaging settings on the packaging of the camera or the lighting. The assembly staff on the line can easily assemble the jig by referring to the information.

[0150] FIG. 14 is a diagram showing an example of details of the process of accepting various settings of the system 10. The process of FIG. 14 is executed as internal processing of step S1305. The process of FIG. 14 corresponds to the input processing on each screen of FIG. 4. In a certain aspect, the system 10 may accept any setting other than steps S1410 to S1450. As an example, the system 10 may accept the imaging distance, delivery date, workpiece shape, etc.

[0151] 7 to 9. In this case, the system 10 can narrow the setting range of the camera position and orientation to be changed in step S1315. Similarly, the system 10 can narrow the setting range of the lighting position and orientation to be changed in step S1325.

[0152] In step S1410, the system 10 accepts input of the purpose of the camera and lighting (inspection content). In step S1420, the system 10 accepts input of the selection of monochrome or color for the camera. In step S1430, the system 10 accepts input of the environment-resistant settings for the camera and lighting. In step S1440, the system 10 accepts input of the imaging time. In step S1450, the system 10 accepts input of the budget. The system 10 may propose different inspection jig configurations depending on the budget. That is, the system 10 may select the camera and lighting depending on the budget. As an example, the system 10 may propose a first inspection jig 1470 using low-angle lighting, a second inspection jig 1480 using coaxial lighting, etc. depending on the budget.

[0153] In step S1460, the system 10 filters cameras and lighting that meet the conditions. Through the processing of this step, the system 10 selects one or more candidate cameras from among the multiple cameras registered in the system 10. Similarly, the system 10 selects one or more candidate lighting from among the multiple lighting devices registered in the system 10.

[0154] As described above, by executing a control program, system 10 can perform the following operations: accept input of one or more selection criteria related to placement; select one or more candidate cameras as candidate inspection cameras from among a plurality of cameras based on the one or more selection criteria; select one or more candidate lights as candidate inspection lights from among a plurality of lights based on the one or more selection criteria; select a camera from among the one or more candidate cameras; select a light from among the one or more candidate lights; capture images of the workpiece multiple times in a virtual space while changing at least a portion of the position of the selected camera, the attitude of the selected camera, the position of the selected light, and the attitude of the selected light; analyze the multiple images obtained by capturing images; and output evaluation information for each combination of the multiple selected cameras, the positions and attitudes of the selected cameras, the selected lights, and the positions and attitudes of the selected lights based on the analysis results.

[0155] <F. Summary>

[0156] As described above, system 10 captures images of a workpiece in a virtual space while exhaustively changing the positions and orientations of the camera and lighting for each combination of camera model number and lighting model number. That is, system 10 captures images of the workpiece for each imaging setting. System 10 then analyzes each of the multiple images obtained using an inspection tool and outputs an evaluation value. By using system 10, a user (such as a line designer) can significantly reduce the amount of work required to select cameras and lighting and determine their positions and orientations. That is, a user (such as a line designer) can significantly reduce the amount of work required to design an inspection jig. Furthermore, by using system 10, a user (such as a line designer) can order the selected cameras and lighting.

[0157] <G. Notes>

[0158] As described above, the present embodiment includes the following disclosures.

[0159] [Configuration 1] A system (10) for assisting in determining an inspection camera, an inspection light, and their placement for a workpiece, comprising: a memory unit (203) for storing information on a plurality of cameras and a plurality of lights; an input unit (206) for receiving input of one or more selection criteria related to the placement; and a processor (201), wherein the processor (201) selects one or more candidate cameras as candidates for the inspection camera from among the plurality of cameras based on the one or more selection criteria; selects one or more candidate lights as candidates for the inspection light from among the plurality of lights based on the one or more selection criteria; selects a camera from among the one or more candidate cameras; selects a light from among the one or more candidate lights; images the workpiece multiple times in a virtual space while changing at least a portion of the position of the selected camera, the orientation of the selected camera, the position of the selected light, and the orientation of the selected light; analyzes the multiple images obtained by image capture; and outputs, based on the analysis results, evaluation information for each combination of the positions and orientations of the multiple selected cameras and the position and orientation of the selected light used to image the workpiece. [Configuration 2] The system (10) of Configuration 1, wherein the processor (201) repeatedly captures images of the workpiece multiple times, the number of times corresponding to the number of combinations of the one or more candidate cameras and the one or more candidate lighting, and analyzes the multiple images obtained by capturing the images. [Configuration 3] The system (10) of Configuration 2, wherein the one or more selection criteria include some or all of the following: the purpose of the inspection camera and the inspection lighting, environmental resistance settings, budget, delivery date, imaging time of the inspection camera, imaging distance, monochrome / color, hue, saturation, and brightness of the inspection lighting, and shape information of the workpiece. [Configuration 4] The system (10) of Configuration 3, wherein the processor (201) limits the setting range of a portion of the position and orientation of the selected camera based on a selection criterion related to the inspection camera included in the one or more selection criteria.[Configuration 5] The system (10) of Configuration 3, wherein the processor (201) limits a setting range for a part of the position and orientation of the selected lighting based on a selection criterion for the inspection lighting included in the one or more selection criteria. [Configuration 6] The system (10) of any of Configurations 1 to 5, wherein the evaluation information includes the model number, position, and orientation of a camera used to image the workpiece, the model number, position, and orientation of a lighting used to image the workpiece, an evaluation value of the analysis result, and the captured image. [Configuration 7] The system (10) of any of Configurations 1 to 5, wherein imaging the workpiece in the virtual space includes imaging each of a plurality of workpieces, and the evaluation information includes overall evaluation information when imaging each of the plurality of workpieces for each combination of the position and orientation of the selected camera and the position and orientation of the selected lighting. [Configuration 8] The system (10) according to any one of configurations 1 to 5, further comprising a communication unit (205) that communicates with a shop server, wherein the input unit (206) accepts input of order information for at least one of a camera and a light included in the evaluation information from a user, and the communication unit (205) transmits the order information to the shop server based on the input of the order information.[Configuration 9] A control method for a system (10) that assists in determining an inspection camera, an inspection light, and their placement for a workpiece, the control method comprising: accepting input of one or more selection criteria related to the placement; selecting one or more candidate cameras from among a plurality of cameras as candidates for the inspection camera based on the one or more selection criteria; selecting one or more candidate lights from among a plurality of lights as candidates for the inspection light based on the one or more selection criteria; selecting a camera from the one or more candidate cameras; selecting a light from the one or more candidate lights; capturing images of the workpiece multiple times in a virtual space while changing at least a portion of the position of the selected camera, the orientation of the selected camera, the position of the selected light, and the orientation of the selected light; analyzing the multiple images obtained by capturing images; and outputting, based on the analysis results, evaluation information for each combination of the positions and orientations of the multiple selected cameras and the position and orientation of the selected light used to capture images of the workpiece.[Configuration 10] A control program for a system (10) that assists in determining an inspection camera, an inspection light, and their placement for a workpiece, the program causing the system (10) to execute the following operations: accepting input of one or more selection criteria related to the placement; selecting one or more candidate cameras from among a plurality of cameras as candidates for the inspection camera based on the one or more selection criteria; selecting one or more candidate lights from among a plurality of lights as candidates for the inspection light based on the one or more selection criteria; selecting a camera from the one or more candidate cameras; selecting a light from the one or more candidate lights; capturing images of the workpiece multiple times in a virtual space while changing at least a portion of the position of the selected camera, the orientation of the selected camera, the position of the selected light, and the orientation of the selected light; analyzing the multiple images obtained by capturing images; and outputting, based on the analysis results, evaluation information for each combination of the positions and orientations of the multiple selected cameras and the position and orientation of the selected light used to capture images of the workpiece.

[0160] The embodiments disclosed herein should be considered to be illustrative in all respects and not restrictive. The scope of the present disclosure is defined by the claims, not by the above description, and is intended to include all modifications within the meaning and scope equivalent to the claims. Furthermore, the disclosures described in the embodiments and each modification are intended to be implemented, as far as possible, either alone or in combination.

[0161] 10 System, 20 Work, 30 Camera, 40 Lighting, 200, 300, 310 Device, 201 Processor, 202 RAM, 203 Memory unit, 204 External device connection unit, 205 Communication unit, 206 Input unit, 207 Output unit, 208 Bus, 216 Input device, 217 Output device, 220 Virtual space generation unit, 230 Placement unit, 240 Imaging unit, 250 Image analysis unit, 260 Evaluation unit, 410, 420, 430, 440, 450, 460, 470, 600, 700, 800, 900, 1000, 1100, 1200 Screen, 510, 520, 530 Setting items, 610, 620, 710, 720, 810, 820 Input items, 630, 730 Preview, 1010, 1110 Check items, 1020, 1120 Items, 1210 Evaluation value, 1220 Camera model number, 1230 Camera position and orientation, 1240 Lighting model number, 1250 Lighting position and orientation, 1260 Lighting settings, 1270 Captured image, 1280 Image, 1290 Total price, 1295 Order check items, 1470 First inspection jig, 1480 Second inspection jig.

Claims

1. A system for assisting in the determination of an inspection camera, inspection lighting, and their arrangement for a workpiece, comprising: a memory unit for storing information on a plurality of cameras and a plurality of lighting; an input unit for accepting input of one or more selection criteria related to the arrangement; and a processor, wherein the processor: selects one or more candidate cameras as candidates for the inspection camera from among the plurality of cameras based on the one or more selection criteria; selects one or more candidate lighting as candidates for the inspection lighting from among the plurality of lighting based on the one or more selection criteria; selects a camera from among the one or more candidate cameras; selects lighting from among the one or more candidate lighting; images the workpiece multiple times in a virtual space while changing at least a portion of the position of the selected camera, the attitude of the selected camera, the position of the selected lighting, and the attitude of the selected lighting; analyzes the multiple images obtained by imaging; and outputs evaluation information for each of the combinations of the multiple selected cameras, the positions and attitudes of the selected cameras, the selected lighting, and the positions and attitudes of the selected lighting based on the analysis results.

2. The system of claim 1, wherein the processor repeatedly captures images of the workpiece multiple times and analyzes the multiple images obtained by capturing images for the number of combinations of the one or more candidate cameras and the one or more candidate lighting.

3. The system of claim 2, wherein the one or more selection criteria include some or all of the following: the purpose of the inspection camera and the inspection lighting, environmental resistance settings, budget, delivery date, the imaging time of the inspection camera, imaging distance, monochrome / color, the hue, saturation, and brightness of the inspection lighting, and shape information of the workpiece.

4. The system of claim 3, wherein the processor limits a range of positions and orientations of the selected camera based on a selection criterion for the inspection camera included in the one or more selection criteria.

5. The system of claim 3, wherein the processor limits a range of positions and orientations of the selected illumination based on a selection criterion for the test illumination included in the one or more selection criteria.

6. A system according to any one of claims 1 to 5, wherein the evaluation information includes the model number, position, and orientation of the camera used to image the workpiece, the model number, position, and orientation of the lighting used to image the workpiece, an evaluation value of the analysis result, and the captured image.

7. A system as described in any one of claims 1 to 5, wherein imaging the workpiece in the virtual space includes imaging each of a plurality of workpieces, and the evaluation information includes overall evaluation information when imaging each of the plurality of workpieces for each combination of the plurality of selected cameras, the positions and orientations of the selected cameras, the selected lighting, and the positions and orientations of the selected lighting.

8. A system as described in any one of claims 1 to 5, further comprising a communication unit that communicates with a shop server, wherein the input unit receives input of order information for at least one of a camera and lighting included in the evaluation information from a user, and the communication unit transmits the order information to the shop server based on the input of the order information.

9. A control method for a system that assists in determining an inspection camera, inspection lighting, and their arrangement for a workpiece, comprising: accepting input of one or more selection criteria related to the arrangement; selecting one or more candidate cameras from a plurality of cameras as candidates for the inspection camera based on the one or more selection criteria; selecting one or more candidate lighting from a plurality of lighting as candidates for the inspection lighting based on the one or more selection criteria; selecting a camera from the one or more candidate cameras; selecting lighting from the one or more candidate lighting; capturing images of the workpiece multiple times in a virtual space while changing at least a portion of the position of the selected camera, the attitude of the selected camera, the position of the selected lighting, and the attitude of the selected lighting; analyzing the multiple images obtained by capturing images; and outputting evaluation information for each of the multiple combinations of the selected cameras, the positions and attitudes of the selected cameras, the selected lighting, and the positions and attitudes of the selected lighting based on the analysis results.

10. A control program for a system that assists in determining an inspection camera, inspection lighting, and their arrangement for a workpiece, the control program causing the system to execute the following operations: accepting input of one or more selection criteria related to the arrangement; selecting one or more candidate cameras from among a plurality of cameras as candidates for the inspection camera based on the one or more selection criteria; selecting one or more candidate lighting from among a plurality of lighting as candidates for the inspection lighting based on the one or more selection criteria; selecting a camera from the one or more candidate cameras; selecting a lighting from the one or more candidate lighting; capturing an image of the workpiece multiple times in a virtual space while changing at least a portion of the position of the selected camera, the attitude of the selected camera, the position of the selected lighting, and the attitude of the selected lighting; analyzing the multiple images obtained by capturing the images; and outputting evaluation information for each of the multiple combinations of the selected cameras, the positions and attitudes of the selected cameras, the selected lighting, and the positions and attitudes of the selected lighting based on the analysis results.

Images