Information processing system, information processing device, information processing method, and program

Abstract

Supports the product inspection process. [Solution] An information processing system comprising: a display control unit that displays an image captured from an inspection area specified in a configuration file on a display unit; an acquisition unit that acquires the operator's inspection results for the captured image; and a storage control unit that stores the captured image in a storage unit in association with the inspection results.

Description

[Technical Field]

[0001] The present invention relates to an information processing system, an information processing apparatus, an information processing method, and a program. [Background Art]

[0002] In recent years, it has been proposed to support component assembly work and the like using a storage shelf having a plurality of storage sections. By using the storage shelf, an operator can sequentially take out components stored in each of the plurality of storage sections and perform component assembly work on a workbench.

[0003] Furthermore, in order to support component assembly work, it has been proposed that instructions on components to be taken out and procedures for assembling the components be projected onto the workbench of the storage shelf by a projector.

[0004] For example, Patent Document 1 below discloses a production navigation system that projects and displays information assisting an operator onto regions corresponding to a plurality of storage sections that store components. [Prior Art Literature] [Patent Literature]

[0005] [Patent Document 1] Japanese Unexamined Patent Application Publication No. 2024-51079 [Summary of the Invention] [Problem to be Solved by the Invention]

[0006] However, in the production navigation system described in Patent Document 1, no consideration has been given at all to supporting the inspection process of a product after the components are assembled.

[0007] Therefore, the present invention has been made in view of the above problems, and the object of the present invention is to provide a novel and improved information processing device, information processing system, information processing method, and program that can support the product inspection process. [Means for solving the problem]

[0008] To solve the above problems, according to one aspect of the present invention, an information processing system is provided which includes a display control unit that displays an image of an inspection area specified in a configuration file on a display unit, an acquisition unit that acquires the operator's inspection results for the image, and a storage control unit that stores the image in a storage unit in association with the inspection results.

[0009] The captured image is an image taken by magnifying the area specified in the configuration file, and the configuration file may specify the position coordinates of the inspection area and the magnification ratio of the image.

[0010] If the unit detects that the worker's body has entered a predetermined area, it may acquire the inspection result.

[0011] If the unit detects that the worker's body has entered the first area within the predetermined area, it may acquire a passing inspection result. If the unit detects that the worker's body has entered a second area within the predetermined area that is different from the first area, it may acquire a failing inspection result.

[0012] If the configuration file specifies multiple inspection areas and the inspection result is satisfactory, the display control unit may cause the display unit to display the image captured of the next inspection area specified in the configuration file.

[0013] If the inspection result is unsatisfactory, the display control unit may display an image on the display unit instructing the worker to perform the task corresponding to the unsatisfactory result.

[0014] Furthermore, in order to solve the above problems, according to another aspect of the present invention, an information processing device is provided, comprising: a display control unit that displays an image captured of an inspection area specified in a configuration file on a display unit; an acquisition unit that acquires the operator's inspection results for the image captured; and a storage control unit that stores the image captured in a storage unit in association with the inspection results.

[0015] Furthermore, in order to solve the above problems, according to another aspect of the present invention, a computer-based information processing method is provided, which includes the steps of: displaying an image of an inspection area specified in a configuration file on a display unit; obtaining the operator's inspection results for the image; and storing the image in a storage unit in association with the inspection results.

[0016] Furthermore, in order to solve the above problems, according to another aspect of the present invention, a program is provided to cause a computer to function as a display control unit that displays an image of an inspection area specified in a configuration file on a display unit, an acquisition unit that acquires the operator's inspection results for the image, and a storage control unit that stores the image in a storage unit in association with the inspection results. [Effects of the Invention]

[0017] As described above, the present invention makes it possible to support the product inspection process. [Brief explanation of the drawing]

[0018] [Figure 1] This is a perspective view showing an example of the configuration of an information processing system according to one embodiment of the present invention. [Figure 2] This is a schematic diagram showing the products to be inspected and the inspection areas in product inspection work using the information processing system according to the same embodiment. [Figure 3] This is a block diagram showing the mechanical configuration of an information processing device. [Figure 4] This is a sequence diagram showing the workflow related to the configuration of an information processing system. [Figure 5] It is a sequence diagram showing a flow of operations related to product inspection work of an information processing system. [Figure 6] It is a schematic diagram showing a state of the information processing system in product inspection work. [Figure 7] It is a schematic diagram showing a state of the information processing system in product inspection work. [Figure 8] It is a schematic diagram showing a state of the information processing system in product inspection work. [Figure 9] It is a block diagram showing an example of the hardware configuration of a computer that implements the information processing system according to the moving embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0019] Preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings. In the present specification and the accompanying drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant description is omitted.

[0020] <1. Configuration example of information processing system> First, an information processing system according to an embodiment of the present invention will be described with reference to FIGS. 1 and 2. FIG. 1 is a perspective view showing an example of the configuration of the information processing system 10 according to the present embodiment. FIG. 2 is a schematic diagram showing products to be inspected and inspection areas in product inspection work using the information processing system 10 according to the present embodiment.

[0021] As shown in FIG. 1, the information processing system 10 includes an information processing device 11, a storage shelf 12, a work table 13, a sensor unit 14, a display projection unit 15, and a mirror 18.

[0022] The information processing system 10 according to this embodiment is a work support system that assists, for example, in parts assembly work using storage shelves 12 and inspection work of products assembled by parts assembly work. Workers can manufacture products by sequentially taking out parts stored in storage shelves 12 and assembling them into products according to the procedure flow of parts assembly work. In addition, workers can perform inspection work to determine whether a product is acceptable or unacceptable by checking the image of the product captured by the zoom camera 141 included in the sensor unit 14.

[0023] In Figure 1, the direction from the storage shelf 12 towards the workbench 13 is defined as the forward direction, and the direction opposite to the forward direction is defined as the backward direction. Furthermore, when viewing the storage shelf 12 directly from the workbench 13 (i.e., when viewing the rear direction directly from the front direction), the up, down, left, and right directions are defined as the upward, down, left, and right directions, respectively.

[0024] The storage shelf 12 is equipped with multiple storage compartments 121. Each of the multiple storage compartments 121 stores parts that will be assembled into a product during the parts assembly process. A worker can reach into any of the storage compartments 121 of the storage shelf 12 and take out the parts stored in the compartment, and then assemble the retrieved parts into the product. As shown in Figure 1, the storage shelf 12 may be equipped with a total of 16 storage compartments 121, with 4 rows in the vertical direction and 4 columns in the horizontal direction. However, the number of storage compartments 121 equipped with the storage shelf 12 is not particularly limited and may be 17 or more.

[0025] The workbench 13 is a table used by workers when assembling parts. The workbench 13 has a flat work surface 131 that extends horizontally across the entire width of the storage shelf 12. The work surface 131 also functions as a projection area on which an image projected from the display projection unit 15, which will be described later, is formed.

[0026] The display projection unit 15 displays support images containing information to assist workers in their parts assembly and product inspection tasks, based on control by the information processing device 11. For example, the display projection unit 15 may be a projector that uses so-called projection mapping technology to project support images onto the front of the storage shelf 12 or the work surface 131 of the workbench 13.

[0027] The display projection unit 15 may be placed on top of the storage shelf 12, for example, as shown in Figure 1. The display projection unit 15 projects projection light onto the front of the storage shelf 12 or the work surface 131 of the workbench 13 by shining projection light toward a mirror 18 provided opposite the display projection unit 15. Thus, the display projection unit 15 can project a support image onto the front of the storage shelf 12 or the work surface 131 of the workbench 13 via the mirror 18.

[0028] The display projection unit 15 may be positioned in a location that allows projection light to be directly projected onto the front of the storage shelf 12 or the work surface 131 of the workbench 13 (for example, the ceiling or wall, not shown, of the space where the information processing system 10 is located). Furthermore, the display projection unit 15 may consist of multiple projectors, including a projector that projects projection light onto the front of the storage shelf 12 and a projector that projects projection light onto the work surface 131 of the workbench 13.

[0029] The mirror 18 is positioned opposite the display projection unit 15, for example, as shown in Figure 1, at a position above the head height of a worker of standard height (for example, above the top storage compartment 121) and in front of the front of the storage shelf 12. Specifically, the mirror 18 may be positioned above the workbench 13 and diagonally above the storage shelf 12, such that its reflective surface faces both the front of the storage shelf 12 and the work surface 131 of the workbench 13, as well as the display projection unit 15. In this configuration, the mirror 18 can reflect the projected light emitted from the display projection unit 15 toward the front of the storage shelf 12 or the work surface 131 of the workbench 13.

[0030] The mirror 18 may be rotatably supported by the support frame 17 so that its reflection angle is variable in the vertical direction. This allows the mirror 18 to reflect the projected light emitted from the display projection unit 15 at any desired reflection angle. Therefore, the mirror 18 can selectively reflect the projected light emitted from the display projection unit 15 to the front of the storage shelf 12 or to the work surface 131 of the workbench 13.

[0031] The sensor unit 14 includes various sensors and senses information related to the work performed by the worker. Specifically, the sensor unit 14 may include a camera and capture still images or videos (hereinafter also referred to as images) of the work performed by the worker. For example, the sensor unit 14 may include a live camera (also referred to as a USB camera) connected to the information processing device 11 via USB (Universal Serial Bus). In this case, the sensor unit 14 can output images of the work performed by the worker as sensing results to the information processing device 11. Note that the types of sensors included in the sensor unit 14 are not limited to those mentioned above. For example, the sensor unit 14 may include an infrared camera, a stereo camera, or a ToF (Time Of Flight) camera.

[0032] The camera included in the sensor unit 14 may be positioned above the head height of a worker of average height (for example, above the top storage compartment 121) and in front of the front of the storage shelf 12, as shown in Figure 1. In this case, the sensor included in the sensor unit 14 can capture an image including part or the whole of a worker located in front of the information processing system 10. For example, the camera included in the sensor unit 14 may be mounted on a support frame 17 that supports the mirror 18, as shown in Figure 1.

[0033] Furthermore, the sensor unit 14 may further include a camera positioned to capture images of the storage shelf 12. In this case, the information processing device 11 can recognize, based on the images of the storage shelf 12, which storage compartment 121 the worker put their hand into to retrieve a part.

[0034] The number and location of cameras included in the sensor unit 14 are not particularly limited. The sensor unit 14 may include, for example, cameras positioned to capture images of the work surface 131 of the workbench 13, or cameras positioned to capture images of the environment surrounding the information processing system 10.

[0035] In the information processing system 10 according to this embodiment, the sensor unit 14 further includes a zoom camera 141 having a zoom function. The zoom camera 141 is positioned to capture images of a product placed at a predetermined position on the work surface 131, and can enlarge (zoom in) and capture images of at least one inspection area of ​​the product. As a result, the information processing system 10 according to this embodiment can support product inspection work by an operator by presenting the operator with an image of the product with the inspection area enlarged.

[0036] Specifically, the zoom camera 141 may be positioned to capture an imaging area IA that includes the product 30 placed at a predetermined position on the work surface 131 (the upper left position in Figure 2), as shown in Figure 2. The product 30 may be placed, for example, on a jig fixed on the work surface 131 to ensure that its position is fixed.

[0037] Product 30 is, for example, a control board on which a component 31 such as an IC chip is attached. The zoom camera 141 enlarges (zooms in) and images the inspection area ZA, which includes the component 31 attached to product 30, and the image of the captured inspection area ZA is projected onto the work surface 131 of the workbench 13 by the display projection unit 15. As a result, an operator who views the projected image of the inspection area ZA can determine whether or not the component 31 has been properly attached to product 30 (i.e., whether it is a pass or fail).

[0038] The position coordinates and magnification of the inspection area ZA, which includes part 31, may be pre-set by being described in a configuration file or the like. For example, the position coordinates of the inspection area ZA may be set using a coordinate system with pixels as the origin, with one point in the imaging area IA of the zoom camera 141 (for example, the upper left corner), as shown in Table 1 below. The magnification of the inspection area ZA may also be set using the zoom magnification of the zoom camera 141. Such a configuration file may be stored, for example, in the storage unit 230 of the information processing device 11, which will be described later.

[0039] [Table 1]

[0040] The information processing device 11 is a computer or similar device that controls the overall operation of the information processing system 10. The functional configuration of the information processing device 11 will be described later with reference to Figure 3.

[0041] A card reader 19 may be connected to the information processing device 11 to read the worker's ID (IDentification) card. The card reader 19 is installed near the storage shelf 12 or workbench 13 and can output worker identification information (e.g., worker identification number and name, etc.) read from the ID card to the information processing device 11. Based on the worker identification information read from the ID card, which is a magnetic card or an IC (Integrated Circuit) card, the information processing device 11 may recognize which worker initiated the parts assembly work or product inspection work.

[0042] The information processing system 10 may, in place of or in addition to the card reader 19, include a biometric information recognition device for recognizing biometric information such as fingerprints or palm prints, an identification number input device for inputting an identification number, or a reader device for reading an image code (for example, a QR code®) containing the identification information of a worker. Based on the information read by the biometric information recognition device or the reader device, the information processing system 10 can recognize which worker initiated the work.

[0043] <2. Example of Information Processing Device Configuration> Next, with reference to Figure 3, the functional configuration of the information processing device 11 included in the information processing system 10 according to this embodiment will be described. Figure 3 is a block diagram showing the mechanical configuration of the information processing device 11.

[0044] As shown in Figure 3, the information processing device 11 comprises a control unit 210, an I / F (interface) unit 220, and a storage unit 230.

[0045] (Control unit 210) The control unit 210 controls the overall operation of the information processing system 10. For example, the control unit 210 includes an arithmetic unit such as a CPU (Central Processing Unit) and a storage device such as ROM (Read Only Memory) and RAM (Random Access Memory). The control unit 210 can control the overall operation of the information processing system 10 by temporarily loading a program stored in ROM into RAM and executing the program with the CPU.

[0046] Specifically, the control unit 210 includes a work sequence control unit 211, a display control unit 212, an acquisition unit 213, and a storage control unit 214.

[0047] The work sequence control unit 211 controls the flow and sequence of the work processes supported by the information processing system 10. Specifically, the work sequence control unit 211 controls the work processes by sequentially switching processes based on the flow of processes and switching timings defined in the work sequence file.

[0048] The work sequence file may define, for example, the order of inspection processes in product inspection work and the timing of switching between inspection processes, as shown in Table 2 below. The work sequence file is stored in the storage unit 230, for example, as data for operating the control unit 210.

[0049] [Table 2]

[0050] Specifically, the process sequence file shown in Table 2 defines that images of inspection areas 1 to 5 of product 30, captured by the zoom camera 141, are projected sequentially according to processes 1 to 5. The images of inspection areas 1 to 5 of product 30 are magnified (zoomed in) images of the area including the parts 31 attached to product 30, and are projected onto, for example, the work surface 131 of the workbench 13. Furthermore, the process sequence file shown in Table 2 defines that processes 1 to 5 are switched sequentially when "pass" information is entered by the worker. "Pass" information from the worker is acquired, for example, by the acquisition unit 213 described later.

[0051] The display control unit 212 controls the projection of support images onto the front of the storage shelf 12 or the work surface 131 of the workbench 13. Specifically, the display control unit 212 may generate support images to assist the worker at each stage of the work process and control the projection of the generated support images onto the work surface 131. For example, the display control unit 212 may control the projection of an image of the inspection area ZA of the product 30, captured by the zoom camera 141, onto the work surface 131.

[0052] Furthermore, the display control unit 212 may control the projection of an image onto the work surface 131 that indicates an area for inputting inspection results for the image of the inspection area ZA. The area for inputting inspection results is, for example, an area for inputting whether the attachment of the part 31 included in the image of the inspection area ZA to the product 30 is acceptable or unacceptable. For example, a first area for inputting an acceptable inspection result and a second area for inputting an unacceptable inspection result may be projected onto the work surface 131. The first area and the second area are projected onto different (i.e., non-overlapping) areas.

[0053] The acquisition unit 213 acquires the worker's inspection results for the image of the inspection area ZA. Specifically, the acquisition unit 213 acquires the worker's inspection results for the installation of the parts 31 included in the inspection area ZA from the images of the first area and the second area captured by the camera included in the sensor unit 14. For example, if it is determined from the image that the worker has placed part of their body (e.g., their hand) into the first area for inputting a passing inspection result, the acquisition unit 213 may acquire a passing inspection result. Also, if it is determined from the image that the worker has placed part of their body (e.g., their hand) into the second area for inputting a failing inspection result, the acquisition unit 213 may acquire a failing inspection result.

[0054] Whether or not a part of the worker's body (e.g., a hand) has entered the first or second area is determined, for example, based on whether or not a part of the worker's body of a different color from the monochromatic (e.g., white) work surface 131 is included in the first or second area in proportion to a threshold. That is, the acquisition unit 213 may determine that a part of the worker's body has entered the first or second area if a portion of an area of ​​a different color from the monochromatic work surface 131 is included in proportion to a threshold.

[0055] When the memory control unit 214 obtains the operator's inspection results for the image of the inspection area ZA, it stores the image of the inspection area ZA in the storage unit 230 in association with the inspection results. Specifically, the memory control unit 214 may store the image of the inspection area ZA in the storage unit 230 with a file name that includes the operator's inspection results. For example, the memory control unit 214 may store the image of the inspection area ZA in the storage unit 230 with a file name that includes the date and time the image of the inspection area ZA was taken, information identifying the work sequence in which the image of the inspection area ZA was taken, and the operator's inspection results. This makes it possible for the memory control unit 214 to store the image of the inspection area ZA in the storage unit 230 with a file name that identifies the date and time of inspection and the work sequence, and identifies the inspection results. Therefore, the memory control unit 214 can more easily manage the images of the inspection area ZA.

[0056] (I / F section 220) The I / F unit 220 performs data input and output with external devices connected by wire or wireless. Specifically, the I / F unit 220 may receive captured images from the sensor unit 14, or ID information from the card reader 19. The I / F unit 220 may also output support images projected onto the front of the storage shelf 12 or the work surface 131 of the workbench 13 to the display projection unit 15, or it may output information regarding the progress of work to an administrator terminal that is not illustrated. For example, the I / F unit 220 may be an interface composed of a communication device for connecting to a network.

[0057] (Storage unit 230) The storage unit 230 is a storage device that stores programs for operating the control unit 210, and data (for example, the above-mentioned configuration file and work procedure file). The storage unit 230 may also store images of the inspection area of ​​the product in association with the inspection results of the inspection area, based on the control of the storage control unit 214. The storage unit 230 may be configured as, for example, an HDD (Hard Disk Drive).

[0058] The information processing system 10, which includes the information processing device 11 having the above configuration, can project an image of the inspection area ZA of the product 30, which has been enlarged and captured, onto the work surface 131, thereby reducing the physical burden on workers performing product inspection work. Furthermore, the information processing system 10 can sequentially project the inspection area ZA of the product 30 onto the work surface 131 according to a set work sequence, thereby further shortening the working time for product inspection work. In addition, the information processing system 10 can automatically store the image of the inspection area ZA in the storage unit 230 of the information processing device 11, thereby enabling more appropriate management of evidence for product inspection work.

[0059] <3. Examples of Information Processing System Operation> Next, the operation of the information processing system 10 according to this embodiment will be described with reference to Figures 4 to 8. Figure 4 is a sequence diagram showing the flow of operations related to the setup of the information processing system 10. Figure 5 is a sequence diagram showing the flow of operations related to product inspection work of the information processing system 10. Figures 6 to 8 are schematic diagrams showing the state of the information processing system 10 in product inspection work.

[0060] (Operations related to the settings of the information processing system 10) First, an action related to the configuration of the information processing system 10 is performed. This action related to the configuration of the information processing system 10 is performed only the first time, for example, when starting up the product inspection operation. This action configures the product inspection operation.

[0061] As shown in Figure 4, the worker logs in to the information processing device 11 (S101). For example, the worker may log in to the information processing device 11 by having their ID card read by the card reader 19.

[0062] Next, the operator sets the display position of the image of the inspection area ZA captured by the zoom camera 141 (S102). The display position of the image of the inspection area ZA can be any position on the work surface 131 of the workbench 13, as long as it does not interfere with other images. The display position of the image of the inspection area ZA set by the operator is stored, for example, in the storage unit 230. The operator may also set the size of the image of the inspection area ZA displayed on the work surface 131.

[0063] Next, the operator sets the position coordinates of the inspection area ZA to be imaged by the zoom camera 141 (S103). For example, the operator may set the position coordinates of the inspection area ZA using a pixel-based coordinate system with the upper left corner of the imaging area IA of the zoom camera 141 as the origin. Alternatively, the operator may set the magnification ratio of the inspection area ZA using the zoom magnification of the zoom camera 141. These position coordinates and magnification ratios of the inspection area ZA are written to a setting file and stored in the storage unit 230.

[0064] Next, the operator sets the positions of the first and second regions for inputting inspection results for the image of the inspection region ZA (S104). The positions of the first and second regions can be any position on the work surface 131 of the workbench 13 that does not interfere with other images. The positions of the first and second regions set by the operator are stored, for example, in the storage unit 230.

[0065] Furthermore, the worker sets the work sequence (S105). For example, the worker may set a work sequence file that defines the order of inspection processes in product inspection work and the timing of process changes. The work sequence file may define the identification number of the inspection area ZA to be projected for each process, and that the process is switched when "pass" information is entered. The work sequence file set by the worker is stored in, for example, the storage unit 230.

[0066] (Actions related to product inspection work) Next, the product inspection task configured as shown in Figure 4 above is executed.

[0067] As shown in Figure 5, first, the worker initiates the process of starting the work procedure (S201). Based on this, the information processing device 11 instructs the zoom camera 141 to image the inspection area ZA to be inspected, based on the work procedure file (S202). The zoom camera 141 performs imaging of the inspection area ZA at the position coordinates and magnification based on the setting file (S203) and transmits the image of the inspection area ZA to the information processing device 11 (S204). The information processing device 11 transmits the image of the inspection area ZA received from the zoom camera 141 to the display projection unit 15 (S205), and the image of the inspection area ZA is displayed on the work surface 131 by the display projection unit 15 (S206).

[0068] For example, as shown in Figure 6, the zoom camera 141 may, based on instructions from the information processing device 11, capture an enlarged image of the inspection area ZA, which includes the part 31 of the product 30 placed on the upper left of the work surface 131. The image of the inspection area ZA captured by the zoom camera 141 may be projected as an image display P1 on the upper right of the work surface 131 by the display projection unit 15, for example. Below the image display P1, images showing the first area D1 and the second area D2 are projected from the display projection unit 15. The first area D1 is the area where the operator inputs a pass inspection result, and the second area D2 is the area where the operator inputs a fail inspection result.

[0069] The worker can determine whether a part 31 included in the inspection area ZA passes or fails by visually viewing the image of the inspection area ZA projected onto the work surface 131. Subsequently, as shown in Figure 7, the worker inputs the pass or fail inspection result into the information processing system 10 by placing a part of their body U (e.g., a hand) into either the first area D1 or the second area D2.

[0070] If the worker determines that the part 31 included in the inspection area ZA is acceptable, the worker places their hand into the first area D1 (S216). As a result, the information processing device 11 obtains the acceptance inspection result from the image of the first area D1 captured by the camera included in the sensor unit 14 and stores the image of the inspection area ZA in the storage unit 230 (S217). At this time, the storage control unit 214 of the information processing device 11 may store the image of the inspection area ZA in the storage unit 230 with a file name that includes the date and time the image of the inspection area ZA was captured, information identifying the work sequence in which the image of the inspection area ZA was captured, and the acceptance inspection result by the worker. Subsequently, the information processing device 11 instructs the zoom camera 141 to capture the next inspection area ZA based on the work sequence file (S218).

[0071] On the other hand, if the worker determines that a part 31 included in the inspection area ZA is unacceptable, the worker moves into the second area D2 (S211). As a result, the information processing device 11 obtains the unacceptable inspection result from the image of the second area D2 captured by the camera included in the sensor unit 14 and stores the image of the inspection area ZA in the storage unit 230 (S212). At this time, the storage control unit 214 of the information processing device 11 may store the image of the inspection area ZA in the storage unit 230 with a file name that includes the date and time the image of the inspection area ZA was captured, information identifying the work sequence in which the image of the inspection area ZA was captured, and the unacceptable inspection result by the worker.

[0072] Next, the information processing device 11 transmits an image of the failed inspection result to the display projection unit 15 (S213), and causes the display projection unit 15 to display the transmitted image (S214). For example, as shown in Figure 8, the display projection unit 15 may project a uniform image P2, such as red, that clearly indicates that the inspection result is a failure, onto the front of the storage shelf 12 and an area including the work surface 131 of the workbench 13. Alternatively, the display projection unit 15 may project an image P3 that instructs the worker to take action to address the failure, such as "Please take action on the NG areas," onto the front of the storage shelf 12 or the work surface 131 of the workbench 13. In this case, the worker can take action to address the failure as instructed by image P3 so that the inspection result becomes a pass (S215).

[0073] If the part 31 included in the inspection area ZA passes the work performed to address the failure in step S215, the process returns to step S216, and the worker enters the first area D1. As a result, the information processing device 11 stores an image of the inspection area ZA in the storage unit 230 (S217) and can proceed to the next step defined in the work sequence file (S218).

[0074] As a result of the above operations, the information processing system 10 can project an enlarged image of the inspection area ZA of the product 30 onto the work surface 131, thereby enabling workers to perform product inspection work more efficiently. Furthermore, if the inspection result is unsatisfactory, the information processing system 10 can instruct the worker to take the necessary steps to address the unsatisfactory result, thereby enabling the worker to resolve the issue. In addition, the information processing system 10 can store the image of the inspection area ZA in the storage unit 230 of the information processing device 11 in association with the work sequence identification information and the inspection result, thereby enabling more appropriate management of product inspection work.

[0075] <4. Hardware Configuration Example> Embodiments of the present invention have been described above. The various information processing operations performed by the information processing system 10 described above are realized through the cooperation of software and the hardware of the information processing system 10, which will be described below.

[0076] Figure 9 is a block diagram showing an example of the hardware configuration of a computer 900 that embodies the information processing system 10 (more specifically, the information processing device 11) according to this embodiment.

[0077] As shown in Figure 9, the computer 900 includes a CPU (Central Processing Unit) 901, a ROM (Read Only Memory) 902, a RAM (Random Access Memory) 903, a host bus 904, a bridge 905, an external bus 906, an interface 907, an input device 908, an output device 909, a storage device 910, a drive 911, and a communication device 913.

[0078] The CPU 901 functions as an arithmetic processing unit and control unit, controlling the overall operation of the computer 900 according to various programs. The CPU 901 may be a microprocessor. The ROM 902 stores programs and arithmetic parameters used by the CPU 901. The RAM 903 temporarily stores programs used in the execution of the CPU 901, and parameters that change as appropriate during program execution. These CPU 901, ROM 902, and RAM 903 are interconnected by a host bus 904, which consists of a CPU bus and the like. The functions of the control unit 210 described above are realized through the cooperation of the CPU 901, ROM 902, and RAM 903.

[0079] The host bus 904 is connected to an external bus 906, such as a PCI (Peripheral Component Interconnect / Interface) bus, via a bridge 905. Note that the host bus 904, bridge 905, and external bus 906 do not necessarily have to be separate; these functions may be implemented on a single bus.

[0080] The input device 908 consists of an input means for an operator to input information, such as a mouse, keyboard, touch panel, buttons, switches, or microphone, and an input control circuit that generates an input signal based on the operator's input and outputs it to the CPU 901. By operating the input device 908, an operator can input various types of data to the computer 900 or instruct it to perform processing operations.

[0081] Furthermore, the input device 908 may be comprised of a device that detects information related to the work. For example, the input device 908 may include various sensors such as an image sensor (e.g., a camera), a depth sensor (e.g., a stereo camera or ToF sensor), an acceleration sensor, a gyroscope, a geomagnetic sensor, a light sensor, a sound sensor, a distance sensor, and a force sensor.

[0082] The output device 909 includes, for example, a CRT (Cathode Ray Tube) display device, a Liquid Crystal Display (LCD) device, an OLED (Organic Light Emitting Diode) device, a display device such as a lamp, or an audio output device such as a speaker.

[0083] The storage device 910 is a device for storing data. The storage device 910 may include a storage medium, a recording device for recording data on the storage medium, a reading device for reading data from the storage medium, and a deletion device for deleting data recorded on the storage medium. The storage device 910 can, for example, implement the functions of the storage unit 230.

[0084] Drive 911 is a reader / writer for storage media and is externally connected to the computer 900. Drive 911 reads information recorded on removable storage media 912, such as a magnetic disk, optical disk, magneto-optical disk, or semiconductor memory, and outputs it to RAM 903. Drive 911 can also write information to the removable storage media 912.

[0085] The communication device 913 is a communication interface composed of communication devices and the like for performing communication. The communication device 913 may be a wireless LAN (Local Area Network) compatible communication device or a wired communication device for wired communication. The communication device 913 can, for example, implement the functions of the I / F unit 220.

[0086] Note that the hardware configuration of computer 900 is not limited to the configuration shown in Figure 9. For example, computer 900 does not need to have input devices 908 or output devices 909, etc. Also, some or all of the configuration shown in Figure 9 may be implemented with one or more ICs (Integrated Circuits).

[0087] Although preferred embodiments of the present invention have been described in detail above with reference to the attached drawings, the present invention is not limited to these examples. It is clear to any person with ordinary skill in the art to which the present invention belongs that various modifications or alterations can be conceived within the scope of the technical idea described in the claims, and these are also understood to fall within the technical scope of the present invention.

[0088] For example, the processes described herein using sequence diagrams do not necessarily have to be executed in the order shown. Some processing steps may be executed in parallel. Additional processing steps may be adopted, and some processing steps may be omitted.

[0089] Furthermore, the series of processes performed by the information processing device described herein may be implemented using software, hardware, or a combination of software and hardware. The programs constituting the software are pre-stored in a storage medium (non-transitory media) provided inside or outside each device. Each program is then loaded into RAM 903, for example, when executed by a computer, and executed by a processor such as CPU 901. The storage medium is, for example, a magnetic disk, optical disk, magneto-optical disk, flash memory, etc. The computer programs may also be distributed without using a storage medium, for example, via a network. [Explanation of symbols]

[0090] 10. Information Processing Systems 11. Information Processing Device 12 Storage shelves 13 Workbench 131 Work surface 14 Sensor section 141 Zoom Camera 15 Display projection section 30 products 31 parts 210 Control Unit 211 Work Procedure Control Unit 212 Display Control Unit 213 Acquisition Department 214 Memory Control Unit 220 I / F section 230 Storage section ZA Inspection Area D1 First region D2 Second area

Claims

1. A display control unit that displays an image of the inspection area specified in the configuration file on the display unit, An acquisition unit that acquires the operator's inspection results for the captured image, A storage control unit that stores the captured image in a storage unit in association with the inspection results, An information processing system equipped with the following features.

2. The aforementioned captured image is an image captured by enlarging the area specified in the configuration file. The information processing system according to claim 1, wherein the configuration file specifies the position coordinates of the inspection area and the magnification of the image.

3. The information processing system according to claim 1 or 2, wherein the acquisition unit acquires the inspection result when it detects that the worker's body has entered a predetermined area.

4. When the unit detects that the worker's body has entered the first area within the predetermined area, it acquires the inspection result of passing the test. The information processing system according to claim 3, wherein the acquisition unit detects that the worker's body has entered a second area different from the first area within the predetermined area, and acquires the inspection result as a failure.

5. The aforementioned configuration file specifies multiple inspection areas, If the inspection result is satisfactory, the display control unit causes the display unit to display the image of the next inspection area specified in the setting file on the display unit, the information processing system according to claim 4.

6. The information processing system according to claim 4, wherein if the inspection result is unsatisfactory, the display control unit causes the display unit to display an image instructing the worker to perform the task corresponding to the unsatisfactory result.

7. A display control unit that displays an image of the inspection area specified in the configuration file on the display unit, An acquisition unit that acquires the operator's inspection results for the captured image, A storage control unit that stores the captured image in a storage unit in association with the inspection results, An information processing device equipped with the following features.

8. The steps include: displaying the captured image of the inspection area specified in the configuration file on the display unit; The steps include obtaining the operator's inspection results for the captured image, A computer-based information processing method comprising the step of storing the captured image in a storage unit in association with the inspection results.

9. Computers, A display control unit that displays an image of the inspection area specified in the configuration file on the display unit, An acquisition unit that acquires the operator's inspection results for the captured image, A storage control unit that stores the captured image in a storage unit in association with the inspection results, A program designed to function as such.

Images