Control device, control program, control method, and control system

Abstract

To present information useful for manufacturing a device, to an assembling worker.SOLUTION: A navigation system (production navigation system 10) is a navigation system for assisting a worker by projecting information on a shelf with a plurality of placement sections on which articles to be picked by the worker are placed. The navigation system includes a projector which projects first information for indicating a position of an article to be picked by the worker, and second information related to a work process. The projector is configured to project the first information in a first display region prepared in a placement section corresponding to the article to be picked, and project the second display information in a second display region prepared in a part above the first display region.SELECTED DRAWING: Figure 8

Description

[Technical Field]

[0001] The present invention relates to a navigation system. Mu Regarding. [Background technology]

[0002] In recent years, the following trends have been observed in factories that produce various types of equipment. -The number of parts handled per person is high. New products are launched one after another in a short period of time. ·Production lines are frequently changed for each product. · Production systems using temporary workers have become commonplace. This tendency is more pronounced in factories that produce a wide variety of equipment in small quantities. However, as this trend continues, there is an increasing possibility that assembly omissions, assembly errors, etc. will occur, and therefore factories are required to realize a system to prevent assembly omissions, assembly errors, work omissions, etc.

[0003] One system to prevent assembly omissions is the parts kitting system. Parts kitting is the work of a parts distributor arranging parts from a parts storage warehouse into dedicated kit boxes with designated storage locations for each part, thereby supplying all parts to assembly workers without any omissions. The parts storage warehouse is configured to notify workers of the location of the parts to be collected by lighting up an LED installed at the collection port. With this system, assembly workers can prevent assembly omissions by checking that no parts remain in the kit box after assembly is completed.

[0004] An example of a device related to this kitting work is the device described in Patent Document 1. The device described in Patent Document 1 improves workability by transporting parts to the vicinity of a workbench equipped with a sensor, and when the sensor detects that a part has been removed, moving the next part to the vicinity of the workbench (see, for example, Patent Document 1).

[0005] Another mechanism for preventing assembly errors is to place work instructions around the assembly worker, instructing them on how to assemble the product, until they become proficient in the assembly work, and have the assembly worker assemble the product while checking the work instructions.

[0006] In addition, one system to prevent work omissions is to have each worker fill out a list of work items called a checklist based on the work they have performed, and then stamp or sign the list to record the work history. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] Japanese Patent Application Publication No. 10-29139 Summary of the Invention [Problem to be solved by the invention]

[0008] However, the above-mentioned conventional mechanism for preventing assembly omissions requires a parts distributor, which increases the number of personnel required. The parts storage warehouse is designed to notify workers of the location of the parts to be collected by lighting up an LED installed at the collection port. However, the parts storage warehouse is designed in such a way that the LED's lighting status can be difficult to see depending on the work environment, and simply lighting up the LED does not provide workers with information such as how many parts to collect and where and how to install them, which is an issue. Furthermore, conventional mechanisms for preventing assembly omissions had the problem that when launching a new product, a significant amount of preparation work was required, such as arranging parts in the order in which they would be used, and creating the kit boxes used to prevent assembly omissions. Furthermore, the conventional system for preventing assembly omissions had the problem that, since many design changes, such as part changes, occurred during the start-up period, the kit box had to be remade each time, which meant that it took a long time to complete manufacturing preparations.

[0009] Furthermore, the mechanism for preventing assembly errors described above has the problem that, because the work instructions are stored in a paper file, it takes time and effort to search for the desired page and to maintain and manage the work instructions, and there is a demand for a support system to prevent assembly errors.

[0010] Furthermore, the above-mentioned system for preventing work omissions has the problem that the checklists are paper-based, which requires the time and effort of writing the work history on paper and stamping or signing, and the maintenance of the checklists requires a lot of man-hours, so there is a demand for a support system to prevent work omissions.

[0011] The present invention has been made to solve the above-mentioned problems, and provides a navigation system that presents suitable information to an assembly worker when manufacturing a device. M The main purpose is to provide [Means for solving the problem]

[0012] In order to achieve the above object, the first invention is The system assists workers by projecting information onto shelves equipped with multiple placement areas where workers can place items to be collected. A navigation system, The navigation system includes a projector that projects first information for indicating the location of the item to be collected by the worker and second information related to the work process, and the projector projects the first information onto a first display area provided on the placement unit corresponding to the item to be collected, and projects the second information onto a second display area provided above the first display area. The composition is as follows. [Effects of the Invention]

[0013] According to the present invention, it is possible to present suitable information to an assembly worker when manufacturing a device. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a diagram illustrating an overall configuration of a system according to an embodiment. [Figure 2] FIG. 1 is a diagram (1) showing the configuration of a production navigation system according to an embodiment. [Figure 3] FIG. 2 is a diagram (2) showing the configuration of the production navigation system according to the embodiment. [Figure 4A]1 is a diagram (1) showing the configuration of the main part of a production navigation system according to an embodiment. [Figure 4B] FIG. 2 is a diagram (2) showing the configuration of the main part of the production navigation system according to the embodiment. [Figure 5] 3 is a flowchart showing the overall operation of the production navigation system according to the embodiment. [Figure 6] 4 is a flowchart showing the operation of the production navigation system according to the embodiment during part navigation. [Figure 7A] FIG. 1 is a diagram showing an example of a display screen. [Figure 7B] FIG. 2 shows an example of a display screen. [Figure 7C] FIG. 3 shows an example of a display screen. [Figure 7D] FIG. 4 is a diagram showing an example of a display screen. [Figure 7E] FIG. 5 is a diagram showing an example of a display screen. [Figure 7F] FIG. 6 shows an example of a display screen. [Figure 8] FIG. 1 is a diagram (1) showing a display example during part navigation in the production navigation system according to the embodiment. [Figure 9] FIG. 10 is a diagram (2) showing a display example during part navigation in the production navigation system according to the embodiment. [Figure 10] 1 is a diagram showing an outline of a hand position detection operation in a production navigation system according to an embodiment. FIG. [Figure 11] FIG. 10 is a diagram illustrating an example of image analysis performed by the production navigation system according to the embodiment. [Figure 12] FIG. 10 is a diagram showing a procedure for determining the position of a hand when the position of a hand is detected in the production navigation system according to the embodiment. [Figure 13] FIG. 1 is a diagram (1) showing an example of settings for image analysis in the production navigation system according to the embodiment. [Figure 14] FIG. 10 is a diagram (2) showing an example of settings for image analysis in the production navigation system according to the embodiment. [Figure 15] 4 is a flowchart showing the operation of the production navigation system according to the embodiment during image analysis. [Figure 16] 4 is a flowchart showing the data management operation of the production navigation system according to the embodiment. [Figure 17] FIG. 10 is a diagram showing an example of a management information file. [Figure 18] FIG. 10 is a diagram illustrating an example of a main screen. [Figure 19A] FIG. 10 is a diagram illustrating an example of a production status screen. [Figure 19B] FIG. 10 is a diagram showing an example of a production volume performance screen. [Figure 19C] FIG. 10 is a diagram illustrating an example of a task comparison screen. [Figure 19D] FIG. 10 is a diagram showing an example of a work performance (daily) screen. [Figure 19E] FIG. 10 is a diagram showing an example of a work performance (by process) screen. [Figure 20] FIG. 10 is a diagram illustrating an example of a motion analysis screen. [Figure 21] FIG. 10 is a diagram showing an example of an end screen. DETAILED DESCRIPTION OF THE INVENTION

[0015] Hereinafter, an embodiment of the present invention (hereinafter referred to as "the present embodiment") will be described in detail with reference to the drawings. Note that each drawing is merely a schematic illustration to allow a sufficient understanding of the present invention. Therefore, the present invention is not limited to the illustrated examples. Furthermore, in each drawing, common or similar components are given the same reference numerals, and redundant explanations thereof will be omitted.

[0016] [Embodiment] <Overall system configuration> The overall configuration of the system according to this embodiment will be described below with reference to Fig. 1. Fig. 1 is a diagram showing the overall configuration of the system according to this embodiment.

[0017] As shown in FIG. 1, a production navigation system 10 and a support system 60 are installed in a factory that produces some kind of equipment. The production navigation system 10 is a system that supports product assembly work by presenting various information to assembly workers (hereinafter sometimes simply referred to as "workers"). The production navigation system 10 can output captured images (still images or videos) of the worker's work status, audio information recorded from the worker's voice, and the like to the support system 60. Note that Fig. 1 shows a control device 11, a management unit 71, a display information creation unit 72, a storage unit 73, and the like, which will be described later. The support system 60 is a system that supports the production navigation system 10 .

[0018] The production navigation system 10 and the support system 60 are connected to each other via wire or wirelessly so that they can communicate with each other. The support system 60 may be installed outside the factory.

[0019] The support system 60 includes a data collection device 61 , a data processing device 62 , and a support device 63 .

[0020] The data collection device 61 stores in a storage unit the photographic information, audio information, etc. output from the production navigation system 10. The data collection device 61 also stores in a storage unit various types of information (e.g., work instruction information, work process information, guidance information, projection position information for marks, etc.) and various programs (e.g., control programs for display mechanisms, control programs for image analysis, etc.) and can provide these to the production navigation system 10. The data processing device 62 analyzes the work content and the like based on the photographic information, audio information, and the like stored in the data collecting device 61 . The support device 63 is used for analyzing the work content of the worker, creating work instructions, etc. The support device 63 is configured by a personal computer (PC). The support device 63 is operated by a manager, a support staff member, etc. (hereinafter referred to as "manager, etc.") and is used for creating analysis data (for example, see Figures 19B to 19E described later) etc.

[0021] The data collection device 61 has functional means such as an acquisition unit 81, a storage unit 82, an analysis unit 83, and a provision unit 84. The acquisition unit 81 is a functional means that acquires work data from the production navigation system 10 during product assembly work. The work data includes images captured by the production navigation system 10 during product assembly work. The storage unit 82 is a functional means that saves the work data from the product assembly work. The analysis unit 83 is a functional means that analyzes the work progress and status based on the work data from the product assembly work. The provision unit 84 is a functional means that provides the production navigation system 10 with navigation information that is displayed as display information on the production navigation system 10. One or more of these functional means may also be provided in the data processing device 62, support device 63, control device 11 of the production navigation system 10, etc. of the support system 60.

[0022] <Configuration of the production navigation system> The configuration of the production navigation system 10 will be described below with reference to Figures 2 and 3. Figures 2 and 3 are diagrams showing the configuration of the production navigation system 10.

[0023] As shown in Figure 2, the production navigation system 10 includes a control device 11, a shelf 12, a workbench 13, a photographing unit 14 (14a, 14b), a display mechanism 15, a display 16 (16a, 16b), a microphone 17, a speaker 18, a code input unit 19, and a motion capture camera 20.

[0024] The control device 11 controls the operation of the imaging unit 14, display mechanism (display unit) 15, display 16, microphone 17, speaker 18, etc. The control device 11 is connected to these devices via wired or wireless communication so that they can communicate with each other. The control device 11 is also connected to a data collection device 61 of the support system 60 via wired or wireless communication so that they can communicate with each other. The control device 11 is configured as a personal computer (PC) and operates based on a control program stored in advance. The control device 11 determines the display information to be displayed on the display mechanism 15 according to the position of a specific part of the worker photographed by the imaging unit 14.

[0025] As shown in Fig. 1, the control device 11 has functional means such as a management unit 71, a display information creation unit 72, and a storage unit 73. The management unit 71 is a functional means for managing work results based on work data stored in the storage unit 73. The display information creation unit 72 is a functional means for creating display information representing work results at any time. The storage unit 73 is a functional means for storing work data acquired during product assembly work. One or more of these functional means may also be provided in the data collection device 61, data processing device 62, support device 63, etc. of the support system 60.

[0026] Returning to FIG. 2 , shelf 12 has a large number of component placement sections 21 (here, four vertical rows and six horizontal rows, for a total of 24) and functions as a component warehouse. Shelf 12 is provided with a plurality of partitions 22. Each partition 22 is disposed in a substantially vertical direction, separating the component placement sections 21, and is disposed so that its tip protrudes forward from the front surface of shelf 12. Each component placement section 21 carries a tray 40, which stores a single type of component (e.g., screws, brackets, rollers, etc.). Control device 11 manages component placement information indicating the type of component 41 placed on each component placement section 21. When a worker picks a component 41 from a component placement section 21, control device 11 can identify the type of component 41 picked. Note that component placement sections 21 may carry assembly tools, such as an electric screwdriver.

[0027] Command input positions can be set in advance as desired on the shelf 12. In the illustrated example, for example, the position of the component placement unit 21 that is first from the bottom and first from the right is set as a special command input unit 23 used for inputting special commands. Command contents include, for example, reset, return (confirm), menu selection, and assembly confirmation. When the worker holds his / her hand over the command input position (special command input unit 23 in the illustrated example) (i.e., when the worker holds his / her hand out as if covering an object), the control device 11 determines the display information according to the command corresponding to the input position.

[0028] The workbench 13 is a table used by a worker for assembling parts, etc. The workbench 13 has a flat table surface (upper surface). The table surface is disposed so as to extend horizontally across the entire width of the shelf 12.

[0029] The photographing units 14 (14a, 14b) photograph the working status of the worker, etc. The photographing units 14 are disposed by a support frame 44 that supports the mirror 32 at a position above the head height of a worker of average height (for example, above the top component placement unit) and in front of the front surface of the shelf 12. The photographing units 14 are configured with USB cameras. The USB cameras 14 serving as photographing units output the photographed images to the control device 11. There are two or more USB cameras 14. The two or more USB cameras 14 are disposed apart from each other in the left-right direction. In the illustrated example, two USB cameras 14 are attached to the left and right of the mirror 32. However, the number of USB cameras 14 may be three or more. The USB cameras 14 are not limited to being disposed on the support frame 44, but may be disposed in various locations (for example, outside the shelf 12 or inside each component placement unit 21).

[0030] The display mechanism 15 is a display unit that displays various information, marks, etc. In this embodiment, the display mechanism 15 is composed of a projector 31, a mirror 32, and plate-like members 33 and 34. However, instead of these devices, the display mechanism 15 can also be composed of devices such as an EL display or a liquid crystal display arranged at the positions of the plate-like members 33 and 34.

[0031] The displays 16 (16a, 16b) display various information to the worker. The display 16 is preferably configured as a touch panel display. The display 16a displays, for example, a screen IM20 shown in FIG. 18 (described later). On the other hand, the display 16b displays, for example, main screens IM21 to IM25 shown in FIGS. 19A to 19E (described later) and a screen IM31 shown in FIG. 20 (described later). Alternatively, one of the displays 16a and 16b may be used as a motion capture display that displays video images of the worker captured by the motion capture camera 20, and the other may be used as a data analysis display that displays the results of an analysis of the worker's work based on the video images. In this embodiment, the control device 11 is described as controlling the display operations of the two displays 16a and 16b. However, the production navigation system 10 may be configured such that each of the two displays 16a and 16b is provided with its own individual control device. The microphone 17 records the voice of the worker. The speaker 18 outputs an alarm sound and voice guidance information.

[0032] A code is input to the code input unit 19. In the illustrated example, the code input unit 19 is configured by a barcode scanner and is attached to the workbench 13. The code input unit 19 is not limited to a barcode scanner, and can be configured by various input means such as an IC card reader or a keyboard.

[0033] The motion capture camera 20 captures a nearly complete image of the worker, including at least the upper half of the worker's body. The motion capture camera 20 is placed, for example, to the side of the workbench 13. The control device 11 can analyze the worker's movement line, facial orientation, and the like based on the video images captured by the motion capture camera 20. Note that in this embodiment, the production navigation system 10 is described as including a display 16 and a motion capture camera 20. However, the production navigation system 10 can function without these. Furthermore, it is not necessary to provide one display 16 and one motion capture camera 20 for each shelf 12; one display 16 and one motion capture camera 20 may be provided for multiple shelves 12.

[0034] A rail 45, which functions as a suspending portion for suspending an assembly tool 46 such as an electric screwdriver, is provided on the support frame 44 supporting the mirror 32 in a position forward of the USB camera 14. The worker can easily perform assembly work by using the assembly tool 46 suspended from the rail 45. Furthermore, when the component 41 is a screw and an electric screwdriver is used as the assembly tool 46, the control device 11 can change the torque of the electric screwdriver depending on the type of component 41. This allows the production navigation system 10 to assist the assembly work. Furthermore, when the production navigation system 10 is provided with multiple electric screwdrivers and the worker selects the wrong electric screwdriver, the control device 11 can display an error message as display information to alert the worker and prevent the worker from proceeding to the next process. This also allows the production navigation system 10 to assist the assembly work. The type of part 41 can be identified, for example, based on the position information of the part placement section 21 (collection port), which is the collection position of the part 41 analyzed from the video image captured by the USB camera 14, and the position information of the part placement section 21 (collection port), which instructs the worker to collect the part 41, as display information.

[0035] 3, in this embodiment, the display mechanism 15 projects various display images from the projector 31 onto the plate-like members 33, 34 and the surface of the workbench 13 via the mirror 32. That is, the display mechanism 15 uses the plate-like members 33, 34 and the surface of the workbench 13 as projection units and projects various display images onto the plate-like members 33, 34 and the surface of the workbench 13 using projection mapping technology. The projector 31 corrects the display information and projects it onto the plate-like members 33, 34 and the surface of the workbench 13 in a shape that approximates a rectangle.

[0036] The projector 31 is disposed on the shelf 12. The mirror 32 is disposed by the support frame 44 at a position above the head height of a worker of average height (for example, above the top component placement section) and in front of the front surface of the shelf 12. Specifically, the mirror 32 is disposed above the workbench 13 and diagonally upward from the shelf 12, extending in the left-right direction (horizontal direction) across the entire width of the shelf 12. The mirror 32 is disposed so that its reflective surface faces the projector 31. The mirror 32 is rotatably supported by a support mechanism so that the reflection angle can be changed in the vertical direction. The mirror 32 can be fixed at any reflection angle.

[0037] The plate-like members 33 and 34 function as projection units on which the image (display information) projected from the projector 31 is focused. The plate-like members 33 and 34 are disposed above and / or below the component placement unit 21. However, the plate-like members 33 and 34 are preferably disposed above the component placement unit 21 so that the operator's hands do not obstruct the image (display information) projected from the projector 31 (i.e., so that the projected image is not reflected on the back of the operator's hand, and the shadow of the operator's hand is not cast on the position where the image should be projected). This allows the projector 31 to project the image (display information) at a position where the operator's hands will not obstruct it. In particular, when projecting, for example, an arrow-shaped mark Mk (see FIG. 8 ), the image (display information) is preferably displayed (projected) above the component placement unit 21. At least the upper surfaces (exposed surfaces) of the plate-like members 33 and 34 are flat. The plate-shaped member 33 is larger than the plate-shaped member 34. The plate-shaped member 33 is suitable for presenting information that requires a relatively large area (for example, assembly guidance information Gu (see FIG. 8), work instruction information, graphic information, process flow information, etc.). The plate-shaped member 34 is suitable for presenting information that requires only a relatively small area (for example, marks Mk (see FIG. 8)).

[0038] In the illustrated example, the plate-like members 33 and 34 are arranged at an angle on the front surface of the shelf 12 so as to extend in the left-right direction (lateral direction) across the entire width of the shelf 12. However, the plate-like member 33 may also be configured to have a width shorter than the width of the shelf 12.

[0039] The plate-like members 33 and 34 are preferably whitish in color so that the display image can be clearly projected and so that the USB camera 14 (see FIG. 2) can clearly capture the worker's hands (specifically, so that the hands can be extracted from the background). The workbench 13 is also preferably whitish in color.

[0040] The plate-like members 33, 34 are arranged so that the lower they are, the more they protrude forward (towards the work space) depending on their height (see FIG. 4A). The plate-like members 33, 34 are also arranged at an incline so that their front portions are lower than their rear portions. In particular, the lower the plate-like members 34 are, the more they are inclined so that they are in a more laid-down state (a leaning state) depending on their height (see FIG. 4B). This allows the display mechanism 15 to properly project the display information reflected by the mirror 32 onto the plate-like members 33, 34.

[0041] In this configuration, the display mechanism 15 reflects the display image using the mirror 32, ensuring sufficient distance between the projector 31 and the plate-like members 33 and 34 and between the projector 31 and the surface of the workbench 13 to project the display image. Such a display mechanism 15 allows the projector 31 to project the display image over a wide area at a narrow projection angle. As a result, the production navigation system 10 can ensure a projection distance sufficient to properly project the display information onto the projection unit (the plate-like members 33 and 34 and the surface of the workbench 13), thereby avoiding the need to place the projector 31 in a high position, such as on a ceiling. Therefore, the production navigation system 10 can be used even in places with low ceilings, for example.

[0042] Furthermore, the production navigation system 10 can project display information onto the surface of the workbench 13 using the projector 31, so that a relatively large image can be presented in a position that is easy for the worker to see, thereby improving work efficiency.

[0043] Furthermore, the display mechanism 15 can place the projector 31 on the shelf 12 by reflecting the display image with the mirror 32. Because the top of the shelf 12 is a relatively unused space, the display mechanism 15 can make effective use of that space. Furthermore, the display mechanism 15 can place the projector 31, which is a relatively heavy object, in a location away from the worker's work area, thereby ensuring the safety of the worker. The mirror 32 is made of a shatter-resistant material, such as reinforced plastic. This also allows the production navigation system 10 to ensure the safety of the worker.

[0044] Furthermore, the production navigation system 10 can simultaneously display display information in multiple locations by increasing the number of plate-like members 34. In other words, the production navigation system 10 can easily and inexpensively increase the number of locations where display information is displayed, simply by increasing the number of plate-like members 34.

[0045] 4A and 4B are diagrams each showing the configuration of the main part of the production navigation system 10. FIG.

[0046] As shown in FIG. 4A, in this embodiment, the shelf 12 is provided with four component placement sections 21. Hereinafter, when distinguishing between the four component placement sections 21, they will be referred to as "component placement sections 21a, 21b, 21c, and 21d" in order from the top (see FIG. 4B). The front portion of the shelf 12 is inclined so that the lower side protrudes forward more than the upper side. Furthermore, each component placement section 21 is inclined so that the front portion is lower than the rear portion. The space below the lowest component placement section 21 is dead space that is difficult to access from the front portion of the shelf 12, and the control device 11 is disposed in this space. In this way, the production navigation system 10 makes full use of the space within the shelf 12.

[0047] A work space SP1 for performing assembly work is provided in front of the shelf 12. Meanwhile, an aisle space SP2 for distributing trays 40 containing parts 41 is provided behind the shelf 12. A distributor or worker can arrange (insert) the tray 40 containing parts 41 from the aisle space SP2 into the part placement section 21, and then slide the tray 40 inside the part placement section 21, thereby disposing the tray 40 in a position near the front of the shelf 12.

[0048] The projector 31 is disposed on the rear portion of the shelf 12. This allows the production navigation system 10 to prevent the shelf 12 from tipping forward.

[0049] As shown in FIG. 4B, the component placement section 21 is preferably configured so that multiple trays 40 can be placed on each level. In other words, the depth direction length of the component placement section 21 is preferably longer than the depth direction length of at least two trays 40. This allows the production navigation system 10 to remove the front empty tray 40 and position the new tray 40 that was placed behind the empty tray 40 near the front of the shelf 12. The depth direction length of the component placement section 21 is shorter at the higher levels. Therefore, the depth direction length of the topmost component placement section 21a is the shortest.

[0050] Each of the partition plates 22a, 22b, 22c, and 22d that divide the component placement section 21 has a shape including inclined surfaces 22aaa, 22ab, 22ac, and 22ad, each of which has an obliquely cut upper portion. The shape of each partition plate 22 differs between those arranged on the upper side of the shelf 12 and those arranged on the lower side of the shelf 12. For example, in the illustrated example, the depth direction length of each partition plate 22 is longer toward the lower side. Furthermore, the vertical length of each inclined surface 22aaa, 22ab, 22ac, and 22ad is longer toward the lower side. Note that each of the partition plates 22a, 22b, 22c, and 22d may have a shape including a rounded surface formed in an arc instead of the inclined surfaces 22aaa, 22ab, 22ac, and 22ad.

[0051] FIG. 5 is a flowchart showing the overall operation of the production navigation system 10. FIG. 6 is a flowchart showing the operation of the production navigation system 10 during part navigation. FIGS. 7A to 7F are diagrams showing examples of display screens. FIGS. 7A to 7F respectively show that the top display screen is displayed (projected) on the plate-shaped member 33, the middle display screen is displayed (projected) on the plate-shaped member 34 (any of the four plate-shaped members 34), and the bottom display screen is displayed (projected) on the surface of the workbench 13. Note that the display (projection) operation of each display screen will be described here as being mainly performed by the display information creation unit 72 (see FIG. 1) or the management unit 71 (see FIG. 1) of the control device 11. Note that the flows shown in FIGS. 5 and 6 and the display screens shown in FIGS. 7A to 7F can be changed as appropriate depending on the operation.

[0052] 5, the production navigation system 10 first displays (projects) the top display screen IM11a, the middle display screen IM11b, and the bottom display screen IM11c shown in Fig. 7A onto the plate-like member 33, the plate-like member 34, and the surface of the workbench 13, respectively, and accepts a mode selection (S105). However, in the example shown in Fig. 7A, the top display screen IM11a and the middle display screen IM11b are blank.

[0053] In the example shown in FIG. 7A, three modes, "task learning mode," "expert mode," and "demo mode," are selectably displayed on the bottom display screen IM11c. The "task learning mode" is a mode that displays navigation information including a task video and instruction information with detailed explanations of various matters as display information. The "expert mode" is a mode that displays navigation information including no task video and instruction information with explanations only of relatively important matters as display information. The "demo mode" is a mode that displays a temporary screen navigation as display information. In S105, the worker selects a mode by waving his / her hand over any of the three modes displayed on the bottom display screen IM11c. Here, the description will be given assuming that the worker selects the "task learning mode."

[0054] Next, when a mode is selected, the production navigation system 10 displays (projects) as start screens, for example, the top display screen IM12a, the middle display screen IM12b, and the bottom display screen IM12c shown in Figure 7B, onto the plate-shaped member 33, the plate-shaped member 34, and the surface of the workbench 13, respectively (S110).

[0055] In the example shown in Fig. 7B, the display information "Finished" is displayed in the top display screen IM12a, and the display information "Job ID", "Order ID", and "Progress % (progress rate)" are displayed in the bottom display screen IM12c.

[0056] Next, the production navigation system 10 accepts input of the worker ID and the order ID (S115, S120). The input is accepted, for example, by the worker inputting a code from the code input unit 19 (see FIG. 2). The code is input, for example, by having a label bearing barcode information indicating the personal ID assigned to each worker and the product ID assigned to each product to be manufactured affixed to a portable medium, and having the code input unit 19 (see FIG. 2) read the barcode information from the label. The label can be created at any time. The barcode information can have any content depending on the operation.

[0057] Next, the production navigation system 10 determines whether or not the process is the final process (S125). If it is determined in S125 that the process is not the final process ("No"), the production navigation system 10 performs part navigation (S130).

[0058] The part navigation is performed, for example, according to the flow shown in FIG. As shown in Figure 6, the production navigation system 10 first displays (projects) the top display screen IM13a, the middle display screen IM13b, and the bottom display screen IM13c shown in Figure 7C onto the plate-shaped member 33, the plate-shaped member 34, and the table surface of the workbench 13, respectively, and displays the position and quantity of parts to be collected using arrow-shaped marks Mk (S205).

[0059] As shown in FIG. 7C , the middle display screen IM13b displayed on the plate-shaped member 34 at this time includes a mark Mk as navigation information. In the example shown in FIG. 7C , the mark Mk contains information (e.g., arrow information) indicating the position of the component placement unit 21 (collection port) on which the components 41 to be collected are placed, and information indicating the number of components 41 to be collected (e.g., “×n (pieces)”). The worker collects the number of components 41 indicated by the mark Mk from the component placement unit 21 (collection port) indicated by the mark Mk. Preferably, the worker repeatedly inserts his / her hand into the indicated component placement unit 21 (collection port) the number of times indicated by the mark Mk to collect the components 41. Note that the shape of the “information indicating the position of the component placement unit 21 (collection port) on which the components 41 to be collected are placed” is not limited to “arrow information” and may be any shape pointing in any direction, such as a triangle.

[0060] Next, the production navigation system 10 detects the removal of the component 41 based on the video captured by the USB camera 14 (S210), and determines whether the removed component 41 was picked up from the component placement unit 21 (picking port) at the correct address indicated by the mark Mk (S215). Note that the "address" here means, for example, the arrangement position of each component placement unit 21 on the shelf 12 shown in FIG. 2.

[0061] If it is determined in S215 that the removed part 41 was not picked from the part placement section 21 (picking port) with the correct address (if "No"), the production navigation system 10 displays error display information (S220).

[0062] At this time, the production navigation system 10 displays (projects), for example, the top display screen IM14a, the middle display screen IM14b, and the bottom display screen IM14c shown in FIG. 7D onto the plate-like member 33, the plate-like member 34, and the surface of the workbench 13, respectively. In the example shown in FIG. 7D, each of the display screens IM14a, IM14b, and IM14c displays a red alarm image across the entire screen as error display information. The worker redoes the collection of the components 41 in accordance with the error display information. The redo of the collection of the components 41 is performed by returning the components 41 collected in S210 to the original component placement unit 21 and collecting the number of components 41 indicated by the marks Mk from the component placement unit 21 (collection port) at the correct address indicated by the marks Mk.

[0063] Next, the production navigation system 10 detects a retry of picking up the part 41 based on the video image captured by the USB camera 14 (S225). After this, the process returns to S215.

[0064] On the other hand, if it is determined in S215 that the removed part 41 was picked from the part placement section 21 (picking port) with the correct address (if "Yes"), the production navigation system 10 writes the code information, quantity information, etc. of the picked part 41 into the database (storage section) (S230).

[0065] Next, the production navigation system 10 determines whether the number of collected parts 41 has reached the number indicated by the mark Mk (S235). If it is determined in S235 that the number of collected parts 41 has not reached the number indicated by the mark Mk (if "No"), the process returns to S210. On the other hand, if it is determined in S235 that the number of collected parts 41 has reached the number indicated by the mark Mk (if "Yes"), the production navigation system 10 displays the relevant video, instruction manual, etc. as navigation information (S240).

[0066] At this time, the production navigation system 10 displays (projects), for example, the top display screen IM15a, the middle display screen IM15b, and the bottom display screen IM15c shown in FIG. 7E onto the plate-like member 33, the plate-like member 34, and the surface of the workbench 13, respectively. In the example shown in FIG. 7E, the bottom display screen IM15c displayed on the surface of the workbench 13 includes, as navigation information, assembly guidance information Gu composed of videos, instructions, etc. The worker performs assembly work in accordance with the assembly guidance information Gu. After this, the process returns to S125 in FIG. 5.

[0067] Returning to Figure 5, if the judgment in S125 determines that this is the final process (if "Yes"), the production navigation system 10 displays (projects) the top display screen IM16a, the middle display screen IM16b, and the bottom display screen IM16c shown in Figure 7F as the final screens onto the plate-shaped member 33, the plate-shaped member 34, and the surface of the workbench 13, respectively, and terminates the processing.

[0068] In this production navigation system 10, the control device 11 is configured to determine the display information according to the detection information of the worker's hands. In addition, the control device 11 is configured to present error display information if the worker inserts his / her hand into the wrong part placement section 21 (picking opening).

[0069] 8 shows an example in which display information is projected onto the plate-shaped members 33 and 34. In the illustrated example, assembly guidance information Gu that explains the assembly work is projected onto the plate-shaped member 33, and a mark Mk that indicates the collection port for the part to be collected is projected onto the plate-shaped member 34. Note that the mark Mk can also be projected onto the plate-shaped member 33.

[0070] The mark Mk can indicate not only the location of the collection port but also the order in which the parts are to be collected. The mark Mk can also indicate the name or code of the part to be collected. The mark Mk can also indicate not only the parts but also the collection ports of various assembly tools.

[0071] 9 shows an example in which display information is projected onto the workbench 13. In the illustrated example, assembly guidance information Gu is projected onto the workbench 13. The projector 31 can project, as display information, at least one of a moving image showing the assembly work of a product, a manual image, information on the number of parts 41, and information on the assembly process of the product, for example.

[0072] FIG. 10 is a diagram showing an overview of the hand position detection operation in the production navigation system 10. The control device 11 detects the position of the worker's hands using, for example, the principles of trigonometry. Specifically, as shown in FIG. 10, the control device 11 detects the position of the worker's hands based on an AND condition between the left image captured by the left USB camera 14a and the right image captured by the right USB camera 14b. In this case, the control device 11 analyzes the position of the worker's hands using, for example, the process shown in FIG. 11.

[0073] FIG. 11 is a diagram showing an example of image analysis by the production navigation system 10. As shown in FIG. 11, the control device 11 (see FIG. 2) has a function of performing image analysis on images captured by the USB camera 14. By performing image analysis, the control device 11 can identify, for example, the hand movements of the worker. As a result, the control device 11 can also identify information such as whether the worker has picked up a part and the number of parts picked up.

[0074] An example of image analysis will be described below with reference to Fig. 11. In order to distinguish the worker's hands from the background, the production navigation system 10 uses a whitish color for the background (for example, the colors of the plate-like members 33, 34, the surface of the workbench 13, the partition board 22, etc.). This allows the control device 11 to suitably detect the position of the worker's hands.

[0075] First, as shown in Fig. 11(a), the control device 11 acquires in advance a master image taken by the USB camera 14. The master image is an image in which the worker is not visible. A plurality of master images are prepared in advance corresponding to any part of the shelf 12. Next, as shown in Fig. 11(b), when the assembly work starts, the control device 11 acquires images of the work in progress from the USB camera 14. Note that in Fig. 11(b), the dashed line area indicates the area where the master image in Fig. 11(a) was acquired. Next, as shown in FIG. 11(c), an image of the worker's hands is extracted from the image of the worker in action (see FIG. 11(b)).

[0076] Next, as shown in FIG. 11(d), the control device 11 performs binarization processing on the extracted image of the worker's hand (see FIG. 11(c)). Next, as shown in Figure 11(e), the control device 11 performs a matching process between the image after the binarization process (see Figure 11(d)) and the master image (see Figure 11(a)), removes noise (the part overlapping with the plate-like member 34) from the obtained image, and then detects the center of gravity of the worker's hand from the obtained image. After that, as shown in Figure 11(f), the control device 11 detects the position of the part placement section 21 (collection port) where the worker's hand is placed based on the detected center of gravity of the worker's hand (see Figure 11(e)).

[0077] FIG. 12 is a diagram showing the determination procedure when detecting the hand position in the production navigation system 10. The control device 11 sets squares in the left-right (horizontal) and up-down (vertical) directions corresponding to each component placement section 21 provided on the shelf 12, and determines the hand position in each square according to the position of the worker's hand relative to each square. For example, in this embodiment, the shelf 12 has component placement sections 21 in six columns in the left-right (horizontal) direction and four rows in the up-down (vertical) direction (see FIG. 2). Therefore, the control device 11 sets squares in six columns in the left-right (horizontal) direction and four rows in the up-down (vertical) direction.

[0078] Then, as in image Ph14a in Fig. 12, the control device 11 adds a mark (a black dot in the illustrated example) to the square where the position of the worker's hand is detected from the left captured image taken by the left USB camera 14a after the processing shown in Fig. 11. Similarly, as in image Ph14b in Fig. 12, the control device 11 adds a mark (a black dot in the illustrated example) to the square where the position of the worker's hand is detected from the left captured image taken by the right USB camera 14b after the processing shown in Fig. 11. Then, the control device 11 determines the position of the worker's hand (i.e., the position of the component placement unit 21 (picking opening) where the worker's hand is placed) based on the AND condition between image Ph14a and image Ph14b.

[0079] In order to achieve good image analysis, the control device 11 performs image analysis using multiple types of filters (for example, two types). FIG. 13 shows an example of filter settings for skin color. FIG. 13(a) shows a captured image of a worker's hand being inserted into a collection port and an example of filter settings for the captured image. FIG. 13(b) shows example captured images of a woman's hand and a man's hand. For example, the control device 11 uses a blue filter to distinguish skin color. The filter is set to a desired gradation value so that the hand shown in FIG. 13(b) can be distinguished.

[0080] FIG. 14 shows an example of filter settings for the color of work clothes. FIG. 14(a) shows a captured image of a worker's hand being inserted into a sampling port and an example of filter settings for the captured image. FIG. 14(b) shows example captured images of women's work clothes and men's work clothes. Here, the colors of women's work clothes are yellow and light green or yellow and dark green, while the colors of men's work clothes are black and dark green. In this case, the control device 11 uses a red filter to identify the color of the work clothes. The filter is set to a desired gradation value so that the work clothes shown in FIG. 14(b) can be identified.

[0081] The filter to be used may be automatically adjusted (set) depending on the skin color or the color of the work clothes of the worker.

[0082] FIG. 15 is a flowchart showing the operation of the production navigation system 10 during image analysis. 15, the control device 11 performs the processes of S305 to S350 on the left photographed image captured by the left USB camera 14a. In parallel with the processes of S305 to S350, the control device 11 also performs the processes of S405 to S450 on the right photographed image captured by the right USB camera 14b.

[0083] Specifically, the control device 11 acquires the left photographed image (S305) and deletes unnecessary images (images of parts other than the worker's hands) from the left photographed image (S310). The control device 11 then masks the left photographed image with a red filter (S322), binarizes the obtained image (S324), extracts the difference from the master image (S326), and deletes noise (portions overlapping with the plate-like member 34) from the obtained image (S328). Similarly, the control device 11 masks the left photographed image with a blue filter (S332), binarizes the obtained image (S334), extracts the difference from the master image (S336), and deletes noise (portions overlapping with the plate-like member 34) from the obtained image (S338). Thereafter, the control device 11 combines the image obtained by applying the red filter with the image obtained by applying the blue filter (S340), detects the center of gravity of the worker's hand from the obtained image (S345), and determines the position of the collection port where the hand is inserted (S350).

[0084] The control device 11 also acquires a right-hand captured image (S405) and deletes unnecessary images (images of parts other than the worker's hands) from the right-hand captured image (S410). The control device 11 then masks the right-hand captured image with a red filter (S422), binarizes the obtained image (S424), extracts the difference from the master image (S426), and deletes noise (portions overlapping with the plate-like member 34) from the obtained image (S428). Similarly, the control device 11 masks the left-hand captured image with a blue filter (S432), binarizes the obtained image (S434), extracts the difference from the master image (S436), and deletes noise (portions overlapping with the plate-like member 34) from the obtained image (S438). Thereafter, the control device 11 combines the image obtained by applying the red filter with the image obtained by applying the blue filter (S440), detects the position of the center of gravity of the worker's hand from the obtained image (S445), and determines the position of the collection port where the hand is inserted (S450).

[0085] Thereafter, if the determination result indicates that the sampling port is the same, the control device 11 writes the position of the sampling port into the database (S500). In this way, the control device 11 performs image analysis.

[0086] The control device 11 of the production navigation system 10 can perform data management operations such as storing work data acquired during product assembly work in a storage unit 73, managing work results in a management unit 71 based on the stored work data, and creating display information representing the work results at any time in a display information creation unit 72. In this case, the management unit 71 can manage the work data and work results based on codes input from a code input unit 19. The management unit 71 can also assign semi-finished product codes to semi-finished products, which are intermediate products, at any time, and register the work data in a management information file in the storage unit 73, associating them with finished product codes and semi-finished product codes assigned to finished products, which are final products.

[0087] The data management operation of the production navigation system 10 for realizing such processing will be described below with reference to Fig. 16. Fig. 16 is a flowchart showing the data management operation of the production navigation system 10.

[0088] As shown in FIG. 16, the control device 11 of the production navigation system 10 determines whether work has started (S605), and if it is determined that work has started (if "Yes"), starts storing work data (S610).

[0089] Next, the control device 11 determines whether or not there is an instruction to assign a semi-finished product code (S615). If it is determined in S615 that there is no instruction to assign a semi-finished product code (if "No"), the process proceeds to S625. On the other hand, if it is determined in S615 that there is an instruction to assign a semi-finished product code (if "Yes"), the control device 11 assigns an arbitrary semi-finished product code to the semi-finished product and registers it in the management information file D11 (see FIG. 17) stored in the database (storage unit) 11a (see FIG. 17) (S620).

[0090] Fig. 17 is a diagram showing an example of the management information file D11. In the example shown in Fig. 17, the management information file D11 is configured to register information such as a semi-finished product code, a worker ID, an order ID, a station ID, a work date, a start time, an end time, an elapsed time, and a finished product code.

[0091] 16, if the determination in S615 is "No" or after S620, the control device 11 determines whether or not there is an instruction to display the work record (S625). If it is determined in S625 that there is no instruction to display the work record (if "No"), the processing proceeds to S635. On the other hand, if it is determined in S625 that there is an instruction to display the work record (if "Yes"), the control device 11 analyzes the work record and displays the analysis results of the work record on the display 16 (S630).

[0092] If the determination in S625 is "No" or after S630, the control device 11 determines whether or not an instruction to end the work has been given (S635). If the determination in S635 determines that an instruction to end the work has not been given (if "No"), the process returns to S615. On the other hand, if the determination in S635 determines that an instruction to end the work has been given (if "Yes"), the control device 11 ends the accumulation of the work data (S640) and displays a work performance analysis / end screen IM41 (see FIG. 21) on the display 16 (S645). This completes the processing of the series of routines.

[0093] Such a production navigation system 10 can effectively detect the picking of parts. Note that the production navigation system 10 can detect the picking of parts not only by detecting hand movements but also by detecting parts. The production navigation system 10 can also detect when a worker picks up multiple parts at once.

[0094] This production navigation system 10 can prevent a worker from judging that he or she has picked up a part even if he or she has not actually picked up a part. Furthermore, if the production navigation system 10 detects that a part has been picked up incorrectly, it can point out the mistake by sounding an alarm from the speaker 18 or displaying an alarm on the plate-like members 33 and 34.

[0095] Furthermore, the control device 11 of the production navigation system 10 can display, for example, analysis results IM21 to IM25 (see FIGS. 19A to 19E) of work performance corresponding to items BT11 to BT15 of the main screen IM20 shown in Fig. 18 on the display 16. Fig. 18 is a diagram showing an example of the main screen IM20. Figs. 19A to 19E are diagrams showing examples of screens IM21 to IM25 of the analysis results of work performance, respectively.

[0096] In the example shown in FIG. 18, the main screen IM20 contains items BT11 to BT15. "Item BT11" is a field for specifying the display of production status. "Item BT12" is a field for specifying the display of production results for each order ID of the target worker designated as the target of work results. "Item BT13" is a field for specifying the display of the results of a comparison of the work content between the target worker and an experienced worker. "Item BT14" is a field for specifying the display of the target worker's work results by day. "Item BT15" ​​is a field for specifying the display of the target worker's work results by process. Unless a specific person is designated as the target worker, the person working at the current work site (work station) will be used as the target worker. However, the production navigation system 10 can also designate any person as the target worker by entering that person's ID code using the code input unit 19 (see FIG. 2), a keyboard, etc.

[0097] The main screen IM20 is displayed on the display 16a, which is configured, for example, as a touch panel display. When one of the items BT11 to BT15 is specified on the display 16a, the production navigation system 10 displays, on the display 16b, a screen showing the analysis results of the work performance corresponding to the specified item.

[0098] For example, when "item BT11" in FIG. 18 is specified, the production navigation system 10 displays screen IM21 shown in FIG. 19A on the display 16b. Screen IM21 shows the work results of the production status. In the example shown in FIG. 19A, screen IM21 contains content including "Today's target quantity," "Planned" quantity, "Actual" quantity, and "Difference" quantity.

[0099] Furthermore, for example, when "item BT12" in FIG. 18 is specified, the production navigation system 10 displays screen IM22 shown in FIG. 19B on the display 16b. Screen IM22 shows the production performance of the target worker for each order ID in bar graph data. In the figure, the horizontal axis represents the product order ID, and the vertical axis represents the production time (seconds).

[0100] Furthermore, for example, when "item BT13" in FIG. 18 is selected, the production navigation system 10 displays screen IM23 shown in FIG. 19C on the display 16b. Screen IM23 shows the results of a comparison of the work content between the target worker (comparator) and the skilled worker. Screen IM23 displays a comparison of video images of the target worker (comparator) working, captured by the USB cameras 14a and 14b, with video images of the skilled worker working, prepared in advance. In FIG. 19C, the four squares on the comparator side are areas where video images of the target worker (comparator) working are displayed. The four squares on the skilled worker side are areas where video images of the skilled worker working are displayed. Three rows of images are arranged on the right side of the areas where each video image is displayed. The image in the top row of the squares shows the position of the hand detected from the left captured image (the image captured by the left USB camera 14a). The image in the bottom square represents the hand position detected from the right captured image (the image captured by the right USB camera 14b). The image in the middle square represents the worker's hand position determined based on the AND condition between the hand position detected from the left captured image and the hand position detected from the right captured image.

[0101] Furthermore, for example, when "item BT14" in FIG. 18 is specified, the production navigation system 10 displays screen IM24 shown in FIG. 19D on the display 16b. Screen IM24 shows the target worker's daily work performance as bar graph data. In the figure, the horizontal axis shows the date, and the vertical axis shows the production time (minutes). The bar graph data is data that includes process information that shows each process of the product assembly work and information about the time required for each process. Each bar graph data represents the range between the minimum and maximum work times for a certain product produced by the target worker. The rectangle in each bar graph data represents the average value of the minimum and maximum work times. The longer the bar graph length, the greater the variation in process time for that process.

[0102] Furthermore, for example, when "item BT15" ​​in FIG. 18 is specified, the production navigation system 10 displays screen IM25 shown in FIG. 19E on the display 16b. Screen IM25 shows the work performance of the target worker by process using bar graph data. In the figure, the horizontal axis indicates the process number, and the vertical axis indicates the production time (seconds). Each bar graph data represents the range between the minimum and maximum work time for a certain product produced by the target worker. One of the two rectangles in each bar graph data represents the average value of the minimum and maximum work time, and the other represents the previous work time. The longer the bar graph length, the greater the variation in process time for that process.

[0103] The control device 11 can create graph data for screen IM24 (FIG. 19D) or screen IM25 (FIG. 19E) for each semi-finished product and display it on the display 16b. The control device 11 can also create graph data for each worker and display it on the display 16b. By referring to the graph data, a manager or the like can understand which process is taking time.

[0104] The results of the analysis of work performance performed by the control device 11 of the production navigation system 10 are provided to the data collection device 61 (see FIG. 1) of the support system 60. As a result, the results of the analysis of work performance can be viewed not only by the control device 11 of the production navigation system 10, but also by the data collection device 61, data processing device 62, support device 63, etc. that make up the support system 60 (see FIG. 1).

[0105] Furthermore, the control device 11 of the production navigation system 10 can analyze the movements of a worker based on video images of the worker captured by the motion capture camera 20 during product assembly work. For example, as shown in FIG. 20, the display information creation unit 72 of the control device 11 of the production navigation system 10 can identify the movement distance of the worker's hands based on video images of the worker captured by the motion capture camera 20 during product assembly work and create the line graph data shown in FIG. 20 as display information. FIG. 20 shows an example of a movement analysis screen IM31 resulting from the analysis of the worker's movements. In FIG. 20, the line graph data represents the correlation between frame information of the video images and information on the movement distance of the worker's head and hands. The movement analysis screen IM31 is displayed, for example, on the display 16b.

[0106] In the example shown in Figure 20, the motion analysis screen IM31 includes a video display section Mo1 that displays video, a command section Co1 that displays commands that instruct the operation of the video, and a graph display section Gr1 that displays line graph data. The video in the video display section Mo1 displays additional information in the form of small circles that represent analysis results such as the worker's movement line and facial orientation. The line graph data in the graph display section Gr1 has the horizontal axis representing the number of frames in the video, and the vertical axis representing the distance traveled by the head and hands. The vertical lines in the graph display section Gr1 indicate the divisions between each process, and the numbers represent the process numbers.

[0107] In addition, analysis of worker movements such as the movement analysis screen IM31 shown in Figure 20 can be performed not only by the control device 11 of the production navigation system 10, but also by the data collection device 61, data processing device 62, support device 63, etc. that make up the support system 60 (see Figure 1).

[0108] Furthermore, the control device 11 of the production navigation system 10 can create a screen (for example, an end screen IM41 (see FIG. 21)) that displays elapsed time and the like based on the management information file D11 (see FIG. 17). FIG. 21 is a diagram showing an example of the end screen IM41. In the example shown in FIG. 21, the end screen IM41 includes a standard time column that displays the standard time, a productivity column that displays the production rate, an elapsed time column that displays the elapsed time, a start time column that displays the start time, and an end time column that displays the end time.

[0109] <Major functions of the production navigation system> The production navigation system 10 has a control function, an image (moving image) capturing function, a voice recognition function, and a voice instruction function. The production navigation system 10 has a guidance function, an image acquisition function during part picking and a judgment function therefor, an image acquisition function during part installation and a judgment function therefor, a work flow acquisition function and a judgment function therefor, and the like. The production navigation system 10 outputs captured image (video) information, input audio information, etc. to the data collection device 61 of the support system 60, and can have this information analyzed and processed as big data by the data processing device 62 and the support device (PC) 63. The production navigation system 10 can be expanded later with image analysis functions and voice recognition functions. The production navigation system 10 can guide the worker through the assembly work by presenting various information to the worker. The production navigation system 10 can analyze the type and number of parts picked by the worker, the work content performed by the worker, and the like. The production navigation system 10 can sound an alarm and present alarm information. As shown in FIG. 21, the production navigation system 10 can display the progress of assembly work on the display 16 at any timing, such as when assembly is completed or during the assembly process.

[0110] <Main functions of the support system> The data collection device 61 can acquire and store photographic information and audio information from the control device 11 of the production navigation system 10. A manager or the like can view the photographic information stored in the data collection device 61 on a support device (PC) 63, thereby remotely checking the status of the assembly work site, such as production progress.

[0111] <Main features of the production navigation system> (1) About projection mapping (a) The production navigation system 10 can present (project) the location and quantity of parts to be picked by the worker. For example, the production navigation system 10 can project arrow information indicating the location of the part placement section 21 (picking port) where the parts 41 to be picked are placed, an arrow-shaped mark Mk (see FIG. 8) containing information on the number of parts, etc. This allows the production navigation system 10 to easily inform the worker how many parts 41 should be picked from which picking port, just by glancing at the mark Mk (see FIG. 8). (b) The production navigation system 10 can determine the display information to be displayed on the display mechanism 15 according to the position of a specific body part of the worker photographed by the USB camera 14. For example, when the worker picks up parts, the production navigation system 10 can automatically reduce the count of parts placed on the shelf according to the number of picked parts (picked parts). The production navigation system 10 can also present (project) display information for the next process. Furthermore, when the worker holds his / her hand over a command input position such as the special command input unit 23 (see FIG. 2), the production navigation system 10 can determine the display information according to the command corresponding to the input position. (c) When a worker picks up a part, the production navigation system 10 can present (project) the position and quantity of the next part. (d) If the worker places his / her hand in the wrong part placement section 21, the production navigation system 10 displays warning information such as error display information (see FIG. 7D). This allows the production navigation system 10 to display (project) warning information even if the worker picks up the wrong part. (e) The production navigation system 10 can change the presentation position of the attention-calling information for each part position. (f) The production navigation system 10 can project, as display information from the projector 31, at least one of a moving image showing the assembly work of the product, a manual image, information on the number of parts 41, and information on the assembly process of the product. This allows the production navigation system 10 to efficiently support the assembly work of the product. (g) The projector 31 corrects the display information and projects it in a shape that approximates a rectangle onto the plate-like members 33, 34 and the surface of the workbench 13. This allows the production navigation system 10 to present highly visible display information with little distortion to the worker. (h) The production navigation system 10 can selectively project from the projector 31 two display modes: a task learning mode, which displays navigation information that includes task animations and instruction information with detailed explanations of various matters, and a skill mode, which displays navigation information that does not include task animations and only includes instruction information with explanations of relatively important matters. This allows the production navigation system 10 to change navigation according to the level of mastery of product assembly tasks. (i) The projector 31 can display (project) arrow information indicating the position on the component placement unit 21 on which the component 41 to be picked is placed, as display information, above the component placement unit 21. This allows the projector 31 to display (project) the arrow information at a position that does not interfere with hand detection. (j) In the production navigation system 10, simply by increasing the number of plate-like members 34, the number of areas on which display information is displayed can be increased easily and inexpensively. (k) The production navigation system 10 changes the angle of the plate-like member 34 depending on the presentation (projection) position. The lower the plate-like member 34 is positioned, the more the plate-like member 34 is laid down. This improves the visibility of the displayed information. (l) The color of the plate-like members 33, 34 onto which the display information is projected is preferably white, which allows the display information to be clearly presented (projected) and improves the visibility of the display information. (m) When the collection and assembly of parts is completed, the production navigation system 10 can display on the plate-like members 33, 34, the display 16, etc., what percentage of the entire process has been completed. (n) The production navigation system 10 can project information such as worker ID, temporary ID, work progress information, location and quantity of parts to be collected, work procedure (including image diagrams, etc.), and necessary assembly tools. (o) When the production navigation system 10 detects the collection of a necessary part, it can reduce the part quantity and manage the part quantity. (p) When the production navigation system 10 detects that a necessary part has been collected, it can present the location and quantity of the next part as display information. (q) The production navigation system 10 can project warning information if the worker makes a mistake in picking. The production navigation system 10 can also detect when the worker returns a part. (r) When presenting attention-calling information, the production navigation system 10 can adjust the attention-calling information to an optimal position and present it in accordance with the position of the part. (s) The production navigation system 10 can use the surface of the workbench 13 as a projection unit. (t) The production navigation system 10 can present the location of the next part to the worker when it detects that a part has been picked up. The production navigation system 10 can also present the location of the next part to the worker when all work procedures related to a desired task have been completed. (u) The production navigation system 10 may be configured to switch the display information after the assembly using the electric screwdriver is completed.

[0112] (2) Shelf configuration (a) The shelf 12 is provided with a plurality of partition plates 22. The partition plates 22 are shaped to have inclined surfaces 22aa, 22ab, 22ac, and 22ad (see FIG. 4B) or rounded surfaces (not shown) at the upper portions. This allows the production navigation system 10 to separate adjacent component placement sections 21 with each partition plate 22, while also ensuring ease of insertion of a worker's hand into the component placement section 21. (b) The plate-like members 34 are tilted so that the lower they are placed, the more they lie down (fallen down) depending on their height (see FIG. 4B). This allows the production navigation system 10 to effectively project the display information reflected by the mirror 32 onto the plate-like members 33.

[0113] (3) Image analysis of hand position (a) The production navigation system 10 can perform image analysis of the hand position by performing any image processing (e.g., image acquisition processing, binarization processing, center of gravity detection processing, position detection processing, etc.) based on the captured image. (b) The production navigation system 10 can improve the accuracy of distinguishing between hands and the background by using a white-based color for the background parts (e.g., the color of the plate-like members 33, 34, the surface of the workbench 13, the partition board 22, etc.).

[0114] (4) Image detection (a) The production navigation system 10 can perform good image judgment even if the environment (external brightness) changes. (b) The production navigation system 10 can suppress a decrease in detection accuracy due to differences in clothing (for example, a decrease in detection accuracy for whitish clothing). (c) The production navigation system 10 can accurately detect the position of the hand by analyzing the captured image, even when the hand is inserted into the collection opening from an oblique direction. (d) The production navigation system 10 can identify the insertion amount of the worker's hand into the collection opening (amount of movement in the depth direction) by analyzing the captured image. (e) The production navigation system 10 can use commercially available general-purpose programs as image analysis programs.

[0115] (5) Image analysis (a) The production navigation system 10 can detect an operator's mistaken picking of a part through image analysis. In this regard, conventional technology could only identify whether the operator's hand passed through a sensor installed at the picking port (i.e., whether the operator's hand was inserted into the picking port), which could result in a false detection that a part had been picked even when the operator had not actually picked a part. Furthermore, conventional technology could not detect when an operator picked multiple parts, which could result in an incorrect count of the number of stored parts. In contrast, the production navigation system 10 according to this embodiment can identify whether the operator actually picked a part and how many parts the operator picked by analyzing video images of the parts being picked. As a result, the production navigation system 10 can eliminate the false detection of parts that have not been picked as having been picked and the incorrect count of stored parts. (b) The production navigation system 10 can use image analysis to identify when a worker has picked multiple parts. In this regard, in conventional technology, for example, when a worker picks two of the same parts, the worker has to insert and remove their hand into and from the picking opening twice. In contrast, the production navigation system 10 according to this embodiment can identify that two parts have been picked by simply inserting and removing their hand into and from the picking opening once. (c) The production navigation system 10 can confirm the completion of parts assembly through image analysis. In this regard, conventional technologies only managed whether parts assembly was complete based on the worker's declaration, forcing the worker to perform the process. In contrast, the production navigation system 10 according to this embodiment can automatically manage whether parts assembly is complete without the worker having to declare it. (d) The production navigation system 10 performs image analysis using multiple types of filters (for example, two types) and can detect the worker's skin color, work clothes, etc. The production navigation system 10 can then identify the position of the worker's hand by, for example, binarizing the captured image and detecting the center of gravity (in the illustrated example, the base of the index finger and middle finger).

[0116] (6) Centralized data management (a) The support system 60 can collectively store the captured images in the data collection device 61. (b) The support system 60 allows a manager or the like to analyze the work content at each work site based on the photographed images stored in the data collection device 61, thereby enabling short-term work improvements (for example, reducing variations in work time and adjusting line balance). In other words, the support system 60 allows the manager or the like to check assembly operations, etc. from a remote location. In this regard, in the prior art, for example, a manager or the like had to visit the assembly work site to check the assembly operations, etc. In contrast, the production navigation system 10 according to this embodiment allows the manager or the like to check assembly completion, etc. from a remote location without having to visit the assembly work site. (c) The support system 60 can remotely check the operation of a process that has variations in work time.

[0117] (7) Regarding audio confirmation (a) The production navigation system 10 can also confirm the completion of the assembly operation by acquiring voice information from the worker using the microphone 17.

[0118] (8) Selection of assembly tools (a) The production navigation system 10 can confirm that the tightening has been completed with the electric screwdriver. (b) When the torques of the electric screwdrivers differ, the production navigation system 10 can instruct the worker to select the desired electric screwdriver. Furthermore, when multiple electric screwdrivers are provided and the worker selects the wrong electric screwdriver, the production navigation system 10 can display an error message as display information to alert the worker and prevent them from proceeding to the next process. (c) The production navigation system 10 can automatically adjust the torque of the electric screwdriver by changing the current value according to the type of screw.

[0119] (9) Parts management and work management (a) The production navigation system 10 can provide functions such as providing work instructions, production status, and warnings at a low cost by presenting information through projection mapping using the projector 31 and by performing image processing on images captured by the USB camera 14. (b) The production navigation system 10 has a management unit 71 (see FIG. 1), a display information creation unit 72 (see FIG. 1), and a storage unit 73 (see FIG. 1), and by using these, a data management method can be realized in which work data acquired during product assembly work is stored in the storage unit 73, the management unit 71 manages work results based on the stored work data, and the display information creation unit 72 creates display information representing the work results at any time. Furthermore, when the support system 60 is provided with the management unit 71, the display information creation unit 72, and the storage unit 73, it can realize a similar data management method by using these units. This allows, for example, the production navigation system 10 and the support system 60 to check whether there are any missing parts at the work site, whether screw tightening and adjustment work are being performed, whether test results are satisfactory, etc. (c) The production navigation system 10 can improve efficiency by strictly recording work history and by using IT. (d) The production navigation system 10 can display the work time at the assembly site in real time, which makes it possible to improve work in a short period of time (for example, reducing variations in work time and adjusting line balance). (e) The production navigation system 10 can assign semi-finished product codes to semi-finished products, which are intermediate products, at any time and input them via the code input unit 19 (see FIG. 2), a keyboard, etc. The production navigation system 10 and the support system 60 can then manage work data and work results based on the input codes. For example, the production navigation system 10 and the support system 60 can associate work data with finished product codes and semi-finished product codes assigned to finished products, which are final products, and register them in the management information file D11 (see FIG. 17) in the storage unit 73 (see FIG. 1), thereby reading out work data related to the associated products at any time. This allows the production navigation system 10 and the support system 60 to read and verify information such as which worker assembled which semi-finished product, and video images of the work performed during the assembly of the semi-finished product, from a database, based on the finished product codes of the finished products, which are final products, for example. Furthermore, the production navigation system 10 and the support system 60 can read out from the database, for example, a moving image of the worker's work based on the worker's ID, and verify whether the work of the worker with that ID is good or bad. (f) The production navigation system 10 and the support system 60 can display video images of unskilled workers working and video images of skilled workers working on a display so that they can be compared (see FIG. 19C). This allows the production navigation system 10 and the support system 60 to easily verify issues with the work performed by each worker (for example, factors that require time in each process, the appropriateness of the work content, etc.). (g) The production navigation system 10 and the support system 60 can analyze the movements of the worker based on video images of the worker captured by the motion capture camera 20 during product assembly work (see FIG. 20). (h) The production navigation system 10 eliminates the need for parts kitting, which can eliminate the need for parts distribution work and parts distributors. Therefore, the production navigation system 10 can reduce production costs. (i) The production navigation system 10 can automatically record work history as electronic data by using information from sensors, electric screwdrivers, etc., and analyzing the work content of workers. This allows the production navigation system 10 to reduce the amount of work required to create checklists (i.e., it reduces the effort required to enter the work history for each worker on a checklist (a list of work items) and to stamp or sign it). It also promotes a paperless system. Therefore, the production navigation system 10 can also reduce production costs. (j) The production navigation system 10 can immediately respond to changes in workers. (k) The display information creation unit 72 (see FIG. 1) can create and display, as display information, graph data including process information representing each process of a product assembly work and information about the time required for each process (see, for example, FIGS. 19B, 19D, and 19E). The display information creation unit 72 (see FIG. 1) can also create and display, as display information, work image information that displays work video images of an expert and a non-expert so that they can be compared (see, for example, FIG. 19C). Furthermore, the display information creation unit 72 (see FIG. 1) can create, as display information, navigation information for a work acquisition mode that includes work videos and instruction information with detailed explanations on various matters, and navigation information for a skilled mode that does not include work videos and instruction information with only explanations on relatively important matters, and selectably display these pieces of display information (see FIG. 7A). Furthermore, the display information creation unit 72 (see FIG. 1) can identify the distance the worker's hands have moved based on the video of the worker captured by the motion capture camera 20 during product assembly work, and create and display graph data that shows the correlation between frame information of the video and information on the distance the worker's hands have moved as display information (see FIG. 20). By providing such a display information creation unit 72 (see FIG. 1) within the production navigation system 10 and the support system 60, they can efficiently support product assembly work.

[0120] (10) Functionality Extensibility (a) The production navigation system 10 can be equipped with a USB camera 14, etc. This allows the production navigation system 10 to check, for example, whether or not a nameplate is missing. (b) The production navigation system 10 can be equipped with an additional microphone 17 and speaker 18. This allows the production navigation system 10 to improve the accuracy of voice recognition and the appropriateness of voice instructions. (c) The production navigation system 10 can update various programs (for example, the control program of the production navigation system 10, the image analysis program, the voice recognition program, etc.) as needed. (d) The production navigation system 10 may be configured to provide separate lighting devices around the workbench 13. This allows the production navigation system 10 to acquire images with good brightness even when the brightness around the workbench 13 varies depending on the location, time of day, etc., thereby improving the accuracy of analyzing the captured images.

[0121] As described above, the production navigation system 10 according to this embodiment can present suitable information to an assembly worker when producing a device.

[0122] The present invention is not limited to the above-described embodiment, and various changes and modifications can be made without departing from the spirit of the present invention. For example, the above-described embodiment has been described in detail to clearly explain the gist of the present invention. Therefore, the present invention is not necessarily limited to an embodiment including all of the components described. Furthermore, the present invention may include some components by adding other components to them, or by replacing some components with other components. Furthermore, the present invention may also include some components by deleting them.

[0123] 5, 6, 15, and 16 can be changed as appropriate depending on the operation. Also, for example, the configuration of the contents of the display screen, graph data, etc. can be changed as appropriate depending on the operation. [Note] The following is a summary of this application: Original application The invention described in the claims originally attached to the application shall be added. (1) a display control unit that controls the display of information to assist the worker in areas corresponding to a plurality of placement units on which the worker places the items to be collected; an analysis control unit that analyzes the actions of the worker based on a moving image including the worker captured by a capturing unit; a storage unit that stores work performance based on the analysis results by the analysis control unit; an output unit that outputs information including a work time based on the work performance stored in the storage unit, Navigation system. (2) the display control unit displays assembly guidance information related to the item. The navigation system according to (1) above. (3) The storage unit further stores information for identifying the worker in association with the work record. The navigation system according to (1) above. (4) The work performance includes a start time and an end time. The navigation system according to (1) above. (5) the analysis control unit analyzes the worker's movement by detecting the center of gravity of the worker's hand based on the moving image. The navigation system according to (1) above. (6) the analysis control unit analyzes the motion of the worker by detecting a center of gravity position of the worker's hand based on a difference between the moving image and a master image that does not include the worker. The navigation system according to (5) above. (7) the navigation system further includes an assembly tool control unit that controls an assembly tool; the assembly tool control unit acquires a work history using the assembly tool based on information from the assembly tool. The navigation system according to (1) above. (8) a step of controlling the display of information to assist the worker in areas corresponding to a plurality of placement units on which the worker places the items to be collected; analyzing the movement of the worker based on a moving image including the worker captured by a capturing unit; storing the work performance based on the analysis result in a storage unit; and outputting information including a work time based on the work performance stored in the storage unit. Information processing methods. (9) Computer, a display control unit that controls the display of information to assist the worker in areas corresponding to a plurality of placement units on which the worker places the items to be collected; an analysis control unit that analyzes the actions of the worker based on a moving image including the worker captured by a capturing unit; a storage unit that stores work performance based on the analysis results by the analysis control unit; an output unit that outputs information including a work time based on the work performance stored in the storage unit; A program to function as a [Explanation of symbols]

[0124] 10 Production Navigation System 11 Control device (PC) 11a Database (Storage Section) 12 shelves (parts warehouse) 13 Workbench 14(14a, 14b) Photography unit (USB camera) 15 Display mechanism (display section) 16(16a,16b) Display 17. Mike 18 speakers 19 Code input section 20 Motion Capture Camera 21 (21a, 21b, 21c, 21d) Component placement section 22(22a, 22b, 22c, 22d) Partition plate 22aa,22ab,22ac,22ad Slope 23 Command input section 31 Projector 32 Mirror 33, 34 Plate-shaped member (projection part) 40 trays 41 parts 44 Support Frame 45 Rail (hanging part) 46 Assembly tools (electric screwdriver) 60 Support System 61 Data Collection Equipment 62 Data processing device 63 Support Device (PC) 71 Management Department 72 Display information creation unit (display control unit, analysis control unit, output unit) 73 Storage unit (storage unit) 81 Acquisition Department 82 Preservation Department 83 Analysis Department 84 Providing Department DP imaging data D1 Master image acquisition D2 Image acquisition D3 binarization processing D11 Management Information File Gu Guidance Mk mark P0 Center of gravity detection R1, R2, R3 position detection P1 position

Claims

1. A control device comprising: a control unit that determines display information including first information indicating the location of an item to be picked up by a worker and second information indicating a work process; and transmits a signal including the display information to a projector to generate a display image to be projected by the projector, the first information is projected onto a first display area located on a shelf on which the item is placed; The control device, wherein the second information is projected onto a second display area located above the first display area.

2. A control device as described in Claim 1, wherein the control unit determines the display information depending on the position of a specific part of the worker photographed by the imaging unit.

3. The control unit identifying a placement unit on which the worker has placed his / her hand based on the captured image captured by the imaging unit, and determining whether the placement unit on which the worker has placed his / her hand is different from the placement unit corresponding to the first display area on which the first information is projected; 3. The control device according to claim 2, wherein when it is determined that the placement unit into which the worker places his / her hand is different from the placement unit corresponding to the first display area onto which the first information is projected, the display information is determined to project error display information.

4. The control device described in Claim 3, wherein the control unit determines the display information so that a red alarm image is projected onto the front of the shelf as the error display information.

5. The control device described in Claim 3, wherein the control unit identifies the placement area on which the worker placed his / her hand for each of the images captured by the multiple imaging units, and if the placement areas on which the worker placed his / her hand identified for each of the images match, determines whether the placement area is different from the placement area corresponding to the first display area onto which the first information is projected.

6. The control device described in claim 1, wherein the control unit determines display information including second information including text information and image information related to work procedures.

7. The control device described in Claim 1, wherein the control unit starts part navigation by obtaining a code containing information related to the product from a code reading unit.

8. The control unit determines display information including second information including assembly guidance information including assembly process information of the product in the part navigation; The control device according to claim 1 , wherein the assembly guidance information includes information indicating a progress status.

9. A control program causing a computer to function as a control unit, which determines display information including first information indicating a position of an item to be picked up by a worker and second information indicating a work process, and transmits a signal including the display information to a projector to generate a display image to be projected by the projector, the first information is projected onto a first display area located on a shelf on which the item is placed; The second information is projected onto a second display area located above the first display area.

10. A control method, performed by a computer, comprising the steps of: determining display information including first information indicating a location of an item to be picked up by a worker and second information indicating a work process; and transmitting a signal including the display information to a projector to generate a display image to be projected by the projector, the first information is projected onto a first display area located on a shelf on which the item is placed; A control method in which the second information is projected onto a second display area located above the first display area.

11. A control system comprising: a control unit that determines display information including first information indicating a location of an item to be picked up by a worker and second information indicating a work process, and transmits a signal including the display information to a projector to generate a display image to be projected by the projector, the first information is projected onto a first display area located on a shelf on which the item is placed; The second information is projected onto a second display area located above the first display area.

Images