Work Assurance System

Abstract

To provide a work guarantee system with which it is possible for a worker to recognize during work that improper work has been carried out.SOLUTION: A work operation guarantee system 10 comprises: an imaging device 30 for imaging a worker 12 and a workpiece 14; an information projection device 20 for projecting information onto the workpiece; a storage unit 42 for storing area division information in which the workpiece is divided into a plurality of areas and proper work operation information that defines proper work operation carried out for each of the areas; a properness assessment unit 44 for assessing for each of the areas whether or not the work operation carried out by the worker is proper, on the basis of the proper work operation information and the work operation information carried out by the worker that is included in the imaged data acquired by the imaging device; and a projection information control unit 48 for controlling the information projection device so as to project information notifying the fact of being improper to the area having been assessed as improper by the properness assessment unit, and an area division display body.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a work assurance system, and more particularly to a work assurance system that judges the appropriateness of a work movement based on imaging data obtained by imaging the work movement of a worker with an imaging device. [Background technology]

[0002] In product production lines, the issue of how to prevent operational mistakes in the work processes performed by workers is an issue. As a measure to prevent operational mistakes, a work assurance system has been known that uses sensors to obtain information on the movements of workers and analyzes the obtained information with a computer to check whether the work is being performed correctly.

[0003] For example, Patent Document 1 discloses a work assurance system that judges whether simple tasks such as product sorting performed by a worker are appropriate. This work assurance system reads the worker's movements using a motion sensor, and a control device analyzes the data acquired by the motion sensor. The control device judges whether the correct movements of a simple task performed by a pre-registered worker match the movements of the worker read by the motion sensor. In this work assurance system, if the control device determines that there is a mismatch, it outputs a warning on a display installed at the work site. In this way, by guaranteeing the work movements of workers in a work process performed by humans, the occurrence of work errors is suppressed. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2013-174980 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, in the work assurance system described in Patent Document 1, if a worker is concentrating on the work, he or she may miss the warning information displayed on the display and may not be able to grasp which work action was inappropriate. In particular, in a work process in which a worker performs work on multiple different parts of a single workpiece, such as assembling an engine made up of many components, if the worker misses the warning displayed on the display while working, it becomes difficult to know which work part was inappropriately performed, and the worker may have to search for the inappropriate work part, which can delay subsequent work.

[0006] Therefore, for such processed objects, the properness of the work operations is not judged during the work, but rather an inspection process is provided after the work to check whether the product functions normally.

[0007] However, when there is a defect in a product, there is a problem that it takes a lot of work to disassemble the assembled product and find the defect. Therefore, there is a need for a work assurance system that can easily identify the part of the work that was done while the worker is working if the worker has performed an improper work.

[0008] The present invention has been made in consideration of the above-mentioned problems, and aims to provide a work assurance system that enables a worker to recognize that improper work has been performed during a work process in which the worker performs work on multiple parts of a single workpiece. [Means for solving the problem]

[0009] In order to achieve the above object, a work assurance system according to one embodiment of the present invention includes an imaging device that images a worker and a workpiece, an information projection device that projects information onto the workpiece, a memory unit that stores area division information that divides the workpiece into a plurality of areas based on the work content to be performed on the workpiece and proper work action information that specifies proper work actions to be performed for each of the areas, an appropriateness determination unit that determines whether the work action performed by the worker is appropriate for each area based on the appropriate work action information and work action information performed by the worker included in the imaging data acquired by the imaging device, and a projection information control unit that controls the information projection device to project information informing the area that is determined to be inappropriate by the appropriateness determination unit and an area division display body. and an area selection unit that selects at least one area from the plurality of areas based on a predetermined movement or a predetermined posture of the worker included in the imaging data, and when the worker is performing a work action in the area selected by the area selection unit, the projection information control unit causes the information projection device to project an image of the work action of the worker in the area acquired by the imaging device onto the area. Characterized by

[0010] According to this configuration, since it is possible to judge whether the worker's work motion was appropriate for each area into which the workpiece is divided based on the work content, it is possible to perform work assurance for each divided area in a work process in which work is performed on multiple parts of one workpiece. Also, since information informing the worker that the work motion was inappropriate and an area division display are projected onto the area where the work motion was inappropriate, the worker can recognize that inappropriate work was performed during the work. Furthermore, according to this configuration, after a worker completes a task on a workpiece, the worker can specify one area from a plurality of areas by performing a predetermined action on the workpiece, and information on the work action performed by the worker can be projected onto the specified area. This allows the worker to check whether the work was appropriate after completing the task while visually checking the work action information projected onto the specified area.

[0011] Moreover, one embodiment of the present invention is characterized in that in the work assurance system, when the suitability judgment unit judges the work to be inappropriate, the projection information control unit projects work action information of the worker judged to be inappropriate onto the workpiece and / or a display unit installed in the vicinity of the workpiece using the information projection device.

[0012] According to this configuration, when improper work is performed, the worker can visually recognize which work was improper by visually checking the improper work actions of the worker projected by the information projection device.

[0015] In addition, one embodiment of the present invention is characterized in that, in the work assurance system, the area selection unit selects one area from the multiple areas based on a pointing gesture of the worker included in the image data.

[0016] According to this configuration, an operator can arbitrarily specify an area on the workpiece with a simple action of pointing. Effect of the Invention

[0017] According to the work assurance system of the present invention, in a work process in which a worker performs work on multiple parts of a single workpiece, the worker can recognize that improper work has been performed during the work. [Brief description of the drawings]

[0018] [Figure 1] 1 is a schematic explanatory diagram of a work guarantee system according to an embodiment of the present invention. [Diagram 2] FIG. 1 is a block diagram showing a configuration of a work guarantee system. [Diagram 3] 4 is a flowchart illustrating a control process executed by a control device in the work assurance system. [Figure 4] FIG. 2 is a schematic explanatory diagram similar to FIG. 1, showing a modified example of the work assurance system. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] Fig. 1 is a schematic explanatory diagram of a work assurance system 10 according to an embodiment of the present invention, and Fig. 2 is a block diagram showing the configuration of the work assurance system 10. The work assurance system 10 is a system that performs work action assurance to confirm whether the work action performed by a worker on a workpiece is appropriate. In the example shown in Fig. 1, an engine component 14 that constitutes an engine to be mounted on a vehicle is described as an example of the workpiece.

[0020] The work assurance system 10 of this embodiment includes an information projection device 20 capable of projecting information onto the engine component 14 and the display unit 28, an imaging device 30 capable of imaging the worker 12 and the engine component 14 placed on the workbench 13, and a control device 40 electrically connected to these devices.

[0021] The information projection device 20 has a function of projecting electronic information (such as text information and image information) transmitted from the control device 40 as an image on an object. For example, a projector such as a DLP (Digital Light Processing) projector or a liquid crystal projector can be used as the information projection device 20. The work assurance system 10 uses the information projection device 20 and the control device 50 to project an image not only on a flat screen but also on a curved surface or the surface of a three-dimensional object, that is, a so-called projection mapping technology, to project work content information such as tools used when assembling the engine and the procedure of the assembly work on the surface of the engine component 14, which is a three-dimensional object.

[0022] 1, one projector is used as the information projection device 20, but the number of projectors may be two or more. In FIG. 1, an information projection area 22 where information is projected onto the engine component 14 by the information projection device 20 is indicated by a two-dot chain line. The information projected by the information projection device 20 is controlled by a projection information control unit 48 of the control device 40.

[0023] The information projection device 20 can further project information onto a display unit 28 installed near the engine component 14. The display unit 28 can be, for example, a white board or a screen. The projection information 29 projected onto the display unit 28 may be the same as or different from the information projected onto the engine component 14. The display unit 28 is installed near the engine component 14 so that the worker 12 can visually recognize the projection information 29 during work, and in this embodiment, is disposed near the workbench 13.

[0024] The imaging device 30 is capable of capturing images, and for example, a camera capable of capturing video images can be used. In this embodiment, a stereo camera that simultaneously captures images of the area to be captured from two different directions is used as the imaging device 30, and the stereo camera makes it possible to measure the distance to the object. The imaging device 30 is installed at a position in the work site where it can capture images of the engine component 14, which is the object to be processed, and the worker 12 performing work on the engine component 14.

[0025] The imaging device 30 may be installed at a position where it can image at least the hand movements of the worker 12 performing the work together with the engine components 14, but it is preferable that it is installed at a position where it can image the engine components 14 and the surrounding work area so that it can image the posture of the worker 12 during work. In this embodiment, as shown in Fig. 1, the imaging device 30 is installed so that the information projection area 22 by the information projection device 20 and the surrounding area are included within the imaging range of the imaging device 30.

[0026] The control device 40 is a calculator including a microcomputer, and is configured to have an information processing unit such as a central processing unit (CPU) or an application specific integrated circuit (ASIC), a storage unit such as a RAM or a ROM, etc. The control device 40 functions as various function realization units (e.g., a control unit, a calculation unit, a processing unit, etc.) by the central processing unit reading and executing a program recorded in the storage unit. As shown in FIG. 2, the control device 40 of this embodiment includes an information input unit 41, a storage unit 42, an appropriateness determination unit 44, an area selection unit 46, an operation completion determination unit 47, and a projection information control unit 48.

[0027] The information input unit 41 of the control device 40 is, for example, a keyboard, a mouse, a touch panel, an input connection unit with other devices, etc., which allows information to be input to the control device 40. The information input via the input unit 41 can be stored in the storage unit 42 or processed by a data processing unit.

[0028] The memory unit 42 of the control device 40 stores programs for executing various processes, and also stores data related to the information projected by the information projection device 20. In this embodiment, the memory unit 42 includes a divided area memory unit 42A, a task content memory unit 42B, a proper action memory unit 42C, and a worker action memory unit 42D that stores data acquired from the imaging device 30. Each piece of information in the divided area memory unit 42A, the task content memory unit 42B, and the proper action memory unit 42C can be set via the information input unit 41.

[0029] The divided area storage unit 42A stores area division information in which the engine component 14 is divided into a plurality of areas based on the work content performed on the engine component 14. In the area division information, the work content that serves as a criterion for dividing the area may be, for example, dividing into a plurality of areas based on the work actions performed by workers, dividing into a plurality of areas based on the order in which the work is performed, or dividing into a plurality of areas based on the use of common tools.

[0030] In this embodiment, the engine component 14 is divided into a plurality of areas where work operations are common. As an example of an area where work operations are common included in the area division information, in Fig. 1, an oil wiping area 24 where oil wiping operations are performed is indicated by dots in the engine component 14.

[0031] The work content storage unit 42B stores information on the work content to be performed in each area divided by the divided area storage unit 42A as work content information in association with each area. The work content information stored in the work content storage unit 42B includes data such as the order of work in each area, parts and tools used in each area, and the work content and work procedures of each area performed during engine assembly work.

[0032] The proper action storage unit 42C stores proper work information that specifies proper work actions to be performed in each area divided by the divided area storage unit 42A. In this embodiment, the proper work actions for each area stored in the proper action storage unit 42C are stored in association with each area set in the divided area storage unit 42A.

[0033] The worker motion storage unit 42D stores work motion information of the worker 12 (hereinafter also referred to as "worker motion information") included in the imaging data acquired by the imaging device 30. The worker motion information is stored for each area divided by the divided area storage unit 42A. The worker motion information may be the imaging data itself capturing an image of the worker 12 performing work in each area, and / or information extracted from the imaging data on the work motions and work areas of the worker 12.

[0034] The appropriateness determination unit 44 determines whether the work movement performed by the worker 12 is appropriate for each divided area, based on the appropriate work movement information stored in the appropriate movement storage unit 42C and the movement information of the worker 12 included in the imaging data acquired by the imaging device 30. In this embodiment, by comparing the appropriate work movement information with the movement information of the worker 12 in the imaging data, it is determined that the movement of the worker 12 is appropriate if it is within the range of appropriate movements, and is determined to be inappropriate if it is not within the range.

[0035] The projection information control unit 48 controls the information projection device 20 to project work content information associated with each divided area onto the engine component 14 based on the information stored in the storage unit 42. In this embodiment, the information projection device 20 is controlled to project work content information for each area in a preset order. The projection information control unit 48 can cause the information projection device 20 to project a frame (area division display) dividing the area onto the engine component 14 together with the work content information.

[0036] Furthermore, when the suitability determination unit 44 determines that the operation is inappropriate for the area where the work was performed, the projection information control unit 48 controls the information projection device 20 to project information informing the operator that the operation is inappropriate on the area. In this embodiment, when the operation is determined to be inappropriate, the information projection device 20 displays the entire area of ​​the engine configuration 14 in red, and further displays on the display unit 28 the work action information of the operator 12 determined to be inappropriate by the suitability determination unit 44 (for example, a video of the work action determined to be inappropriate) together with the information of the appropriate work action (for example, a video of the appropriate action stored in the appropriate action storage unit 42C). Note that the information projected when the operation is determined to be inappropriate is not limited to this, and it is sufficient that the information is projected so that at least the operator 12 can recognize that an inappropriate operation has been performed during the operation, such as by displaying only the area determined to be inappropriate in red, or by projecting the work action information of the operator 12 determined to be inappropriate on the area determined to be inappropriate.

[0037] The area selection unit 46 selects one area from the multiple areas defined by the area division information based on a predetermined movement or a predetermined posture of the worker 12 with respect to the engine component 14 included in the imaging data acquired from the imaging device 30 after the work on the engine component 14. Here, "after work" means after work has been performed on at least some of the multiple divided areas. The area selection unit 46 can select one area from the multiple areas by, for example, selecting an area pointed to by the worker 12 when the worker 12 makes a pointing motion, selecting an area based on the orientation of the body and the direction of the line of sight of the worker 12, or selecting an area based on the position of the hand of the worker 12 with respect to the engine component 14.

[0038] The area selection unit 46 in this embodiment is set to select an area based on a pointing gesture by the worker 12 after work has been completed in all areas. In the following description, the pointing gesture by the worker 12 that allows the area selection unit 46 to select an area is also referred to as a predetermined area selection gesture.

[0039] When one area is selected by the area selection unit 46, the projection information control unit 48 controls the information projection device 20 to project, onto the area selected by the area selection unit 46, information on the work action of the worker 12 performed in that area.

[0040] The work completion determination unit 47 determines that the work is completed when a predetermined condition is satisfied. In this embodiment, the work completion determination unit 47 determines that the work is completed when the properness determination unit 44 determines that the work has been performed properly for all divided areas, and then the worker 12 performs a predetermined action indicating the completion of the work that has been set in advance. The presence or absence of the work completion action by the worker 12 can be determined from the action of the worker 12 included in the imaging data acquired by the imaging device 30. As an example, the work completion action can be an action in which the worker 12 shakes one hand from side to side a predetermined number of times (e.g., three times) with one hand open on the engine component 14.

[0041] Next, the process executed by the control device 40 in the work assurance system 10 of this embodiment will be described with reference to FIG.

[0042] When the work assurance system 10 is started, the control device 40 acquires imaging data from the imaging device 30 (step S11), and causes the information projection device 20 to project work content information onto one of the divided areas (step S12). The projection of the work content information onto each area is performed according to an order preset in the storage unit 42.

[0043] When the work content information is projected, the suitability determination unit 44 of the control device 40 determines whether the work action performed by the worker 12 is proper or not for the area where the work was performed, based on the worker action information included in the image data and the proper work action information stored in the proper action memory unit 42C (step S13). If the suitability determination unit 44 determines that the work action is inappropriate (step S13: No), it projects information notifying the worker that the work action is inappropriate onto the engine components 14 and the display unit 28 (step S14), and then returns to step 13 to determine whether the redone work is proper or not.

[0044] If the work is judged to be appropriate in step S13 (step S13: Yes), the control device 40 judges whether there is a work area onto which the work content information is to be projected next (step S15), and if there is (step S15: Yes), proceeds to step S12 and projects the work content information onto the next work area.

[0045] In step S15, if there is no next work area, i.e., if proper work has been performed in all areas (step S15: No), the control device 40 determines whether or not a predetermined area selection action has been performed by the worker 12 from the imaging data acquired after the determination in step S15 (step S16). If an area selection action has been performed (step S16: Yes), the area selection unit 46 selects one area and projects information on the work action of the worker 12 performed in the selected area onto the engine component 14 and the display unit 28.

[0046] In step S16, if there is no predetermined area selection action, the task completion determination unit 47 determines whether or not the imaging data includes an action indicating completion of the predetermined task (step S18), and if there is a task completion action (step S18: Yes), it determines that the task is completed and ends the process. If there is no task completion action (step S18: No), the process proceeds to step S16 and continues the subsequent process.

[0047] As described above, in the work assurance system 10 of this embodiment, it is possible to determine whether or not the work actions of the worker 12 were appropriate for each of the areas into which the engine component 14 is divided based on the work content, and therefore it is possible to provide work assurance for each divided area in a work process in which one worker 12 performs work on multiple parts of the engine component 14. Furthermore, if the work actions are inappropriate, information informing the worker of the inappropriateness is projected onto the area, allowing the worker 12 to recognize that inappropriate work has been performed during the work.

[0048] Furthermore, in the work assurance system 10 of this embodiment, when an improper work is performed, information indicating the improper work action is projected by the information projection device 20, so that the worker 12 can see the projected information and recognize which work was improper. In this way, by performing a post-work suitability judgment for each divided area, when an improper work action is performed, the part where the improper work action is performed can be pinpointed, improving workability. Also, the speed of work confirmation can be improved, allowing the production line to run smoothly.

[0049] Furthermore, in the work assurance system 10 of this embodiment, when the worker 12 wants to check the work on a specific part after the work, the worker 12 can point with his finger at the area including this part, and a video of the work action performed in this area can be displayed on the work piece 14 and the display unit 28 for confirmation. By displaying the video on the display unit 28 separately from the work piece 14 as in this embodiment, the worker 12 can check the work by looking at the information that is easier to see.

[0050] In the work assurance system 10, the display unit 28 is optional, and the system may not include the display unit 28. By providing the display unit 28, as described above, the worker 12 can select and view the one that is easier to view. Also, for example, when there is a large amount of information, it is possible to project different information onto the engine component 14 and the display unit 28.

[0051] The present invention is not limited to the above-described embodiment, and various modifications are possible without departing from the spirit of the invention.

[0052] For example, in the work assurance system 10 according to the present invention, the area selection unit 46 can be configured to select one area based on a pointing gesture made by the worker 12 to a completed area when work in some areas has been completed even before work in all areas has been completed. When one area is selected, the projection information control unit 48 can temporarily suspend the projection of work content information and project information on the work action performed by the worker 12 to the selected area.

[0053] Furthermore, the work assurance system 10 according to the present invention may not include the area selection unit 46. In this case, in the control process shown in Fig. 3, when it is determined in step S15 that there is no next work area (i.e., when it is determined that proper work has been performed in all areas), the work completion determination unit 47 can determine that the work is completed and end the process.

[0054] Furthermore, the work assurance system 10 according to the present invention can be configured to display frames dividing areas on the engine component 14 by the information projection device 20. FIG. 4 shows an example in which a lattice-shaped frame 26 dividing the engine component 14 into a plurality of areas is projected onto the engine component 14. The information projection device 20 can project work action information for each area divided by the frame 26, and the suitability determination unit 44 can perform suitability determination of work performed for each divided area. In the example shown in FIG. 4, information on the work action performed by the worker 12 is projected onto the area 27 specified by the worker 12 using a pointing motion.

[0055] In the work assurance system 10 according to the present invention, the information projection device 20 is not limited to a projector, and may be any device capable of visually projecting information onto the workpiece. For example, the information projection device may be a transmissive display device that projects information such as an image onto a transparent or semi-transparent transmissive display installed between the worker 12 and the workpiece. The transmissive display device may be, for example, a glasses-type wearable terminal such as smart glasses for augmented reality. When smart glasses are used as the information projection device, the worker can view the workpiece through glasses-type lenses (i.e., a transmissive display) while wearing the smart glasses, and further, the work content information can be displayed on the lens in AR (Augmented Reality) so as to be superimposed on the workpiece. [Explanation of symbols]

[0056] 10 Work Guarantee System 12 Workers 14 Engine components (objects to be processed) 20 Information projection device 28 Display section 30 Imaging device 40 Control device 42 Storage section 44 Appropriateness Judgment Section 46 Area Selection Section 47 Work completion determination section 48 Projection information control unit

Claims

1. an imaging device for imaging a worker and an object to be processed; an information projection device that projects information onto the object; A storage unit that stores area division information that divides the workpiece into a plurality of areas based on the work content to be performed on the workpiece, and proper work action information that specifies a proper work action to be performed for each of the areas; an appropriateness determination unit that determines whether or not the work movement performed by the worker is appropriate for each area based on the appropriate work movement information and work movement information performed by the worker that is included in the imaging data acquired by the imaging device; a projection information control unit that controls the information projection device to project information notifying the area that has been determined to be inappropriate by the appropriateness determination unit and an area division display body; an area selection unit that selects at least one area from the plurality of areas based on a predetermined movement or a predetermined posture of the worker included in the imaging data; The work assurance system is characterized in that, when the worker is performing work operations in an area selected by the area selection unit, the projection information control unit causes the information projection device to project an image of the worker's work operations in the area, acquired by the imaging device, onto the area.

2. The work assurance system described in Claim 1, characterized in that the projection information control unit causes the information projection device to project work content information onto each area in a predetermined order, and when an area is selected by the area selection unit, pauses the projection of the work content information and projects an image of the work actions of the worker onto the area captured by the imaging device.

3. The work assurance system described in claim 1 or 2, characterized in that when the suitability judgment unit judges the work to be inappropriate, the projection information control unit projects work action information of the worker judged to be inappropriate onto the workpiece and / or a display unit installed in the vicinity of the workpiece using the information projection device.

4. The work assurance system according to any one of claims 1 to 3, characterized in that the area selection unit selects one area from the plurality of areas based on the worker's pointing gesture included in the image data.

Images