Work management device, work management system, inference device, machine learning device, work management method, inference method, and machine learning method
The work management device addresses the precision issue in work confirmation by using a front-facing image capture unit to generate and store work information, ensuring high-precision recording and enhancing work quality analysis and instruction reproduction.
Patent Information
- Application Number
- PCT/JP2024/011590
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-31
- Filing Date
- 2024-03-25
- Publication Date
- 2024-12-05
AI Technical Summary
Conventional work management systems lack precision in confirming work completion, especially when using infrared cameras from a distance, resulting in lower accuracy compared to images taken nearby.
A work management device with an image capture unit positioned in front of the worker, acquiring image data to generate and store work information chronologically, enabling high-precision recording and confirmation of work status, and converting individual work quality into organizational knowledge.
The system allows for accurate automatic recording and confirmation of work, enhances work quality analysis, and facilitates instruction reproduction using XR, improving work management efficiency.
Smart Images

Figure JP2024011590_05122024_PF_FP_ABST
Abstract
Description
Work management device, work management system, inference device, machine learning device, work management method, inference method, and machine learning method
[0001] The present invention relates to a work management device, a work management system, an inference device, a machine learning device, a work management method, an inference method, and a machine learning method.
[0002] Conventionally, in order to confirm the completion of work performed by a worker on a predetermined object, a report has been created or a photograph or the like has been taken so that the work can be checked later. For example, Patent Document 1 discloses a work recording device that performs thermal image analysis processing on image data captured by an infrared camera and records the work content of a worker using a tool.
[0003] Utility Model Registration No. 2575432
[0004] The work recording device disclosed in Patent Document 1 photographs workers from a distance, which results in a significantly lower accuracy compared to images taken from nearby. Also, because it processes thermal images using an infrared camera, the accuracy is significantly lower than when analyzing actual images.
[0005] In view of the above-mentioned problems, the present invention aims to provide a work management device, a work management system, an inference device, a machine learning device, a work management method, an inference method, and a machine learning method that can automatically store the work status of workers with high accuracy and appropriately check the work status.
[0006] In order to achieve the above object, a work management device according to one embodiment of the present invention is a work management device that manages multiple tasks performed by a worker on a specified object, and includes: an image data acquisition unit that acquires image data captured by an image capture unit that is positioned facing forward of the worker when the worker wears a worker device equipped with an image capture unit; a work information generation unit that generates work information for each task that the worker performs on a specified work object based on the image data acquired by the image data acquisition unit; and a memory unit that stores the work information generated for each task by the work information generation unit in chronological order.
[0007] According to one aspect of the present invention, a work management device generates work information for each task performed by a worker using a tool on a work target at a specified work position based on image data captured by an image capture unit positioned facing the worker, and records the work information in chronological order. This automatically memorizes the worker's work with high accuracy, allowing for appropriate confirmation of the work status. Furthermore, for tasks that have not been standardized or documented, the quality of the worker's individual work results can be converted into organizational knowledge and explicit knowledge. Furthermore, the work history of workers who perform high-quality work can be analyzed, and instructions can be provided to reproduce the work information using XR.
[0008] Problems, configurations, and effects other than those described above will become apparent from the detailed description of the invention that follows.
[0009] 1 is an overall configuration diagram showing an example of a work management system 1 according to a first embodiment. FIG. 2 is a perspective view showing a reducer 2a as a first example of a work target 2. FIG. 3 shows a torque driver 22a and a grease gun 22b as examples of a tool 22. FIG. 4 is a block diagram showing an example of a work management device 5. FIG. 5 is a functional explanatory diagram showing an example of a work management device 5. FIG. 6 is a data configuration diagram showing example of work information 112 generated by a work information generation unit 502 and example of specification information 522 stored in a memory unit 52. FIG. 7 is a block diagram showing an example of a user terminal device 6. FIG. 8 is a hardware configuration diagram showing an example of a computer 900. FIG. 9 is a flowchart showing an example of a work management method using the work management device 5 and the user terminal device 6. FIG. 10 is a diagram showing a first example in which work information 112 is superimposed on a work target 2 in real space. FIG. 11 is a diagram showing a second example in which work information 112 is superimposed on a work target 2 in real space. FIG. 12 is an overall configuration diagram showing an example of a work management system 1 according to a second embodiment. FIG. 13 is a block diagram showing an example of a machine learning device 4. FIG. 14 is a diagram showing an example of a learning model 10 and learning data 11. FIG. 15 is a flowchart showing an example of a machine learning method using the machine learning device 4. FIG. 16 is a block diagram showing an example of a work management device 5 according to a second embodiment. Fig. 1 is a functional explanatory diagram showing an example of a work management device 5 of a second embodiment. Fig. 2 is a block diagram showing an example of a user terminal device 6. Fig. 3 is a flowchart showing an example of a safety support method using the work management device 5 and the user terminal device 6. Fig. 4 shows a drive device 2b as a second example of a work object 2. Fig. 5 shows a stopper of a roll cleaner 2c as a third example of a work object 2. Fig. 6 shows a filter fan 2d as a fourth example of a work object 2.
[0010] Hereinafter, an embodiment for carrying out the present invention will be described with reference to the drawings. The scope necessary for the explanation to achieve the object of the present invention will be schematically shown, and the scope necessary for explaining the relevant parts of the present invention will be mainly explained, and the parts that are omitted from the explanation will be based on publicly known technology. (Work Management System 1)
[0011] FIG. 1 is an overall configuration diagram showing an example of a work management system 1 according to a first embodiment. The work management system 1 according to this embodiment functions as a system for managing work on a work target 2. The work management system 1 mainly comprises a database device 3, a work management device 5, and a user terminal device 6. The database device 3, the work management device 5, and the user terminal device 6 are configured, for example, as general-purpose or dedicated computers (see FIG. 8 described below), and are connected to a wired or wireless network 7 so as to be able to mutually transmit and receive various data (in FIG. 1, transmission and reception of some data is indicated by dashed arrows). Note that the number of database devices 3, the work management devices 5, and the user terminal devices 6, as well as the connection configuration of the network 7, are not limited to the example shown in FIG. 1 and may be changed as appropriate.
[0012] 2 is a perspective view showing a reducer 2a as a first example of the work object 2. An example of the work object 2 used in this embodiment is a reducer 2a including a circular spline 2a1, a flexspline 2a2, etc. The work object 21 is, for example, a bolt 21a fastened to the circular spline 2a1, or grease 21b applied between the circular spline 2a1 and the flexspline 2a2, etc.
[0013] (Tools 22) FIG. 3 shows a torque driver 22a and a grease gun 22b as examples of the tools 22. The tools 22 may be a torque driver 22a for tightening the bolts 21a, or a grease gun 22b for applying grease 21b between the circular spline 2a1 and the flexspline 2a2. The torque driver 22a preferably has a torque display 22a1 that displays the torque at the time of tightening. The grease gun 22b preferably has a scale 22b1 that displays the amount of grease applied. The tools 22 may be selected based on the work target portion 21. For example, a torque wrench or the like may be used instead of the torque driver 22a, and a brush or the like may be used instead of the grease gun 22b. The work target portion 21 and the work target 2 including the tools 22 constitute a work target.
[0014] The database device 3 is a device that manages work history information 30 relating to the history of when work has been performed. When work is performed on the work object 2, the database device 3 receives various reports R including at least a portion of the work information 112, etc. from the work management device 5 as needed, and registers the reports R in the work history information 30, thereby accumulating reports R including at least a portion of the work information 112, etc. relating to the work. Note that the database device 3 may also store various other information in addition to the above, and in such cases, the work management device 5 may refer to this information.
[0015] (Work management device 5) Fig. 4 is a block diagram showing an example of the work management device 5. Fig. 5 is a functional explanatory diagram showing an example of the work management device 5. The work management device 5 manages a plurality of works performed by a worker U on a predetermined work object 2. The work management device 5 includes a control unit 50 that acquires, generates, or processes various information for the works, a communication unit 51 that is connected to the database device 3, the user terminal device 6, etc. and functions as a communication interface that transmits and receives various data, and a memory unit 52 that stores various information such as various programs (such as an operating system or user terminal program) and data (learning model 10) used in the operation of the work management device 5.
[0016] The control unit 50 functions as an image data acquisition unit 501 , a work information generation unit 502 , an object information generation unit 503 , and an output processing unit 504 .
[0017] The image data acquisition unit 501 acquires image data 111 captured by the image capturing unit 63 that is disposed facing forward of the worker U when the worker U wears the user terminal device 6. In this embodiment, the image data acquisition unit 501 acquires (receives) the image data 111 from the user terminal device 6 via the communication unit 51 and the network 7.
[0018] The work information generation unit 502 generates work information 112 in chronological order for each task, which is to be performed by the worker U using the tool 22 on the work target portion 21 of the work object 2 at a predetermined work position, based on the image data 111 acquired by the image data acquisition unit 501 and the design information 521 previously stored in the storage unit 52. The generated work information 112 is stored in the storage unit 52.
[0019] For example, the work information generation unit 502 determines whether or not a design feature point of the work target part 21 stored as design information 521 is included within the angle of view when the image capturing unit 63 captures an image of real space, and when it detects that a design feature point is included, acquires the work information 112 based on the position of the feature point. The feature point may be based on, for example, the outer shape or outer color of the work target part 21, or on characters or a two-dimensional code on a sticker affixed to the work target part 21.
[0020] In addition, when design drawing data for each part of the work object 2 is stored in the memory unit 52 as design information 521, the work information generation unit 502 may refer to the design drawing data, and when it detects that the imaging range when the image capturing unit 63 captures the real space includes a feature point of the work object 21 in the design drawing data, it may acquire spatial position information of the work object 21 based on the feature point.
[0021] The object information generation unit 503 generates object information 113 based on the work information 112 generated by the work information generation unit 502 and the design information 521, for superimposing the work information 112 on the work object 2 present around the worker U when the user terminal device 6 is worn by the worker U.
[0022] The output processing unit 504 performs output processing to output the work information 112 generated by the work information generation unit 502 and the object information 113 generated by the object information generation unit 503. For example, the output processing unit 504 transmits the work information 112 to the user terminal device 6, causing the user terminal device 6 to output a display screen and audio based on the work information 112. Furthermore, the output processing unit 504 transmits the object information 113 to the user terminal device 6, causing the user terminal device 6 to output a display screen based on the object information 113.
[0023] The storage unit 52 pre-stores various information such as various programs (such as an operating system or a user terminal program) and data (such as the learning model 10) used in the operation of the work management device 5, design drawing data for each part as design information 521, and design specifications for the work target part 21 as specification information 522. The storage unit 52 also stores the work information 112 generated by the work information generation unit 502 as a record of the completion of the work. The storage unit 52 may be substituted by a storage unit of an external computer (for example, a server-type computer or a cloud-type computer), in which case the control unit 50 simply accesses the external computer.
[0024] 6 is a data structure diagram showing an example of various information. The various information shown in FIG. 6 includes work information 112, design information 521, and specification information 522.
[0025] The work information 112 is acquired when a predetermined work is performed on the work object 2. The work information 112 may include, for example, the work number, date and time, worker, work object, work part, tools, work results, and regulations. When one work process is performed by multiple tasks, the work information 112 may be configured as a work table 112a by registering multiple pieces of work information 112 in chronological order. The work information 112 may also have multiple work tables 112a.
[0026] The work number information may be assigned numbers in the order in which they were generated by the work information generation unit 502. Date and time information is identified by a clock or the like built into the work management device 5. The worker information is information that identifies the worker U, and may be assigned a number or the like to each worker in advance, and the number may be input into the work management device 5 when working, or may be acquired from the image data 111 by the image data acquisition unit 501.
[0027] The work object information identifies the work object 2, the work object part information identifies the work object part 21 of the work object 2, and the work tool information identifies the tool to be used to work on the work object part 21. The work object information, work object part information, and work tool information constitute work object information, and may be acquired based on the image data 111 acquired by the image data acquisition unit 501 and the design information 521 previously stored in the storage unit 52. The work status information is work value information, work order information, etc. acquired by the image data acquisition unit 501 from the image data 111 or from a sensor, etc. The work status information may also indicate whether or not work is being done on the work object part 21.
[0028] The design information 521 may be pre-stored information such as the work object 2 and its dimensions, the work target portion 21 and its position, and the tool 22 to be used on the work target portion 21. The design information 521 may be information that has already been input into 3D-CAD or the like.
[0029] The specification information 522 may be pre-stored information such as the work object, the work part, the tool, the specified value, the specified order, etc. The specification information 522 may be configured as a specification table 522b containing specified value information and specified order information (specified work order) of the tool 22 used to work on the work part 21. Even when different tools 22 are used to work on the same work part 21, the specified information is stored in the specification table 522a.
[0030] The specified value is a specified value that is predetermined for each tool 22 that performs work on the work target portion 21. For example, the specified value indicates the tightening torque value for each bolt, the amount of grease to be applied, etc. Furthermore, the specified order indicates a specified work order, such as the order in which bolts are tightened.
[0031] The stipulation information 522 may be acquired from the storage unit 52 that pre-stores appropriate values for the work to be performed by the tool 22 on the work target unit 21. When the work status information indicates whether or not work is to be performed on the work target unit 21, the order of the work, etc., the stipulation information 522 may indicate whether or not work is to be performed on the work target unit 21, the appropriate order of the work, etc.
[0032] (User Terminal Device 6) Fig. 7 is a block diagram showing an example of the user terminal device 6. The user terminal device 6 is a worker device used by the worker U when performing work on the work target 2. The user terminal device 6 is a portable device that the worker U can wear on the head or the like, and is configured as a wearable device such as smart glasses or a see-through head-mounted display. The user terminal device 6 includes a control unit 60, a communication unit 61, a storage unit 62, an image capturing unit 63 that can capture images of real space, an input unit 64, an output unit 65 that can superimpose and display object information 113 on the work target 2 in real space, and a group of sensors 66.
[0033] The image capturing unit 63 is composed of a camera (image sensor) such as a CMOS sensor or CCD sensor having a predetermined resolution (number of pixels), and is positioned facing forward of the worker U when the worker U wears the user terminal device 6. When the worker U works on the work target 2, the user terminal device 6 transmits image data 111 captured by the image capturing unit 63 positioned facing forward of the worker U to the work management device 5 as needed. When the worker U wears the user terminal device 6, the output unit 65 is positioned, for example, in front of one or both of the worker U's eyes. The user terminal device 6 receives object information 113 for the image data 111 and the work information 112 from the work management device 5 as needed, and displays the object information 113, for example, so as to be superimposed on the work target 2 existing in real space within the worker U's field of view.
[0034] The control unit 60 functions as an image data transmission processing unit 600, a work information processing unit 601, and an object information processing unit 602. The communication unit 61 is connected to external devices (e.g., the database device 3 and the work management device 5) via the network 7 and functions as a communication interface for transmitting and receiving various data. The memory unit 62 stores various programs (such as an operating system and a user terminal program) and data used in the operation of the user terminal device 6. The image capturing unit 63 captures images of real space and generates image data 111. The input unit 64 accepts various input operations, and the output unit 65 functions as a user interface by outputting various information via a display screen or audio. The sensor group 66 detects the acceleration, angular velocity, attitude, torque, etc. of the device itself.
[0035] The image data transmission processing unit 600 transmits image data 111 captured by the image capturing unit 63 at a predetermined capturing period to the work management device 5 via the communication unit 61 and the network 7 as needed.
[0036] The work information processing unit 601 receives work information 112 from the work management device 5 in response to the image data 111 sent to the work management device 5, and notifies the worker U of the work information 112, for example, via audio or a display screen from the output unit 65.
[0037] The object information processing unit 602 receives object information 113 from the work management device 5 in response to the image data 111 transmitted to the work management device 5, and based on the object information 113, causes the output unit 65 to superimpose the work information 112 on the reducer 2a, which is the work target 2 present in front of the worker U, thereby notifying the worker U of the work information 112. Note that the work management device 5 may also notify the worker U of specification information 522 pre-stored in the memory unit 52.
[0038] (Hardware Configuration of Each Device) Fig. 8 is a hardware configuration diagram showing an example of a computer 900. The control unit 50 of the work management device 5, the user terminal device 6, etc. are configured by a general-purpose or dedicated computer 900.
[0039] 8, the computer 900 includes, as its main components, a bus 910, a processor 912, a memory 914, an input device 916, an output device 917, a display device 918, a storage device 920, a communication I / F (interface) unit 922, an external device I / F unit 924, an I / O (input / output) device I / F unit 926, and a media input / output unit 928. Note that the above components may be omitted as appropriate depending on the application of the computer 900.
[0040] The processor 912 is composed of one or more arithmetic processing devices (such as a CPU (Central Processing Unit), MPU (Micro-Processing Unit), DSP (Digital Signal Processor), GPU (Graphics Processing Unit), NPU (Neural Processing Unit), etc.), and operates as a control unit that controls the entire computer 900. The memory 914 stores various data and programs 930, and is composed of, for example, a volatile memory (DRAM, SRAM, etc.) that functions as a main memory, a non-volatile memory (ROM), a flash memory, etc.
[0041] The input device 916 is composed of, for example, a keyboard, a mouse, a numeric keypad, an electronic pen, etc., and functions as an input unit. The output device 917 is composed of, for example, a sound (audio) output device, a vibration device, etc., and functions as an output unit. The display device 918 is composed of, for example, a liquid crystal display, an organic EL display, electronic paper, a projector, etc., and functions as an output unit. The input device 916 and the display device 918 may be integrated into one device, such as a touch panel display. The storage device 920 is composed of, for example, an HDD, an SSD, etc., and functions as a storage unit. The storage device 920 stores various data necessary for executing the operating system and the program 930.
[0042] The communication I / F unit 922 is connected to a network 940 such as the Internet or an intranet (which may be the same as the network 7 in FIG. 1 ) via a wired or wireless connection and functions as a communication unit that transmits and receives data to and from other computers in accordance with a predetermined communication standard. The external device I / F unit 924 is connected to an external device 950 such as a camera, printer, scanner, or reader / writer via a wired or wireless connection and functions as a communication unit that transmits and receives data to and from the external device 950 in accordance with a predetermined communication standard. The I / O device I / F unit 926 is connected to I / O devices 960 such as various sensors and actuators and functions as a communication unit that transmits and receives various signals and data, such as detection signals from sensors and control signals to actuators, to and from the I / O devices 960. The media input / output unit 928 is composed of, for example, drive devices such as a DVD drive or a CD drive, a memory card slot, and a USB connector, and reads and writes data from and to media (non-transitory storage media) 970 such as a DVD, CD, memory card, or USB memory.
[0043] In the computer 900 having the above configuration, the processor 912 loads the program 930 stored in the storage device 920 into the memory 914, executes the program, and controls each unit of the computer 900 via the bus 910. The program 930 may be stored in the memory 914 instead of the storage device 920. The program 930 may be recorded on the medium 970 in an installable file format or an executable file format, and provided to the computer 900 via the media input / output unit 928. The program 930 may be provided to the computer 900 by downloading it via the communication I / F unit 922 over the network 940. In addition, the computer 900 may realize the various functions realized by the processor 912 executing the program 930 using hardware such as an FPGA (Field-Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit).
[0044] The computer 900 may be, for example, a desktop computer or a portable computer, and may be any type of electronic device. The computer 900 may be a client computer, a server computer, a cloud computer, or an embedded computer such as a control panel or controller (including a microcomputer, a programmable logic controller, or a sequencer). The computer 900 may also be applied to devices other than the control unit 50 of the work management device 5 and the user terminal device 6.
[0045] (Work management method) Fig. 9 is a flowchart showing an example of a work management method using the work management apparatus 5 and the user terminal device 6. The following describes an example in which a worker U wearing the user terminal device 6 performs work on a work target portion 21 of a work object 2 using a tool 22. Note that the flowchart shown in Fig. 9 is executed repeatedly each time the user terminal device 6 starts capturing images using the image capturing unit 63 when, for example, the worker U inputs an instruction to start adjustment work.
[0046] First, in step S110, the image capturing unit 63 of the user terminal device 6 captures an image of the real space in front of the worker U performing the adjustment work, and generates image data 111. Then, the image data transmission processing unit 600 transmits the image data 111 captured by the image capturing unit 63 to the work management device 5.
[0047] Next, in step S120, the image data acquisition unit 501 of the work management apparatus 5 receives the image data 111 transmitted in step S110.
[0048] Next, in step S130, the work information generation unit 502 generates work information 112 for the image data 111 based on the image data 111 acquired in step S120 and the design information 521 and specification information 522 stored in the database device 3 or the memory unit 52.
[0049] In addition, in step S130, the output processing unit 504 may also transmit the work information 112 to the user terminal device 6, and in that case, the work information processing unit 601 of the user terminal device 6 may notify the worker U of the work information 112 via voice or a display screen.
[0050] Next, in step S140, the database device 3 or the storage unit 52 stores the work information 112. By storing the work information 112, it is possible to keep a record that the work has been completed appropriately.
[0051] Next, in step S150, the object information generation unit 503 generates object information 113 for superimposing the work information 112 on the work target object 2 present around the worker U, based on the work information 112, design information 521, and specification information 522 generated in step S130. Then, in step S160, the output processing unit 504 transmits the object information 113 generated in step S150 to the user terminal device 6 as output processing for outputting the object information 113.
[0052] Next, in step S170, when the object information processing unit 602 of the user terminal device 6 receives the object information 113 transmitted in step S160, it displays a virtual object indicating work information 112 for the work object 2 in real space on the output unit 65 of the user terminal device 6 based on the object information 113.
[0053] 10 is a diagram showing a first example in which task information 112 is superimposed on a task object 2 in real space. FIG. 11 is a diagram showing a second example in which task information 502B is superimposed on a task object 2 in real space. In the following, for simplicity of explanation, it is assumed that the angle of view of the image data 111 matches the field of view of the worker U, but the angle of view of the image data 111 and the field of view of the worker U do not have to match, and either one may be wider.
[0054] Figure 10 illustrates a case where a worker U is standing in front of the work object 2 and can reach the work object 21, and a portion of the work information 112 generated based on image data 111 captured by the image capturing unit 63 is displayed as virtual objects 120A to 120B.
[0055] The virtual object 120A shows work value information of the torque driver 22a as work information 112 for the torque driver 22a tightening the bolt 21a captured in the image data 111. The virtual object 120A also shows specified torque value information for the bolt 21a1 pre-stored in the storage unit 52 as specified information 522. Specifically, the specified tightening torque value for the bolt 21a1 is 15 Ncm, and the current working torque value is 15 Ncm. By showing the specified value to the worker U in this way, it is possible to ensure consistent work among multiple workers U. Furthermore, by storing this video, it is possible to confirm that the worker U tightened the bolt 21a1 with a torque value that is the same as the specified value.
[0056] Furthermore, virtual object 120B indicates bolt 21a2 to be tightened next after tightening bolt 21a1 captured in image data 111. The position of bolt 21a2 to be tightened next is indicated from specified order information stored in advance in storage unit 52 as specified information 522. In this way, by indicating the specified order in which each worker U should perform the next task, it is possible to perform consistent work among multiple workers U. Furthermore, by storing this video, it is possible to confirm the work order information for the bolts tightened by each worker U.
[0057] In FIG. 11 , the virtual object 120C indicates work presence / absence information of the grease application portion as the work target portion 21 captured in the image data 111. Specifically, it indicates whether the grease 21b has been properly applied to the grease application portion. The virtual object 120C indicates a case where the grease 21b has not been applied to at least a portion of the grease application portion. In this way, it is possible to show the worker U that the grease 21b has not been applied to at least a portion of the grease application portion, and to perform the work appropriately. Furthermore, by storing this image, it is possible to confirm that the worker U has properly applied the grease 21b.
[0058] As described above, the work management system 1, work management device 5, and work management method according to this embodiment can obtain work information 112 from image data 111 captured by the image capture unit 63, which is positioned facing forward when the worker U wears the user terminal device 6. The system can then automatically store the worker's work with high accuracy and appropriately check the work status. Furthermore, at least one of the work information 112 and predetermined specification information 522 can be displayed as object information 113 to assist the worker U in their work. Therefore, for work that has not been standardized or documented, the quality of the worker's individual work results can be converted into organizational knowledge and explicit knowledge. Furthermore, the work history of a worker performing high-quality work can be analyzed to provide instructions for reproducing the work information using XR.
[0059] 12 is a diagram illustrating an overall configuration of an example of a work management system 1 according to the second embodiment. The work management system 1 according to the second embodiment includes a machine learning device 4. The other configurations are the same as those of the first embodiment, and therefore will not be described again.
[0060] The machine learning device 4 is a device that operates as a main player in the learning phase of machine learning. The machine learning device 4 generates a learning model 10 to be used in the work management device 5 by machine learning, for example, based on a plurality of pieces of learning data 11. The trained learning model 10 is provided to the work management device 5 via the network 7, a recording medium, or the like.
[0061] The work management device 5 receives image data 111 at any time from a user terminal device 6 worn by a worker U performing work on a work object 2, and inputs the image data 111 into a learning model 10 provided by the machine learning device 4 to generate work information 112 when the worker U performs work on the work object 2, and transmits the work information 112 to the user terminal device 6, etc. at any time.
[0062] 13 is a block diagram showing an example of the machine learning device 4. The machine learning device 4 includes a control unit 40, a communication unit 41, a learning data storage unit 42, a trained model storage unit 43, an input unit 44, and an output unit 45.
[0063] The control unit 40 functions as a learning data acquisition unit 400 and a machine learning unit 401. The communication unit 41 is connected to external devices (e.g., the database device 3, the work management device 5, the user terminal device 6, a three-dimensional model device (not shown), etc.) via the network 7 and functions as a communication interface for transmitting and receiving various types of data. The input unit 44 accepts various input operations, and the output unit 45 functions as a user interface by outputting various types of information via a display screen or voice.
[0064] The learning data acquisition unit 400 acquires learning data 11 consisting of image data 111 as input data and task information 112 as output data. The learning data acquisition unit 400 may acquire the learning data 11, for example, in cooperation with an external device connected via the communication unit 41 and the network 7, or may acquire the learning data 11 by accepting input operations via the input unit 44 and the output unit 45. The learning data 11 is data used as teacher data (training data), verification data, and test data in supervised learning. The task information 112 is data used as a correct answer label in supervised learning.
[0065] The learning data storage unit 42 is a database that stores a plurality of sets of learning data 11 acquired by the learning data acquisition unit 400. The specific configuration of the database that constitutes the learning data storage unit 42 may be designed as appropriate.
[0066] The machine learning unit 401 performs machine learning using multiple sets of learning data 11 stored in the learning data storage unit 42. That is, the machine learning unit 401 inputs multiple sets of learning data 11 to the learning model 10 and causes the learning model 10 to learn the correlation between the image data 111 and the task information 112 included in the learning data 11, thereby generating a trained learning model 10.
[0067] The trained model storage unit 43 is a database that stores the trained learning model 10 (specifically, the adjusted weight parameter group) generated by the machine learning unit 401. The trained learning model 10 stored in the trained model storage unit 43 is provided to the actual system (e.g., the work management device 5) via the network 7, a recording medium, etc. Note that although the training data storage unit 42 and the trained model storage unit 43 are shown as separate storage units in FIG. 13 , they may be configured as a single storage unit.
[0068] 14 is a diagram showing an example of the learning model 10 and the learning data 11. The learning data 11 used for machine learning of the learning model 10 is composed of image data 111 and task information 112.
[0069] The image data 111 constituting the learning data 11 is captured by the image capturing unit 63 that is positioned facing forward of the worker U when the worker U wears the user terminal device 6. The image data 111 includes, for example, images captured when the worker U is performing work on the work object 2, and each part of the work object 2 is captured under various capturing conditions with different capturing positions, capturing angles, capturing ranges, etc.
[0070] The work information 112 constituting the learning data 11 includes at least a work object yield, work object part information, and work tool information. The work object information identifies the work object 2, the work object part information identifies the work object part 21 of the work object 2, and the work tool information identifies the tool to be used to work on the work object part 21.
[0071] The learning data acquisition unit 400 acquires image data 111 captured by the image capturing unit 63 under various capturing conditions, for example, by having an examiner wearing the user terminal device 6 use the work object 2 and assume a position and posture similar to that of an actual worker U when performing work on the work object 2, and also acquires part of the work information 112 for the position and posture when the image data 111 was captured. The work information 112 may be input by the examiner via the user terminal device 6, or may be input via the input unit 44 and the output unit 45.
[0072] In addition, the learning data acquisition unit 400 acquires, as image data 111, virtual space data that is expected to be captured by the image capturing unit 63 when it is assumed that the worker U is present at a specific position in a virtual space in which the three-dimensional shapes of each part of the work object 2 are reproduced, for example, using a three-dimensional model provided by a three-dimensional model device.
[0073] The learning model 10 employs, for example, a neural network structure and includes an input layer 100, an intermediate layer 101, and an output layer 102. Synapses (not shown) that connect each neuron are established between each layer, and each synapse is associated with a weight. A group of weight parameters consisting of the weights of each synapse is adjusted by machine learning.
[0074] The input layer 100 has neurons corresponding in number to each pixel of image data 111 as input data, and the pixel value of each pixel is input to each neuron. The output layer 102 has neurons corresponding in number to work information 112 as output data, and the prediction result (inference result) of the work information 112 for the image data 111 is output as output data.
[0075] The number of learning models 10 stored in the trained model storage unit 43 is not limited to one, and multiple learning models with different conditions may be stored, such as differences in machine learning techniques, mechanisms of the work object 2, specifications of the image capturing unit 63 (resolution and type of image sensor), and types of data included in the work information 112. In this case, the training data storage unit 42 may store multiple types of training data having data structures corresponding to multiple training models with different conditions.
[0076] (Machine Learning Method) FIG. 15 is a flowchart showing an example of a machine learning method performed by the machine learning device 4.
[0077] First, in step S210, the learning data acquisition unit 400 acquires a desired number of pieces of learning data 11 as a preliminary preparation for starting machine learning, and stores the acquired learning data 11 in the learning data storage unit 42. The number of pieces of learning data 11 to be prepared here may be set in consideration of the inference accuracy required for the learning model 10 to be finally obtained.
[0078] Next, in step S220, in order to start machine learning, the machine learning unit 401 prepares a pre-learning learning model 10. The pre-learning learning model 10 prepared here is configured as a neural network model, and the weights of each synapse are set to initial values.
[0079] Next, in step S230 , the machine learning unit 401 acquires, for example, one set of training data 11 at random from the multiple sets of training data 11 stored in the training data storage unit 53 .
[0080] Next, in step S240, the machine learning unit 401 inputs input data (image data 111) included in a set of training data 11 to the input layer 100 of the prepared training model 10 before training (or during training). As a result, output data (work information 112) is output as an inference result from the output layer 102 of the training model 10, and this output data was generated by the training model 10 before training (or during training). Therefore, in the pre-training (or training) state, the output data output as an inference result indicates information different from the correct answer label (work information 112) included in the training data 11.
[0081] Next, in step S250, the machine learning unit 401 performs machine learning by comparing the correct labels included in the set of learning data 11 acquired in step S230 with the output data output as the inference result from the output layer 102 in step S240 and performing a process of adjusting the weights of each synapse (backpropagation). In this way, the machine learning unit 401 causes the learning model 10 to learn the correlation between the input data and the output data.
[0082] Next, in step S260, the machine learning unit 401 determines whether a predetermined learning termination condition has been met, for example, based on the evaluation value of an error function based on the correct label contained in the learning data 11 and the output data output as the inference result, or the remaining number of unlearned learning data 11 stored in the learning data storage unit 42.
[0083] In step S260, if the machine learning unit 401 determines that the learning termination condition is not satisfied and that machine learning should be continued (No in step S260), the process returns to step S230, and the processes of steps S230 to S250 are performed multiple times on the learning model 10 under training using unlearned training data 11. On the other hand, in step S260, if the machine learning unit 401 determines that the learning termination condition is satisfied and that machine learning should be terminated (Yes in step S260), the process proceeds to step S270.
[0084] Then, in step S270, the machine learning unit 401 stores the trained learning model 10 (adjusted weight parameter group) generated by adjusting the weights associated with each synapse in the trained model storage unit 43, thereby completing the series of machine learning methods shown in Fig. 15. In the machine learning method, step S210 corresponds to a training data storage step, steps S220 to S260 correspond to a machine learning step, and step S270 corresponds to a trained model storage step.
[0085] As described above, the machine learning device 4 and machine learning method according to this embodiment can provide a learning model 10 that can predict (infer) work information 112 from image data 111 captured by the image capturing unit 63 that is positioned facing forward of the worker U when the worker U wears the user terminal device 6.
[0086] (Work management device 5) Fig. 16 is a block diagram showing an example of a work management device 5 according to the second embodiment. Fig. 17 is a functional explanatory diagram showing an example of a work management device 5 according to the second embodiment. The work management device 5 includes a control unit 50, a communication unit 51, and a storage unit 52.
[0087] The control unit 50 functions as an image data acquisition unit 501, a work information generation unit 502, an object information generation unit 503, and an output processing unit 504. The communication unit 51 is connected to external devices (e.g., the database device 3, the machine learning device 4, and the user terminal device 6) via the network 7 and functions as a communication interface for sending and receiving various data. The storage unit 52 stores various programs (such as an operating system and a user terminal program) and data (learning model 10) used in the operation of the work management device 5.
[0088] The image data acquisition unit 501 acquires image data 111 captured by the image capturing unit 63 that is disposed facing forward of the worker U when the worker U wears the user terminal device 6. In this embodiment, the image data acquisition unit 501 acquires (receives) the image data 111 from the user terminal device 6 via the communication unit 51 and the network 7.
[0089] The work information generation unit 502 generates work information 112 based on the image data 111 acquired by the image data acquisition unit 501. In this embodiment, the work information generation unit 502 generates work information 112 for the image data 111 by inputting the image data 111 acquired by the image data acquisition unit 501 into a learning model 10 that has undergone machine learning to determine the correlation between the image data 111 and the work information 112.
[0090] The memory unit 52 stores a trained learning model 10 used by the work information generation unit 502. The number of learning models 10 stored in the memory unit 52 is not limited to one, and multiple trained models with different conditions, such as different machine learning methods, different mechanisms of the work object 2, different specifications of the image capture unit 63 (resolution or type of image sensor), and different types of data included in the work information 112, may be stored and used selectively or in parallel. The memory unit 52 may be substituted by a memory unit of an external computer (e.g., a server-type computer or a cloud-type computer), and in that case, the work information generation unit 502 may access the external computer.
[0091] The object information generation unit 503 generates object information 113 based on the work information 112 generated by the work information generation unit 502, for superimposing the work information 112 on the work object 2 present around the worker U when the user terminal device 6 is worn by the worker U.
[0092] The output processing unit 504 performs output processing to output the work information 112 generated by the work information generation unit 502 and the object information 113 generated by the object information generation unit 503. For example, the output processing unit 504 transmits the work information 112 to the user terminal device 6, causing the user terminal device 6 to output a display screen and audio based on the work information 112. Furthermore, the output processing unit 504 transmits the object information 113 to the user terminal device 6, causing the user terminal device 6 to output a display screen based on the object information 113.
[0093] 18 is a block diagram showing an example of the user terminal device 6. The user terminal device 6 includes a control unit 60, a communication unit 61, a storage unit 62, an image capturing unit 63, an input unit 64, an output unit 65, and a group of sensors 66.
[0094] The control unit 60 functions as an image data transmission processing unit 600, a work information processing unit 601, and an object information processing unit 602. The communication unit 61 is connected to external devices (e.g., the database device 3, the machine learning device 4, and the work management device 5) via the network 7 and functions as a communication interface for transmitting and receiving various data. The memory unit 62 stores various programs (such as an operating system and a user terminal program) and data used in the operation of the user terminal device 6. The image capturing unit 63 captures images of real space and generates image data 111. The input unit 64 accepts various input operations, and the output unit 65 functions as a user interface by outputting various information via a display screen or audio. The sensor group 66 detects the acceleration, angular velocity, attitude, etc. of the device itself.
[0095] The image data transmission processing unit 600 transmits image data 111 captured by the image capturing unit 63 at a predetermined capturing period to the work management device 5 via the communication unit 61 and the network 7 as needed.
[0096] The work information processing unit 601 receives work information 112 from the work management device 5 in response to the image data 111 sent to the work management device 5, and notifies the worker U of the work information 112, for example, via audio or a display screen from the output unit 65.
[0097] The object information processing unit 602 receives object information 113 from the work management device 5 in response to the image data 111 sent to the work management device 5, and based on the object information 113, notifies the worker U of the work information 112 by superimposing the work information 112 on the work object 2 present around the worker U using the output unit 65.
[0098] (Work management method) Fig. 19 is a flowchart showing an example of a work management method by the work management system 1. The following description will be given as an example in which a worker U wearing a user terminal device 6 uses a tool 22 to work on a work target portion 21 of a work object 2. Note that the flowchart shown in Fig. 19 is executed repeatedly each time the user terminal device 6 starts capturing images using the image capturing unit 63 when, for example, the worker U inputs an instruction to start adjustment work.
[0099] First, in step S310, the image capturing unit 63 of the user terminal device 6 captures an image of the real space in front of the worker U performing the adjustment work, and generates image data 111. Then, the image data transmission processing unit 600 transmits the image data 111 captured by the image capturing unit 63 to the work management device 5.
[0100] Next, in step S320, the image data acquisition unit 501 of the work management apparatus 5 receives the image data 111 transmitted in step S310.
[0101] Next, in step S330, the work information generation unit 502 generates work information 112 for the image data 111 based on the image data 111 acquired in step S320 and the learning model 10 stored in the storage unit 52.
[0102] In addition, in step S330, the output processing unit 504 may also transmit the work information 112 and the specification information 522 to the user terminal device 6, and in that case, the work information processing unit 601 of the user terminal device 6 may notify the worker U of the work information 112 and the specification information 522 via voice or a display screen.
[0103] Next, in step S340, the storage unit 52 stores the work information 112. By storing the work information 112, it is possible to keep a record that the work has been completed appropriately. The work information 112 may also be stored in the database device 3.
[0104] Next, in step S350, the object information generation unit 503 generates object information 113 based on the task information 112 generated in step S330, for superimposing the task information 112 on the task target 2 present around the worker U. Then, in step S360, the output processing unit 504 transmits the object information 113 generated in step S350 to the user terminal device 6 as output processing for outputting the object information 113.
[0105] Next, in step S370, upon receiving the object information 113 transmitted in step S360, the object information processing unit 602 of the user terminal device 6 displays, on the output unit 65 of the user terminal device 6, a virtual object indicating the task information 112 for the task target 2 in real space, based on the object information 113. An image such as that shown in FIG. 11 or 12 is displayed on the output unit 65.
[0106] As described above, the work management system 1, work management device 5, and work management method according to this embodiment can obtain work information 112 from image data 111 captured by the image capture unit 63, which is positioned facing forward when the worker U wears the user terminal device 6. The system can then automatically store the worker's work with high accuracy and appropriately check the work status. Furthermore, at least one of the work information 112 and predetermined specification information 522 can be displayed as object information 113 to assist the worker U in their work. Therefore, for work that has not been standardized or documented, the quality of the worker's individual work results can be converted into organizational knowledge and explicit knowledge. Furthermore, the work history of a worker performing high-quality work can be analyzed to provide instructions for reproducing the work information using XR.
[0107] (Work object 2 (second example)) Figure 20 shows a drive unit 2b as a second example of the work object 2. An example of the work object 2 in the work information 112 used in this embodiment is a drive unit 2b in which an AC servo motor 2b1 and a reducer 2b2 are connected together. Furthermore, an example of the work target part 21 is a bolt, a shaft, a rubber cap, etc. In work number 1-1, the work object 2 is the reducer 2b2, and the work target part 21 is an input shaft coupling bolt (not shown). However, since the work is simply to align a bolt with a hole, there is no work information 112 related to the tool 22, there is no work value information as work information 112, and the work sequence information is 1. Work information 112 such as that shown in Figure 20 is stored in the memory unit 52 or the database device 3.
[0108] (Work Object 2 (Third Example)) FIG. 21 shows a stopper of a roll cleaner 2c as a third example of the work object 2. An example of the work object 2 in the work information 112 used in this embodiment is a roll cleaner 2c that cleans wafers W in a CMP apparatus. The work object 21 is a stopper bolt 21c1 of a height-adjustable stopper, and the tool 22 is a jig 22c1, a wrench 22c2, a wrench, or the like. In work number 1-1, the jig 22c1, which has the same diameter as the roll, is brought into contact with the wafer W. Next, in work number 1-2, the stopper bolt 21c1 is rotated with the wrench 22c2 to temporarily position the stopper at height a. Next, in work number 1-3, the stopper bolt 21c1 is rotated with the wrench 22c2 to move the stopper a specified amount from the temporarily determined position a. Work information 112 such as that shown in FIG. 21 is stored in the storage unit 52 or the database device 3.
[0109] (Work Object 2 (Fourth Example)) FIG. 22 shows a filter fan 2d as a fourth example of the work object 2. Examples of the work object 2 used in this embodiment are the filter fan 2d1 and the controller 2d2. Examples of the work object 21 are measurement points 21d1 to 21d9 and the dial 21d10 located a predetermined distance below the filter fan 2d1. The tool 22 is an anemometer 22d1. For work numbers 1-1 to 1-9, the anemometer 22d1 measures wind speed as work value information from the first measurement point 21d1 to the ninth measurement point 21d9. During measurement, the work management device 5 may store the measurement points as specified information 522 in the storage unit 52 in advance and display them on the screen. Next, the average value of the measured values from the first measurement point 21d1 to the ninth measurement point 21d9 is calculated. If this average value is within a predetermined specified range, the work information 112 is stored in the storage unit 52 or the database device 3. If the average value is outside the predetermined range, the dial 21d10 is changed and the work is continued until the average value falls within the predetermined range, and the work information 112 is stored in the storage unit 52 or the database device 3.
[0110] As described above, the work management system 1, work management device 5, and work management method according to this embodiment can obtain work information 112 from image data 111 captured by the image capture unit 63, which is positioned facing forward when the worker U wears the user terminal device 6, regardless of the work target 2. The system can then automatically store the worker's work with high accuracy and appropriately check the work status. Furthermore, at least one of the work information 112 and predetermined specification information 522 can be displayed as object information 113 to assist the worker U in their work. Therefore, for work that has not been standardized or documented, the quality of the worker's individual work results can be converted into organizational knowledge and explicit knowledge. Furthermore, the work history of a worker performing high-quality work can be analyzed to provide instructions for reproducing the work information using XR.
[0111] That is, the work information 112 is stored by the work management device 5, and the worker U can grasp the work status by visually checking the virtual objects 120A to 120C displayed on the output unit 65. Then, as the position and orientation of the worker U's body change and the shooting conditions for the image data 111 change, the process of generating the work information 112 and the object information 113 is repeated. This allows the worker U to perform the work appropriately.
[0112] In the above work management method, steps S120 and S220 correspond to an image data acquisition step, steps S130 and S230 correspond to a work information generation step, steps S140 and S240 correspond to a work information storage step, and steps S150 and S250 correspond to an object information generation step.
[0113] (Other Embodiments) The present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the spirit and scope of the present invention, all of which are included in the technical concept of the present invention.
[0114] In the above embodiment, the database device 3, the machine learning device 4, the work management device 5, and the user terminal device 6 are described as being configured as separate devices. However, these four devices may be configured as a single device, or any two or three of these four devices may be configured as a single device. Furthermore, at least one of the machine learning device 4 and the work management device 5 may be incorporated into the user terminal device 6. For example, the learning model 10 may be stored in the memory unit 62 of the user terminal device 6, and the control unit 60 of the user terminal device 6 may function as the image data acquisition unit 501, the work information generation unit 502, and the object information generation unit 503. Furthermore, the work information, design information, specification information, work history information, etc. may be stored in at least one of the database device 3, the memory unit 52 of the work management device 5, the memory unit 62 of the user terminal device 6, etc.
[0115] In the above embodiment, a neural network is used as the learning model for realizing machine learning by the machine learning unit 401. However, other machine learning models may be used. Examples of other machine learning models include tree-type models such as decision trees and regression trees, ensemble learning such as bagging and boosting, neural network-type models (including deep learning) such as recurrent neural networks, convolutional neural networks, and LSTM, clustering-type models such as hierarchical clustering, non-hierarchical clustering, k-nearest neighbors, and k-means, multivariate analysis such as principal component analysis, factor analysis, and logistic regression, and support vector machines.
[0116] (Machine Learning Program and Work Management Program) The present invention can also be provided in the form of a program (machine learning program) that causes the computer 900 to function as each unit included in the machine learning device 4, or a program (machine learning program) that causes the computer 900 to execute each step included in the machine learning method. The present invention can also be provided in the form of a program (work management program) that causes the computer 900 to function as each unit included in the work management device 5 or the user terminal device 6, or a program (work management program) that causes the computer 900 to execute each step included in the work management method according to the above embodiment.
[0117] (Inference Device, Inference Method, and Inference Program) The present invention can be provided not only in the form of the work management device 5 (work management method or work management program) according to the above embodiment, but also in the form of an inference device (inference method or inference program) used to infer work information. In this case, the inference device (inference method or inference program) includes a memory and a processor, and the processor executes a series of processes. The series of processes includes image data processing (image data acquisition step) for acquiring image data 111, and inference processing (inference step) for inferring work information 112 based on the image data acquired in the image data acquisition process.
[0118] By providing it in the form of an inference device (inference method or inference program), it can be more easily applied to various devices than when a work management device is implemented. It will be naturally understood by those skilled in the art that when the inference device (inference method or inference program) infers work information, the inference method implemented by the work information generation unit may be applied using a trained learning model generated by the machine learning device and machine learning method according to the above embodiments.
[0119] LIST OF SYMBOLS 1...Work management system, 3...Database device, 4...Machine learning device, 5...Work management device, 6...User terminal device (worker device), 7...Network, 10...Learning model, 11...Learning data 40...Control unit, 41...Communication unit, 42...Learning data storage unit, 43...Learned model storage unit, 44...Input unit, 45...Output unit, 50...Control unit, 51...Communication unit, 52...Storage unit, 60...Control unit, 61...Communication unit, 62...Storage unit, 63...Image capturing unit, 64...Input unit, 65...Output unit, 66...Sensor group, 111...Image data, 112...Work information, 113...Object information, 120A to 123A...Virtual objects, 400...Learning data acquisition unit, 401...Machine learning unit, 501...Image data acquisition unit, 502...Work information generation unit, 503...Object information generation unit, 504...Output processing unit, 600...image data transmission processing section, 601...work information processing section, 602...object information processing section, 900...computer, U...worker, W...wafer
Claims
1. A work management device that manages a number of tasks performed by a worker on a specified object, comprising: an image data acquisition unit that acquires image data captured by an image capture unit that is positioned facing forward of the worker when the worker wears a worker device equipped with an image capture unit; a work information generation unit that generates work information for each task performed by the worker on a specified work object based on the image data acquired by the image data acquisition unit; and a memory unit that stores the work information generated for each task by the work information generation unit in chronological order.
2. The work management device of claim 1, wherein the work information includes at least one of the following: date and time information indicating the date and time when the work was performed; worker information indicating the worker; work object information indicating the work object; work tool information indicating the tool used in the work; and work status information indicating the status of the work.
3. A work management device as described in claim 2, wherein the work information includes work value information indicating a work value of the tool, and the specified information includes specified value information indicating a specified value of the tool for the work that is predetermined based on the work tool information and the target information.
4. The work management device according to claim 2, wherein the work information includes work order information indicating the order of the work, and the specified information includes specified order information indicating a predetermined order of the work.
5. The work management device of claim 1, further comprising an object information generation unit that generates the object information for superimposing the work information on the work target present around the worker when the worker device is worn by the worker, the object information generation unit being further equipped with an output unit capable of superimposing object information on the work target in real space based on the work information.
6. The work management device of claim 3, wherein the memory unit pre-stores specification information indicating design specifications for the work object, and the object information generation unit generates the object information for superimposing the specification information on the work object present around the worker.
7. The work management device according to claim 1, wherein the memory unit pre-stores design information indicating a design shape of the object, and the work information generation unit generates the work information based on the image data and the design information.
8. The work management device of claim 1, wherein the work information generation unit generates the work information for the image data by inputting the image data acquired by the image data acquisition unit into a learning model that has undergone machine learning to determine the correlation between the image data and the work information.
9. A work management system comprising: a worker device having an image capturing unit arranged facing forward of the worker, worn by the worker performing a plurality of tasks on a specified object; a machine learning device having: a learning data memory unit that stores image data captured by the image capturing unit and a plurality of sets of learning data configured as work information for each task of the task performed by the worker on a specified work object; a machine learning unit that causes the learning model to learn a correlation between the image data and the work information by inputting the plurality of sets of learning data into a learning model, and a trained model memory unit that stores the learning model in which the correlation has been trained by the machine learning unit; and a work management device having an image data acquisition unit that acquires the image data, a work information generation unit that generates the work information for the image data by inputting the image data acquired by the image data acquisition unit into the learning model, and a memory unit that stores the work information generated for each task by the work information generation unit in chronological order.
10. An inference device comprising a memory and a processor, wherein the processor executes an image data acquisition process for acquiring image data captured by an image capturing unit positioned facing forward of a worker performing a specified task on a specified object when the worker wears a worker device equipped with an image capturing unit, and an inference process for inferring the task performed by the worker on a specified work object as task information for each task based on the image data acquired by the image data acquisition process.
11. A machine learning device comprising: a learning data storage unit that stores image data captured by an image capturing unit that is positioned facing forward of a worker when the worker wears a worker device equipped with an image capturing unit to perform a plurality of tasks on a specified object, and multiple sets of learning data configured as task information for each task performed by the worker on a specified work object; a machine learning unit that inputs the multiple sets of learning data into a learning model, thereby causing the learning model to learn a correlation between the image data and the task information; and a trained model storage unit that stores the learning model in which the correlation has been learned by the machine learning unit.
12. A work management method for managing a plurality of tasks performed by a worker on a specified object, comprising: an image data acquisition process for acquiring image data captured by an image capturing unit that is positioned facing forward of the worker when the worker wears a worker device equipped with an image capturing unit; a work information generation process for generating work information for each task performed by the worker on a specified work object based on the image data acquired by the image data acquisition process; and a storage process for storing the work information generated for each task in chronological order.
13. An inference method executed by an inference device having a memory and a processor, wherein the processor executes an image data acquisition process of acquiring image data captured by an image capturing unit positioned facing forward of a worker performing a specified task on a specified object when the worker wears a worker device equipped with an image capturing unit, and an inference process of inferring, when the image data is acquired in the image data acquisition process, the task performed by the worker on the specified work object as task information based on the image data.
14. A machine learning method comprising: a learning data storage step of storing in a learning data storage unit multiple sets of image data captured by an image capturing unit positioned facing forward of a worker when the worker performs work on a specified object wears the worker device having the image capturing unit, and learning data configured as work information for each work performed by the worker on a specified work object; a machine learning step of inputting the multiple sets of learning data into a learning model, thereby causing the learning model to learn the correlation between the image data and the work information; and a learned model storage step of storing in a learned model storage unit the learning model that has learned the correlation by the machine learning step.
Citation Information
Patent Citations
Work support system and work support method
JP2021114700A
Analysis device, analysis system, analysis method, program, and storage medium
JP2023045089A
