Work guidance method and system using a work guidance device

Abstract

The present invention relates to a work guidance method using a work guide device, which is carried out by a system, and includes the steps of: (a) a work guide device worn on a worker's head capturing an image of the worker's forward field of view, transmitting the captured image to an edge server, collecting fastening status data through a fastening device, and transmitting the collected fastening status data to the edge server; (b) the edge server analyzing the image received from the work guide device using a pre-trained work analysis model to determine the type of work being performed by the worker and whether or not there is a work error; and (c) the edge server providing work instruction information corresponding to the determined type of work and whether or not there is a work error through the work guide device, wherein the work instruction information is at least one of guidance information for the current work, information on whether or not there is an error, and guidance information for the next work, and the work analysis model is a model that learns images of worker movements for each work type, images of work objects and work tools as learning data, and outputs the type of work and whether or not there is a work error from the input images.

Description

Technical Field

[0001] The present invention relates to a work guidance method and system using a work guidance device. More specifically, the present invention relates to a method and system for providing a work guidance device that recognizes the work currently being performed by a worker through a work guidance device worn on the worker's head and performs corresponding work instructions and self-inspections.

Background Art

[0002] According to research by the Statistics Korea, the manufacturing industry in Korea is expected to grow at an average annual growth rate of 41% over a period of about four years, from about 22.4 trillion won in 2020 to about 360 trillion won in 2024. It has been investigated that currently, more than about 25,870 business entities are participating in the Korean manufacturing industry, and about 1,316,000 workers are engaged in it.

[0003] Such a manufacturing industry is dominated by assembly and processing-type manufacturing industries such as machinery and equipment, office equipment, electrical machinery, video and communication equipment, precision equipment, optical equipment, automobiles, fabricated metals, and transportation equipment, which account for more than 54% of the total.

[0004] The conventional assembly and processing-type manufacturing industry has a structure in which, in a manner unbefitting its position as a huge industry as described above, multiple workers assemble and process various heterogeneous parts, inevitably resulting in defects during production. To solve this problem, there is a problem that self-inspection must be carried out immediately after production or during the production process.

[0005] Such self-inspection reduces the defect rate of production products as it is carried out more frequently, but there is a possibility that the production quantity may decrease due to loss of working time, rather decreasing the production efficiency.

[0006] Therefore, in order to solve the problems of the prior art, there is a need for a method and system for providing a work guidance device that recognizes the work currently being performed by a worker through a work guidance device worn on the worker's head and performs corresponding work instructions and self-inspections.

Summary of the Invention

[0007] The present invention aims to solve the problems of the prior art described above and to provide a work guidance method and system using a work guidance device.

[0008] This aims to maximize production efficiency by enabling self-inspections to be conducted simultaneously with the work being performed by the worker, rather than interrupting the work and conducting separate self-inspections, and by providing the information from these inspections to the work guidance device.

[0009] The problems that this invention aims to solve are not limited to those mentioned above, and other problems not mentioned can be clearly understood from the following description. [Means for solving the problem]

[0010] As a technical means for achieving the above-mentioned technical challenges, a work guidance method by a work guidance device, performed by a system according to one embodiment of the present invention, includes the steps of: (a) a work guidance device attached to the worker's head captures the worker's forward field of view, transmits the captured image to an edge server, collects fastening status data through a fastening device, and transmits the collected fastening status data to the edge server; (b) the edge server analyzes the image received from the work guidance device through a pre-trained work analysis model to determine the type of work the worker is performing and whether or not there is a work error; and (c) the edge server provides work instruction information corresponding to the determined type of work and whether or not there is a work error through the work guidance device, wherein the work instruction information is at least one of guidance information for the current work, information on whether or not there is an error, and guidance information for the next work, and the work analysis model may be a model that learns images of the worker's movements for each type of work and images of the work object and work tools as training data, and outputs the type of work and whether or not there is a work error from the input image.

[0011] Furthermore, step (a) may involve the fastening device rotating in response to a control signal while in contact with the fastening screw to rotate the fastening screw and fasten it to the workpiece, and collecting at least one of the following as fastening status data: the rotational speed (RPM) of the motor of the fastening device and the fastening screw per unit time generated during the fastening operation, and the torque value, and providing this data to the edge server.

[0012] Furthermore, step (b) may include (b-1) the step in which the edge server classifies the images received from the work guide device into work objects, work tools, the worker's body and other objects according to an image analysis technique; (b-2) the step in which the images of the classified work objects and work tools are compared with pre-stored images of work types and work progress, and if the match is greater than or equal to a preset value, the current work progress is determined to be the work progress; the steps in which the images of the classified work objects and fastening devices are compared with pre-stored images of work errors, and if the match is greater than or equal to a preset value, the presence or absence of work errors in the current work is determined; and (b-3) the step in which the images of the classified work objects and work tools match with images of the worker currently working, greater than or equal to a preset value, the step in which the work progress is determined to be the current work.

[0013] Furthermore, step (b-2) above may involve detecting abnormal fastening between the fastening screw and the workpiece based on the RPM and torque values ​​of the fastening status data collected from the fastening device, by comparing them with a pre-stored graph of abnormal fastening types, thereby determining whether or not there is a work error.

[0014] Furthermore, the work guide device may include a support portion that rests on the bridge of the worker's nose and supports the main body, a main body having a predetermined thickness and formed in a semicircular shape at a predetermined distance above the worker's eyebrows and connected to the upper side of the support portion, and a leg portion that extends from the main body and is fixed to the worker's ears, but is attached to the outside of the worker's head and includes a camera device that captures the worker's forward field of view, and a bone conduction speaker device that outputs voice corresponding to work instruction information and control hand signals in response to control signals received from the edge server is built inside the main body.

[0015] Furthermore, the work guide device may further include a glass portion formed to be separated from the worker's eyeballs by a predetermined distance and not to cover the image captured by the camera device, extending from the lower side of the main body and made of a transparent or translucent material so as not to obstruct the worker's field of view.

[0016] Furthermore, the work guide device includes a main body having a predetermined thickness, which is formed in a semicircular shape to contact and support the back of the worker's head; a support portion having a predetermined thickness, which is formed to contact the side of the worker's head and extends from the main body to the worker's temple above the worker's ear, resting on the worker's ear and supporting the main body; and a camera device attached to the support portion in the direction toward the outside of the worker's head to capture the worker's forward field of view. The main body may have a bone conduction speaker device built inside that outputs audio corresponding to work instruction information and control hand signals in response to control signals received from an edge server.

[0017] Furthermore, in step (c) above, the work guide device may provide the type of work received from the edge server, the work progress, and work instruction information corresponding to work errors, through a bone conduction speaker device inside the work guide device, in the form of voice corresponding to the work instruction information and voice of control hand signals.

[0018] Furthermore, in step (c) above, if the work guide device determines that the current work procedure is incorrect or that a work error has occurred, it may provide a work error signal including at least one of sound, vibration, or video.

[0019] Furthermore, in step (c) above, the work guide device may provide work instruction information corresponding to the type of work and the progress of work received from the edge server through an AR image displayed on the glasses portion of the work guide device, superimposed on the worker's field of view.

[0020] Furthermore, in step (c) above, the type of work being performed by the worker and whether or not there are any work errors are determined, and if it is determined that the work has been completed without errors, the next pre-set work instruction information is provided to the work guide device, the part of the worker's hand in the image received from the camera device of the work guide device is recognized, and if a control hand signal is extracted from the recognized part of the hand, the work instruction information corresponding to the extracted control hand signal is provided to the work guide device with priority over the next pre-set work instruction information.

[0021] Furthermore, (d) the edge server provides the main server with at least one of the following for multiple tasks: video data, the number of work errors, the number of gesture inputs, the work yield, and the work time, and the main server automatically generates an analysis report for each worker who performed each task and provides it to the administrator terminal.

[0022] According to an embodiment of the present invention, a work guidance system using a work guidance device includes a head-mounted bone conduction work guidance device that captures the forward field of view of an operator and transmits the captured image to a server, a fastening device that collects fastening state data and transmits the collected fastening state data to an edge server, and an edge server that analyzes the image and fastening state data received from the work guidance device and the fastening device through a pre-trained work analysis model, grasps the type of work the operator is performing and the presence or absence of work errors, and provides at least one of the grasped work type, work instruction information regarding the work progress and the presence or absence of errors, and guidance information for the next work through the work guidance device. The work instruction information is at least one of information regarding the presence or absence of errors in the current work and guidance information for the next work. The work analysis model may be a model that learns the operation images of the operator by work type and the images of the work object and work tool as learning data and outputs the work type and the presence or absence of work errors from the input image.

Advantages of the Invention

[0023] The present invention provides a work guidance method and system using a work guidance device, so that for the work being performed by an operator during work, an autonomous inspection is carried out simultaneously with the work, and by providing information regarding the inspection to the work guidance device, the production efficiency can be maximized.

[0024] In addition, the work guidance device grasps the work object and work being performed currently, and provides work instruction information corresponding to the grasped work through intuitive means such as voice, video, vibration, and light emission, so as to assist an operator who is not familiar with the work to acquire a work ability above a certain level.

Brief Description of the Drawings

[0025] [Figure 1] It is an exemplary diagram of a work guidance system using a work guidance device according to an embodiment of the present invention. [Figure 2]A block diagram showing the internal configuration of an edge server according to an embodiment of the present invention. [Figure 3] A conceptual diagram of a video and image processing algorithm according to an embodiment of the present invention. [Figure 4] An exemplary diagram of a work guidance device according to an embodiment of the present invention. [Figure 5] An exemplary diagram of gesture recognition of an operator of a work guidance device according to an embodiment of the present invention. [Figure 6] A flowchart of the execution procedure of a work guidance method by a work guidance device according to an embodiment of the present invention.

Embodiments for Carrying Out the Invention

[0026] Hereinafter, referring to the accompanying drawings, embodiments of the present invention will be described in detail so that those having ordinary knowledge in the technical field to which the present invention pertains can easily implement it. However, the present invention can be embodied in various different forms and is not limited to the embodiments described herein. And, in order to clearly explain the present invention in the drawings, parts not related to the explanation are omitted, and similar parts throughout the specification are denoted by similar reference numerals.

[0027] Throughout the specification, when a part is "connected" to another part, it includes not only the case of being "directly connected" but also the case of being "electrically connected" with other elements interposed therebetween. Also, when a part "includes" a certain component, it means that other components can be further included without excluding other components unless otherwise stated.

[0028] In this specification, “Unit” includes units implemented by hardware, units implemented by software, and units implemented using both. Furthermore, one unit may be implemented using two or more hardware components, and two or more units may be implemented by one hardware component. On the other hand, “~Unit” is not limited to software or hardware; “~Unit” may be configured to reside in an addressable storage medium, or to regenerate one or more processors. Therefore, as an example, “~Unit” includes components such as software components, object-oriented software components, class components, and task components, as well as processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functions provided within the components and “~Unit” may be combined with fewer components and “~Unit” or further separated into additional components and “~Unit”. In addition, components and “~Unit” may be embodied to regenerate one or more CPUs within a device or secure multimedia card.

[0029] The “terminal” referred to below can be a computer or mobile device that can connect to a server or other terminals via a network. Here, a computer can include, for example, a laptop, desktop, or laptop computer equipped with a web browser, or a VR HMD (e.g., HTC VIVE, Oculus Rift, GearVR, DayDream, PSVR, etc.). Here, a VR HMD includes all types: PC-based (e.g., HTC VIVE, Oculus Rift, FOVE, Deepon, etc.), mobile-based (e.g., GearVR, DayDream, Storm Mirror, Google Cardboard, etc.), console-based (PSVR), and independently embodied Stand Alone models (e.g., Deepon, PICO, etc.). Mobile devices include, for example, smartphones, tablet PCs, and wearable devices, as wireless communication devices that guarantee portability and mobility, as well as various devices equipped with communication modules such as Bluetooth (BLE, Bluetooth Low Energy), NFC, RFID, Ultrasonic, infrared, Wi-Fi, and LiFi. Furthermore, "network" refers to a connected structure that enables information exchange between each node, such as terminals and servers, and includes local area networks (LANs), wide area networks (WANs), the Internet (WWW), wired and wireless data communication networks, telephone networks, and wired and wireless television communication networks.Examples of wireless data communication networks include, but are not limited to, 3G, 4G, 5G, 3GPP (registered trademark: 3rd Generation Partnership Project), LTE (Long Term Evolution), WiMAX (World Interoperability for Microwave Access), Wi-Fi, Bluetooth communication, infrared communication, ultrasonic communication, visible light communication (VLC), and LiFi.

[0030] This invention relates to a work guidance method using a work guidance device, which recognizes the work currently being performed by the worker through a work guidance device worn on the worker's head, provides corresponding work instructions by various means, and automatically performs self-inspections without interrupting the work.

[0031] Referring to Figure 1, a work guide system using a work guide device according to one embodiment of the present invention will be described.

[0032] A work guide system using a work guide device according to one embodiment of the present invention includes an edge server (100), a work guide device (200), a main server (300), and an administrator terminal (400).

[0033] The edge server (100) may be installed in the workspace where the work is performed, for example, on a workbench or workstation, and communicates with the work guide device (200) and the main server (300) via wired or wireless connections, provides work instruction information to the work guide device (200), and performs self-inspections of the work currently being performed based on the data received from the work guide device (200).

[0034] The main server (300) may be implemented in the form of a normal server structure or a cloud server, and may automatically generate an analysis report based on multiple data received from the edge server (100) and provide it to the administrator terminal (400), which is a user terminal owned by the person responsible for the work or self-inspection.

[0035] Referring to Figure 2, an edge server (100) and a main server (300) according to one embodiment of the present invention may be configured to include a memory in which a program (or application) for performing a work guidance method by a work guidance device (200) is stored, and a processor that executes the program. Here, the processor can perform various functions in response to the execution of the program stored in memory.

[0036] Next, the work guide device (200) can be implemented in various embodiments, but in a preferred embodiment, as shown in Figure 4, it may be configured in the form of lensless glasses or goggles, including a leg portion (230) which includes a support portion (210) that rests on the bridge of the worker's nose and supports the main body portion (220), a main body portion (220) having a predetermined thickness and formed in a semicircular shape at a predetermined distance above the worker's eyebrows and connected to the upper side of the support portion (210), and a camera device (231) that extends from the main body portion (220), is fixed to the worker's ears and attached to the outside of the worker's head, and captures the worker's forward field of view.

[0037] According to another embodiment of the present invention, the work guide device (200) may further include a glasses portion formed to be a predetermined distance from the worker's eyeballs and not to cover the image captured by the camera device (231), extending from the lower side of the main body (220), and made of a transparent or translucent material so as not to obstruct the worker's field of view. In this embodiment, the glasses portion may be an electronic display that provides AR content to the worker's forward field of view, in addition to a lens that complements the worker's visual ability.

[0038] Furthermore, the work guide device (200) can be attached to the back of the worker's head in the form of bone conduction earphones, rather than being mounted in front of the worker's field of vision as described above.

[0039] In such embodiments, the work guide device (200) may also include a main body (not shown) having a predetermined thickness, which is formed in a semicircular shape and supported in contact with the back of the worker's head; a support part (not shown) having a predetermined thickness, which is formed extending from the main body to the worker's temples, in contact with the worker's sides, and which rests on the worker's ears to support the main body; and a camera device (not shown) attached to the support part (not shown) in the direction toward the outside of the worker's head to capture the worker's forward field of view.

[0040] Furthermore, in the above-described embodiment, the work guide device (200) may have a bone conduction speaker device built into the main body that outputs audio corresponding to work instruction information and audio of control hand signals in response to control signals received from the edge server (100).

[0041] The following describes in detail a work guidance method using a work guidance device (200) according to one embodiment of the present invention, which is performed by a system composed of the elements described above.

[0042] First, a work guide device (200) attached to the worker's head captures the worker's forward field of view, transmits the captured image to an edge server (100), collects fastening status data through the fastening device, and transmits the collected fastening status data to the edge server (100).

[0043] At this time, the fastening status data is data generated when the fastening device rotates in response to a control signal input by the operator while remaining in contact with the fastening screw, thereby fastening the fastening screw to the workpiece. This data may include at least one of the following: the rotational speed (RPM) of the motor and fastening screw of the fastening device per unit time, and the torque value, which are generated during the fastening operation.

[0044] Referring to Figure 3, the edge server (100) analyzes the images received from the work guide device (200) through a pre-trained work analysis model to determine the type of work the worker is performing and whether or not there are any work errors.

[0045] At this time, the work analysis model of the edge server (100) is an artificial intelligence learning algorithm that recognizes the work object from a specific video based on the CNN (Convolutional Neural Network) technique, determines the work procedure, detects work errors, and recognizes predetermined gestures performed by the worker for control, and may be pre-trained before the implementation of the present invention.

[0046] Alternatively, the work analysis model may be a model that learns from images of workers' actions for each type of work, as well as images of the work object and work tools, and outputs the type of work and whether or not there is a work error from the input images.

[0047] Therefore, first, the edge server (100) classifies the images received from the work guide device (200) into work objects, work tools, the worker's body, and other objects according to the image analysis technique.

[0048] Image analysis techniques that can be utilized in this invention include object detection techniques that can detect specific objects from still images or videos, YOLO (You Only Look Once) techniques that predict objects present in an image and their positions, and two-stage object detection techniques that utilize Faster RCNN.

[0049] In addition to the image analysis methods described above, various other image analysis techniques can also be used as embodiments of the present invention; however, these fall under the category of prior art and will therefore not be discussed in detail in this specification.

[0050] The edge server (100) compares the images of the categorized work objects and work tools with images of the type of work and work progress that are pre-stored in the edge server (100)'s database or memory. If the number of matches exceeds a pre-set value, the edge server (100) determines the current work progress.

[0051] Furthermore, the images of the categorized work objects and fastening devices are compared with images of work errors that have been stored in advance. If the match exceeds a predetermined value, the presence or absence of a work error in the current work is determined.

[0052] For example, assuming a worker is sealing a box with tape, the edge server (100) receives images from the camera device (231) of a work guide device (200) worn by the worker, including images of the box, tape, and parts of the worker's body (most often, both hands).

[0053] The edge server (100) analyzes the received image according to an image analysis technique, distinguishes between boxes, tapes, and the worker's hands within the image, and compares the distinguished images with pre-stored images of the type of work, images, and images showing whether or not there are errors.

[0054] If an operator has applied tape to a box but there are parts that could not be sealed, the edge server (100) of the present invention has pre-stored images indicating whether or not there was an error in the sealing process, and can determine that an error occurred in that operation by referring to these images.

[0055] According to another embodiment of the present invention, when the type of work is fastening a part to a workpiece, the edge server (100) can determine whether or not there is a work error by detecting abnormal fastening between the fastening screw and the workpiece based on the RPM and torque values ​​of the fastening status data collected from the fastening device, and by comparing it with a pre-stored graph of abnormal fastening types.

[0056] For example, assuming the task of fastening screws to the wheel of an automobile is performed, the fastening device performing the task is connected to the edge server (100) by wired or wireless connection and transmits fastening status data, including the RPM and torque values ​​generated when the task is performed, to the edge server (100).

[0057] The edge server (100) converts the received fastening status data into a graph and detects abnormal fastening types by comparing the graph with graphs stored in the DB or memory.

[0058] At this time, if the images of the workpiece and work tools, as described above, match the image of the worker's current work status by a preset number or more, the edge server (100) can determine that the work is currently in progress.

[0059] Therefore, if the work is currently in progress, that is, if the screws are still being fastened to the car wheel, the comparison with various pre-set images is withheld, taking into consideration that the work is not yet complete, and the presence or absence of errors can be detected after the fastening work is completed.

[0060] Based on the type of work, the progress of the work, and the presence or absence of work errors identified through the process described above, the edge server (100) can provide corresponding work instruction information through the work guide device (200).

[0061] Here, the work instruction information may consist of at least one of the following: guidance information for the current task, information regarding the presence or absence of errors, and guidance information for the next task.

[0062] For example, assuming that the task currently being performed by the worker is to divide the object into four equal parts, the edge server (100) can provide guidance information for the current task, such as "Please divide the object into four equal parts," as work instruction information.

[0063] In the same example, information regarding the presence or absence of errors, such as "The workpiece was not properly separated," may be provided as work instruction information.

[0064] Such means of providing work instruction information can be realized in various embodiments, but as a typical embodiment, the work instruction information can be provided as voice through a bone conduction speaker device inside the work guide device (200).

[0065] If it is determined at this time that the current work procedure is incorrect or that a work error has occurred, a work error signal including at least one of sound, vibration, or video may be provided.

[0066] Furthermore, according to another embodiment of the present invention, if the work guide device (200) includes a glass portion made of a transparent or translucent material so as not to obstruct the worker's view, work instruction information corresponding to the type of work and the progress of the work can be provided through an AR image displayed on the glass portion of the work guide device (200) superimposed on the worker's view.

[0067] The system determines the type of work the worker is performing and whether or not there are any work errors. If it is determined that the work has been completed without errors, the system can provide the work guide device (200) with pre-set work instruction information as work instruction information provided through the work guide device (200).

[0068] Therefore, the edge server (100) may store pre-configured work instruction information for each detailed task and detailed process for multiple tasks in a database or memory.

[0069] In addition, according to one embodiment of the present invention, the edge server (100) can recognize a gesture through the part of the worker's hand recognized in the image received from the camera device (231) of the work guide device (200), and perform an action corresponding to that gesture.

[0070] When an edge server (100) extracts a control hand signal from a recognized hand region, it provides work instruction information corresponding to the extracted control hand signal, and the work instruction information induced by the control hand signal may be provided preferentially over pre-configured next work instruction information or work instruction information provided by error.

[0071] Referring to Figure 5, the gesture according to one embodiment of the present invention, i.e., the control hand signal, may be data recorded as an image or video in which a specific action is performed through a part of the worker's body, but in a preferred embodiment, it may utilize a part of the worker's hand.

[0072] As illustrated, when a worker extends all fingers while keeping the remaining four fingers (excluding the thumb) together, opens their hand, and then waves their hand horizontally to the left or right, waving to the left can provide information about the previous work instruction, and waving to the right can provide information about the next work instruction.

[0073] Furthermore, if a worker grips the device with their thumb facing outwards, the currently provided work instruction information can be immediately stopped and skipped.

[0074] In this case, according to another embodiment of the present invention, the control hand signal may be configured such that, if the operator is left-handed, the right hand is set as the hand for extracting the operator's control hand signal, and if the operator is right-handed, the left hand is set as the hand for extracting the operator's control hand signal.

[0075] Therefore, if a worker is right-handed when performing a task, they can use their left hand to make gestures to proceed with the task while simultaneously controlling work instructions. Conversely, this is also possible if the worker is left-handed.

[0076] On the other hand, the edge server (100) can provide the main server (300) with at least one of the following for multiple tasks: video data, the number of work errors, the number of gesture inputs, the work yield, and the work time.

[0077] The main server (300) can automatically generate analysis reports from the data received from multiple edge servers (100) for each worker who performed the task, and provide them to the administrator terminal (400).

[0078] This allows the main server to calculate aggregate data for multiple edge servers (100) and multiple workers who worked on each edge server (100). Such aggregate data may be included in the aforementioned analysis report as indicators such as total production volume, total production yield, and total working hours.

[0079] Referring to Figure 6, the procedure for performing a work guidance method using a work guidance device (200) according to one embodiment of the present invention will be described.

[0080] First, a work guide device (200) attached to the worker's head captures the worker's forward field of view, transmits the captured image to an edge server (100), collects fastening status data through the fastening device, and transmits the collected fastening status data to the edge server (100) (S101).

[0081] The edge server (100) analyzes the images received from the work guide device (200) through a pre-trained work analysis model to determine the type of work the worker is performing and whether or not there are any work errors (S102).

[0082] The edge server (100) provides work instruction information corresponding to the type of work it has identified and whether or not there are any work errors, through the work guide device (200) (S103).

[0083] One embodiment of the present invention may also be embodied in the form of a recording medium containing computer-executable commands, such as program modules executed by a computer. The computer-readable medium may be any available medium accessible by a computer, and includes all volatile and non-volatile media, removable and non-removable media. The computer-readable medium may also include all computer storage media. Computer storage media include all volatile and non-volatile, removable and non-removable media, embodied by any method or technique for storing information such as computer-readable commands, data structures, program modules, or other data.

[0084] Although the methods and systems of the present invention have been described in relation to specific embodiments, some or all of their components or operations can be embodied using a computer system having a general-purpose hardware architecture.

[0085] The foregoing description of the present invention is illustrative, and a person with ordinary skill in the art to which the invention pertains will understand that it can be readily modified into other specific forms without altering the technical idea or essential features of the invention. Therefore, the embodiments described above should be understood to be illustrative and not limiting in all respects. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described in a distributed manner may be implemented in a combined manner.

[0086] The scope of the present invention is indicated by the claims, which are described below rather than the summary of the invention, and all modified or altered forms derived from the meaning and scope of the claims and the concept of equivalents thereto should be interpreted as being included within the scope of the present invention.

Claims

1. In a work guidance method performed by a work guidance device, which is carried out by a system, (a) A work guide device attached to the worker's head captures the worker's forward field of view, transmits the captured image to an edge server, collects fastening status data including at least one of the torque value and rotational speed (RPM) generated when fastening screws to the workpiece via a fastening device, and transmits the collected fastening status data to the edge server. (b) The edge server analyzes the images received from the work guide device through a pre-trained work analysis model to determine the type of work the worker is performing and whether or not there are any work errors, or analyzes the images received from the work guide device through the work analysis model to determine the type of work and to determine whether or not there are any work errors based on the received fastening status data, and (c) The step of the edge server providing work instruction information corresponding to the type of work and whether or not there is a work error, through the work guide device, The aforementioned work instruction information includes at least one of the following: guidance information for the current task, information regarding the presence or absence of work errors, and guidance for the next task. The work analysis model described above learns from images of workers' movements for each type of work, as well as images of the work object and work tools, as training data, and outputs the type of work and whether or not there is a work error from the input images, in a work guidance method using a work guidance device.

2. In a work guidance method using a work guidance device as described in claim 1, The above step (a) is, A work guidance method using a work guidance device, comprising: the fastening device rotating in response to a control signal while in contact with the fastening screw, rotating the fastening screw, fastening the fastening screw to the workpiece, collecting at least one of the following as fastening state data: the rotational speed (RPM) of the motor of the fastening device and the fastening screw per unit time generated during the fastening operation, and the torque value, and providing this data to the edge server.

3. In a work guidance method using a work guidance device as described in claim 1, The aforementioned step (b) is, (b-1) The edge server classifies the images received from the work guide device into work objects, work tools, the worker's body and other objects according to an image analysis technique. (b-2) The edge server compares the images of the categorized work objects and work tools with images of work type and work progress that have been stored in advance, and if the match is greater than or equal to a preset value, it determines the current work progress as the work progress, and compares the images of the categorized work objects and fastening devices with images of work errors that have been stored in advance, and if the match is greater than or equal to a preset value, it determines whether or not there are work errors in the current work, and (b-3) A work guidance method using a work guidance device, which includes the step of determining that the work is currently in progress when the edge server matches the images of the classified work objects and work tools with an image of the worker's current work state by a preset value or more.

4. In the work guidance method using the work guidance device described in claim 3, The aforementioned (b-2) step is, A work guidance method using a work guidance device, which includes detecting whether or not there is a work error by comparing the abnormal fastening of the fastening screw and the workpiece with a graph of abnormal fastening types that has been stored in advance, based on the RPM and torque values ​​of the edge server.

5. In a work guidance method using a work guidance device as described in claim 1, The aforementioned work guide device is A support part that rests on the bridge of the worker's nose and supports the main body, The main body has a predetermined thickness, is formed in a semicircular shape at a predetermined distance above the worker's eyebrows, and is connected to the upper side of the support portion, The leg portion extends from the main body, is fixed to the worker's ear, is attached to the outside of the worker's head, and includes a camera device for capturing the worker's forward field of view. A work guidance method using a work guidance device, wherein a bone conduction speaker device is built into the main body of the device, which outputs audio corresponding to the work instruction information and audio of the receiving hand signal in response to a control signal received from the edge server.

6. In the work guidance method using the work guidance device described in claim 5, The aforementioned work guide device is A work guidance method using a work guidance device, further comprising a glass portion formed to be separated by a predetermined distance from the worker's eyeballs and not to cover the image captured by the camera device, extending from the lower side of the main body, and made of a transparent or translucent material so as not to obstruct the worker's field of view.

7. In a work guidance method using a work guidance device as described in claim 1, The aforementioned work guide device is A main body having a predetermined thickness, contacting the back of the worker's head, and formed in a semicircular shape for support, Having a predetermined thickness, it contacts the worker's temporal region, extends from the main body to the worker's temple, rests on the worker's ear, and supports the main body; The support portion includes a camera device attached to the outside of the worker's head to capture the worker's forward field of view, A work guidance method using a work guidance device, wherein the main body has a bone conduction speaker device built inside that outputs audio corresponding to the work instruction information and audio of control hand signals in response to control signals received from an edge server.

8. In a work guidance method using a work guidance device as described in claim 1, In the aforementioned (c) stage, The aforementioned work guide device A work guidance method using a work guidance device, which provides the type of work received from the edge server, the work progress, and the work instruction information corresponding to work errors, through a bone conduction speaker device inside the work guidance device, in the form of voice corresponding to the work instruction information and voice of control hand signals.

9. In the work guidance method using the work guidance device described in claim 8, In the aforementioned (c) stage, The aforementioned work guide device A work guidance method using a work guidance device, which provides a work error signal including at least one of sound, vibration, or video if it determines that the current work procedure is incorrect or a work error has occurred.

10. In the work guidance method using the work guidance device described in claim 6, In the aforementioned (c) stage, The aforementioned work guide device A work guidance method using a work guidance device, wherein work instruction information corresponding to the type of work and the progress of work received from the edge server is provided through an augmented reality (AR) image displayed on the glasses of the work guidance device, superimposed on the worker's field of view.

11. In a work guidance method using a work guidance device as described in claim 1, In the aforementioned (c) stage, A work guidance method by a work guidance device, comprising: determining the type of work being performed by the worker and whether or not there are any work errors; if it is determined that the work has been completed without errors, providing the work guidance device with pre-set next work instruction information; recognizing the part of the worker's hand in the image received from the camera device of the work guidance device; and if a control hand signal is extracted from the recognized part of the hand, providing the work guidance device with work instruction information corresponding to the extracted control hand signal, prioritizing it over the pre-set next work instruction information.

12. In a work guidance method using a work guidance device as described in claim 1, (d) A work guidance method using a work guidance device, further comprising the steps of: the edge server providing the main server with at least one of the following for multiple tasks: video data, the number of work errors, the number of gesture inputs, the work yield, and the work time; and the main server automatically generating an analysis report for each worker who performed each task and providing it to the administrator terminal.

13. A work guidance system using a work guidance device, A head-mounted bone conduction work guide device that captures the worker's forward field of view and transmits the captured images to a server, A fastening device that collects fastening status data, including at least one of the torque value and rotational speed (RPM) generated when fastening fastening screws to a workpiece, and transmits the collected fastening status data to an edge server. Includes an edge server that analyzes images received from the work guide device through a pre-trained work analysis model to determine the type of work being performed by the worker and whether or not there are work errors, or analyzes images received from the work guide device through the work analysis model to determine the type of work, and also determines whether or not there are work errors based on the received fastening status data, and provides work instruction information corresponding to the determined type of work, work progress, and presence or absence of work errors through the work guide device. The aforementioned work instruction information includes at least one of the following: guidance information for the current task, information regarding the presence or absence of work errors, and guidance information for the next task. The aforementioned work analysis model is a work guide system using a work guide device, which learns from images of workers' movements for each type of work, as well as images of the work object and work tools, as training data, and outputs the type of work and whether or not there is a work error from the input images.

Images