Work assistance method and work assistance system

Abstract

In the present invention, an operator who performs maintenance work of a substrate processing apparatus wears smart glasses and issues a sound-based operation instruction. The smart glasses receive the sound and textualize and display the sound through sound recognition. The operator visually confirms the displayed text, and if the content is correct, the confirmation result is correct through gestures. The smart glasses capture and analyze gestures of the operator, generate a command of an operation instruction only when the confirmation result is correct, and transmit the command to the substrate processing apparatus. Since the smart glasses display the content textualized by voice recognition and the operator confirms the content, malfunction due to erroneous voice recognition can be prevented.

Description

TECHNICAL FIELD

[0001] The present application relates to a work assisting method and a work assisting system when performing a prescribed work such as a maintenance work on an industrial machine such as a substrate processing apparatus that performs a prescribed process on a substrate. Among substrates that become a processing target by a substrate processing apparatus, for example, a semiconductor substrate, a liquid crystal display device substrate, a flat panel display (FPD) substrate, an optical disc substrate, a magnetic disc substrate, or a solar cell substrate is included. BACKGROUND

[0002] From the past, in a manufacturing process of a semiconductor device, a substrate processing apparatus that performs various processes on a substrate such as a semiconductor substrate is used. As the substrate processing apparatus, for example, a substrate cleaning apparatus, a heat treatment apparatus, an inspection apparatus, or the like is used. Typically, a case where a plurality of substrate processing apparatuses are arranged in order in a wide clean room is common. The substrate processing apparatuses are maintained at an appropriate timing. In Patent Literature 1, it is described that a plurality of substrate processing apparatuses are arranged at a relatively high density in a clean room, and maintenance of the substrate processing apparatuses is performed.

[0003] Further, in Patent Literature 1, it is described that when a worker gives an instruction to perform maintenance on a substrate processing apparatus, the substrate processing apparatus performs a maintenance operation (for example, an operation of ejecting a processing liquid from a processing liquid nozzle). In a typical maintenance work in the past, the worker manually performs an operation on each part of the substrate processing apparatus while holding a remote controller, but in recent years, the worker also attempts to perform a maintenance work while wearing a smart glass without holding the remote controller. A form in which the worker operates an operation panel displayed in the smart glass or by voice recognition or eye tracking is studied. In Patent Literature 2, it is described that a voice of a worker is voice recognized and displayed as text information on a portable terminal.

[0004] PRIOR ART DOCUMENTS

[0005] PATENT LITERATURE

[0006] Patent Literature 1: Japanese Patent Application Publication No. 2020-4866

[0007] Patent Literature 2: Japanese Patent Application Publication No. 2021-83079 SUMMARY

[0008] PROBLEMS TO BE SOLVED BY THE INVENTION

[0009] However, in terms of voice recognition, smart glasses sometimes misinterpret the operator's speech. Furthermore, a single board processing unit typically houses multiple processing units, and sometimes multiple operators simultaneously perform maintenance work on that unit. In such situations, there is a risk of misinterpreting the voice of someone other than the operator wearing the smart glasses. If the board processing unit operates based on incorrectly recognized information, it could potentially endanger the operator.

[0010] The present invention was made in view of the above-mentioned problems, and its object is to provide a work assistance method and work assistance system that can prevent malfunctions caused by incorrect sound recognition.

[0011] means for solving problems

[0012] To address the aforementioned issues, a first embodiment of the present invention provides a work assistance method for performing prescribed operations on industrial machinery, comprising: a sound receiving step, wherein a sound receiving unit receives an operation instruction based on a sound emitted by an operator, the operator wearing a portable terminal including the sound receiving unit, a display unit, a camera unit, and a communication unit; a sound recognition step, wherein the sound received by the operator through the sound receiving unit is analyzed and converted into text of the operation instruction; a display step, wherein the display unit displays the text of the operation instruction; a detection step, wherein a confirmation result is detected and confirmation information is obtained, the confirmation result being displayed by the operator through a part of their body; a motion analysis step, wherein the confirmation information is analyzed and, if the confirmation result is correct, instruction information of the operation instruction is generated; and a transmission step, wherein the communication unit transmits the instruction information of the operation instruction to the industrial machinery.

[0013] Furthermore, the second embodiment is a work assistance method as in the first embodiment, wherein in the detection process, the camera unit captures the operator's gestures.

[0014] Furthermore, the third embodiment is a work assistance method as described in the first or second embodiment, wherein the portable terminal further includes a gaze tracking unit that tracks the gaze of the operator wearing the portable terminal. In the detection process, the gaze tracking unit determines the part of the industrial machine that the operator visually confirms. In the motion analysis process, when the object part that becomes the text of the operation instruction matches the part that the operator visually confirms, the confirmation result is determined to be correct.

[0015] Furthermore, the fourth embodiment is a work assistance method as described in any of the first to third embodiments, wherein in the voice recognition process, voice analysis is performed only when the voiceprint of the voice received by the voice receiving unit matches a pre-registered voiceprint.

[0016] Further, a seventh embodiment is a work assistance method according to any one of the first to sixth embodiments, wherein the portable terminal is smart glasses.

[0017] Further, a sixth embodiment is a work assistance method according to any one of the first to fifth embodiments, wherein the industrial machine is a substrate processing device that performs a predetermined process on a substrate.

[0018] Further, a seventh embodiment is a work assistance method according to any one of the first to sixth embodiments, wherein the portable terminal is smart glasses.

[0019] Further, an eighth embodiment is a work assistance system in a work in which a portable terminal is used for an industrial machine, the portable terminal including: a sound receiving section that receives an operation instruction based on a sound emitted from a worker wearing the portable terminal; a sound recognition section that analyzes a sound of the worker received by the sound receiving section and converts it into a text of the operation instruction; a display section that displays the text of the operation instruction; a detection section that detects a confirmation result and acquires confirmation information, the confirmation result being shown by a part of the body of the worker who confirms the displayed text of the operation instruction; a motion analysis section that analyzes the confirmation information and generates instruction information of the operation instruction when the confirmation result is correct; and a communication section that transmits the instruction information of the operation instruction to the industrial machine.

[0020] Further, a ninth embodiment is a work assistance system according to the eighth embodiment, wherein the detection section includes a camera section that photographs a gesture of the worker.

[0021] Further, a tenth embodiment is a work assistance system according to the eighth or ninth embodiment, wherein the detection section includes a gaze tracking section that tracks a gaze of the worker wearing the portable terminal and determines a part of the industrial machine that is visually confirmed by the worker, and the motion analysis section determines that the confirmation result is correct when a target part that is a target of the text of the operation instruction coincides with the part that is visually confirmed by the worker.

[0022] Further, an eleventh embodiment is a work assistance system according to any one of the eighth to tenth embodiments, wherein the sound recognition section performs sound analysis only when a voiceprint of a sound received by the sound receiving section coincides with a voiceprint registered in advance.

[0023] Further, the 12th embodiment is the work assistance system as in any one of the 8th to 11th embodiments, wherein the sound recognition section determines phonemes contained in the sound of the worker received by the sound receiving section using an acoustic model, and transforms the arrangement of the determined phonemes into a word using a language model; in the language model, only expressions associated with a specific work on the industrial machine are registered.

[0024] Further, the 13th embodiment is the work assistance system as in any one of the 8th to 12th embodiments, wherein the industrial machine is a substrate processing device that performs a prescribed process on a substrate.

[0025] Further, the 14th embodiment is the work assistance system as in any one of the 8th to 13th embodiments, wherein the portable terminal is smart glasses.

[0026] Effects of Invention

[0027] According to the work assistance method of the 1st to 7th embodiments, since the operation instruction based on the sound issued from the worker wearing the portable terminal is received, transformed into text, and displayed, and the instruction information of the operation instruction is transmitted to the industrial machine when the confirmation result of the worker who confirmed the text is correct, the operation instruction is not given to the industrial machine when the sound recognition is wrong, and thus, the erroneous operation due to the erroneous sound recognition can be prevented.

[0028] In particular, according to the work assistance method of the 4th embodiment, since the sound analysis is performed only when the voiceprint of the received sound coincides with the voiceprint registered in advance, the operation instruction based on the person different from the registered worker can be more reliably prevented from being performed.

[0029] In particular, according to the work assistance method of the 5th embodiment, since only the expressions associated with a specific work on the industrial machine are registered in the language model, the operation instruction based on the sound issued from the worker can be more accurately and reliably transformed into text.

[0030] According to the work assistance system of the 8th to 14th embodiments, since the operation instruction based on the sound issued from the worker wearing the portable terminal is received, transformed into text, and displayed, and the instruction information of the operation instruction is transmitted to the industrial machine when the confirmation result of the worker who confirmed the text is correct, the operation instruction is not given to the industrial machine when the sound recognition is wrong, and thus, the erroneous operation due to the erroneous sound recognition can be prevented.

[0031] In particular, according to the work assistance system of the 11th embodiment, since the sound analysis is performed only when the voiceprint of the received sound coincides with the voiceprint registered in advance, the operation instruction based on the person different from the registered worker can be more reliably prevented from being performed.

[0032] In particular, the work assistance system according to the 12th embodiment can convert operation instructions based on the operator's voice into text with greater accuracy and reliability because it only registers terms associated with specific operations for industrial machines in the language model. Attached Figure Description

[0033] Figure 1 The diagram illustrates the general configuration of the work assistance system of the present invention.

[0034] Figure 2 A plan view used to illustrate the internal layout of the substrate processing apparatus.

[0035] Figure 3 A plan view showing the general structure of the processing unit.

[0036] Figure 4 A side view showing the general structure of the processing unit.

[0037] Figure 5 A three-dimensional view showing the appearance of the smart glasses.

[0038] Figure 6 This is a block diagram illustrating the functional configuration of the control unit of a smart glasses, server, work assistance terminal, and board processing device.

[0039] Figure 7 A flowchart illustrating the procedure of the work assistance method of the present invention.

[0040] Figure 8 A diagram illustrating the voice recognition process of the voice recognition unit.

[0041] Figure 9 This diagram illustrates an example of text display performed via smart glasses.

[0042] Figure 10 This is an example of a worker's hand gesture captured by a camera. Detailed Implementation

[0043] The following is with reference to the appendix. Figure 1The embodiments of the present invention will be described in detail below. Hereinafter, expressions indicating relative or absolute positional relationships (e.g., "in one direction," "along one direction," "parallel," "orthogonal," "center," "concentric," "coaxial," etc.) are, unless otherwise specified, intended to indicate not only a strict positional relationship but also a state of relative angular or distance variation within tolerance or achieving the same level of functionality. Furthermore, expressions indicating equality (e.g., "same," "equal," "homogeneous," etc.) are, unless otherwise specified, intended to indicate not only a quantitatively strict equality but also a state of difference in tolerance or achieving the same level of functionality. Furthermore, expressions indicating shape (e.g., "circular shape," "quadrilateral shape," "cylindrical shape," etc.) are, unless otherwise specified, intended to indicate not only a geometrically strict shape but also a shape representing a range of effects, such as having concavity / convexity or chamfers. Moreover, expressions such as "set," "provided," "equipped," "includes," and "have" are not exclusive expressions excluding the presence of other constituent components. In addition, the expression "at least one of A, B and C" includes "A only", "B only", "C only", "any two of A, B and C", and "all of A, B and C".

[0044] <First Embodiment>

[0045] Figure 1 The diagram illustrates the general configuration of the work assistance system of the present invention. The work assistance system of the present invention includes: a plurality of board processing devices 40, smart glasses 10, a server 70, and a work assistance terminal 80. The controllers of the smart glasses 10 and the board processing devices 40 are wirelessly connected to an information communication network 5 (e.g., the Internet). Furthermore, the work assistance terminal 80 and the server 70 are wiredly connected to the information communication network 5. Information can be transmitted and received between machines connected to the information communication network 5, for example, information can be exchanged between the smart glasses 10 and the work assistance terminal 80. Moreover, the connection between each machine and the information communication network 5 is not limited to the above example; suitable configurations can be adopted (e.g., the work assistance terminal 80 can also be wirelessly connected to the information communication network 5).

[0046] Multiple substrate processing devices 40 are arranged, for example, in a cleanroom. A cleanroom is, for example, a room located within a semiconductor device manufacturing plant, where a certain level of air cleanliness is ensured and temperature and humidity are managed. Operators perform operations on the substrate processing devices 40 within the cleanroom.

[0047] Figure 2This is a plan view illustrating the internal layout of the substrate processing apparatus 40. The substrate processing apparatus 40 is a single-wafer substrate cleaning device that processes silicon substrates, i.e., substrates W, in a circular plate shape such as semiconductor wafers one by one. The substrate processing apparatus 40 includes: a transfer unit 43, a plurality of processing units 50, a main transfer robot 48, and a control unit 45.

[0048] The transfer unit 43 has a plurality of (3 in this embodiment) loading ends LP and a transfer robot 41. At each loading end LP, a carrier C is placed to house a plurality of substrates W to be processed in the processing unit 50. As a form of carrier C, in addition to FOUP (front opening unified pod) which houses the substrates W in a closed space, it can also be SMIF (Standard Mechanical Interface) box or OC (open cassette) which exposes the housed substrates W to the outside atmosphere.

[0049] The transfer robot 41 transports the substrate W between the carrier C and the main transfer robot 48. The transfer robot 41 is, for example, a multi-joint robot, capable of transferring the substrate W to any carrier C placed on one of a plurality of loading ends LP.

[0050] Furthermore, the main transfer robot 48 transports the substrate W between the transfer robot 41 and the processing unit 50. The main transfer robot 48 is configured to perform lifting, rotating, and forward / backward movements of its transfer arm. The main transfer robot 48 receives the unprocessed substrate W taken from the carrier C by the transfer robot 41 and moves it into the processing unit 50. Additionally, the transfer robot 41 receives the processed substrate W moved from the processing unit 50 by the main transfer robot 48 and stores it in the carrier C.

[0051] In the substrate processing apparatus 40, for example, three processing units 50 are laminated to form one laminate (tower). Furthermore, for example, four laminates are arranged around the main transport robot 48. That is, one substrate processing apparatus 40 includes, for example, 12 (=3×4) processing units 50. Figure 2 The diagram shows one segment of the three overlapping processing units 50. Furthermore, the number of processing units 50 in the substrate processing apparatus 40 is not limited to 12 and can be varied appropriately.

[0052] The main transfer robot 48 is positioned at the center of the four stacked layers of the processing unit 50. The main transfer robot 48 moves the substrate W, which is to be processed and received from the transfer robot 41, into the inside of the cup 55 of any processing unit 50. In addition, the main transfer robot 48 removes the processed substrate W from each processing unit 50 and delivers it to the transfer robot 41.

[0053] Furthermore, the substrate processing apparatus 40 includes a control unit 45. The control unit 45 is a general-purpose computer that controls the operations of the aforementioned transfer robot 41, main transport robot 48, and mechanisms installed in each processing unit 50 within the apparatus. The control unit 45 has an input / output interface, i.e., a touch panel, installed on the wall of the apparatus, and a communication unit for communicating with the outside of the apparatus. Furthermore, in Figure 2 In this illustration, the control unit 45 is shown in the transmission unit 43, but it is not limited to this. The control unit 45 is provided in an appropriate position in the substrate processing apparatus 40.

[0054] Hereinafter, one of the 12 processing units 50 mounted on the substrate processing apparatus 40 will be described, but the other processing units 50 have the same configuration except for the different arrangement of the nozzles.

[0055] The processing unit 50 sprays a processing solution onto one substrate W and performs a cleaning process. The term "processing solution" refers to a concept encompassing various chemical solutions and pure water. These chemical solutions include, for example, liquids used for etching or for removing particulates; specifically, solutions such as SC-1 (a mixture of ammonium hydroxide, hydrogen peroxide, and pure water), SC-2 (a mixture of hydrochloric acid, hydrogen peroxide, and pure water), or hydrofluoric acid are used.

[0056] Figure 3 A plan view showing the general configuration of the processing unit 50. Furthermore, Figure 4 This is a side view showing the general configuration of the processing unit 50. The processing unit 50 includes a processing chamber 51, a rotating holding part 56, a processing liquid nozzle (first nozzle) 60, a spray nozzle (second nozzle) 65, and a cup 55. The processing chamber 51 is a hollow shell. The rotating holding part 56, the processing liquid nozzle 60, the spray nozzle 65, and the cup 55 are provided inside the processing chamber 51.

[0057] A transfer inlet 52 is provided on the side wall of the processing chamber 51. The transfer inlet 52 is opened and closed by a barrier 53. With the barrier 53 open, the main transport robot 48 transfers substrate W into and out of the processing chamber 51 through the transfer inlet 52. During the processing of substrate W, the barrier 53 closes the transfer inlet 52. When the barrier 53 closes the transfer inlet 52, the processing chamber 51 becomes a semi-enclosed space.

[0058] An FFU (Fan Filter Unit) 54 is installed at the top of the processing chamber 51. The FFU 54 supplies clean air into the processing chamber 51 from the top. Therefore, a downward airflow of clean air is formed in the processing chamber 51. The gas supplied to the processing chamber 51 is discharged through an exhaust pipe 59 located at the bottom of the processing chamber 51.

[0059] The rotary holding section 56 includes a rotary chuck 57 and a rotary motor 58. The rotary chuck 57 is a substrate holding section that holds the substrate W in a horizontal position (the normal of the main surface of the substrate W is along the vertical direction). The rotary chuck 57 is, for example, a vacuum suction chuck. The rotary chuck 57 holds the central portion of the lower surface of the substrate W. Furthermore, the rotary chuck 57 can also be other types of chucks, such as a clamping mechanical chuck that holds the edge portion of the substrate W.

[0060] The rotary chuck 57 has a circular plate shape with a diameter smaller than that of the substrate W. When the lower surface of the substrate W is held in place by the rotary chuck 57, the peripheral edge of the substrate W protrudes to the outside of the outer periphery of the rotary chuck 57.

[0061] The rotary chuck 57 is connected to the rotary motor 58 via a motor shaft. That is, the upper end of the motor shaft of the rotary motor 58 is connected to the center of the lower surface of the rotary chuck 57. When the rotary motor 58 rotates its motor shaft while the substrate W is held in the rotary chuck 57, the substrate W and the rotary chuck 57 rotate in the horizontal plane about a rotation axis along the vertical direction.

[0062] A cup 55 is arranged around the rotating chuck 57. The cup 55 is movable using a lifting mechanism (not shown). The cup 55 has a roughly cylindrical shape, and its upper part slopes upwards towards the rotating chuck 57. However, the inner diameter of the upper portion of the cup 55 is larger than the diameter of the substrate W. During the processing of the substrate W, the upper end of the cup 55 is higher than the height of the substrate W held in the rotating chuck 57. Therefore, liquid that is scattered by centrifugal force from the substrate W rotating using the rotating holding part 56 during processing is collected and recovered by the cup 55. The liquid recovered by the cup 55 is discharged from a drain pipe (not shown) provided at the bottom of the cup 55. Furthermore, the cup 55 may also be a multi-segment structure with multiple recovery ports for different purposes.

[0063] The treatment fluid nozzle 60 includes a nozzle tip 61, a swing arm 62, and a swing motor 63. The treatment fluid nozzle 60 is, for example, a straight nozzle that sprays treatment fluid in a continuous flow. The nozzle tip 61 is mounted at the front end of the swing arm 62, which extends in a generally horizontal direction. Treatment fluid is supplied from a treatment fluid supply source (not shown) to the nozzle tip 61, forming an outlet from which the treatment fluid is sprayed. The swing arm 62 is oscillated in a horizontal plane about a swing axis A1 along the vertical direction using the swing motor 63. The swing motor 63 is, for example, a pulse motor.

[0064] The swing arm 62 is oscillated by the swing motor 63, and the nozzle tip 61 moves along an arc-shaped trajectory between a processing position held above the substrate W in the rotating holding portion 56 and a standby position outside the cup 55. When the nozzle tip 61 is in the processing position, a cleaning solution is sprayed onto the substrate W held in the rotating holding portion 56 through the processing liquid nozzle 60, thereby performing a cleaning process on the substrate W, for example. In addition, pure water is sprayed onto the substrate W through the processing liquid nozzle 60, thereby performing a pure water rinsing process on the substrate W. Furthermore, an encoder for detecting the position of the nozzle tip 61 can also be attached to the swing motor 63.

[0065] On the other hand, the spray nozzle 65 includes a nozzle tip 66, a swing arm 67, and a swing motor 68. The spray nozzle 65 is, for example, a two-fluid nozzle that mixes a processing liquid with pressurized gas to generate droplets, and then sprays the mixture of droplets and gas onto the substrate W. The nozzle tip 66 is mounted at the front end of the swing arm 67, which extends in a generally horizontal direction. At the nozzle tip 66, processing liquid and pressurized gas are supplied from a processing liquid supply source and a gas supply source (not shown), which mix to form a mixed fluid inside or outside the nozzle tip 66. The swing arm 67 swings in a horizontal plane about a swing axis A2 along the vertical direction using a swing motor 68. The swing motor 68 is, for example, a pulse motor.

[0066] The swing arm 67 is oscillated by the swing motor 68, and the nozzle tip 66 moves along an arc-shaped trajectory between a processing position held above the substrate W in the rotating holding portion 56 and a standby position outside the cup 55. When the nozzle tip 66 is in the processing position, the spray nozzle 65 sprays a mixed fluid onto the substrate W held in the rotating holding portion 56, thereby performing a cleaning process on the substrate W, for example. Furthermore, an encoder for detecting the position of the nozzle tip 66 may be attached to the swing motor 68.

[0067] like Figure 3 As shown, the rotation of the processing liquid nozzle 60 and the rotation of the spray nozzle 65 may interfere with each other. That is, if the processing liquid nozzle 60 is in the processing position and the spray nozzle 65 also moves upward toward the substrate W, there is a risk of them colliding. Therefore, an interlock is provided so that when either the processing liquid nozzle 60 or the spray nozzle 65 is in the processing position, the other cannot operate.

[0068] Operators performing tasks such as operating the substrate processing device 40 wear smart glasses 10. Smart glasses 10 is a type of wearable terminal using a head-mounted display (HMD). Smart glasses 10 is also a device for realizing AR (Augmented Reality) or MR (Mixed Reality). For example, Microsoft's "HoloLens" (registered trademark) can be used as smart glasses 10.

[0069] Figure 5 This is a perspective view showing the appearance of the smart glasses 10. The smart glasses 10 includes display lenses 11 and a headband 12. The user wears the smart glasses 10 by placing the headband 12 on their head. The user can adjust the length of the headband 12 according to their head size. In addition, a power button, a brightness button, and a volume button are provided on the headband 12.

[0070] The display lens 11 includes various sensors and a display. This display is a see-through holographic lens. That is, the display can use holograms to display stereoscopic images in the operator's field of vision, and allows light from real objects to pass through, just like ordinary eyeglass lenses. Therefore, the operator wearing the smart glasses 10 can visually confirm real objects and observe the displayed stereoscopic images through the display.

[0071] The sensors in the display lens 11 mainly include, for example, a plurality of visible light cameras that capture images of the front of the display lens 11, an infrared camera that tracks the operator's gaze, a depth sensor that measures the distance to an object, and an inertial measurement sensor. The infrared camera tracks the gaze by measuring the movement of the wearer's eyes. The depth sensor measures the distance to an object, for example, using a ToF (Time of Flight) method. The inertial measurement sensor consists of an accelerometer, a gyroscope, a magnetometer, etc.

[0072] Furthermore, the smart glasses 10 incorporates a built-in computer including a CPU, memory, and storage unit. A wireless communication mechanism is also provided in the smart glasses 10, through which the computer connects to the information communication network 5. Additionally, the smart glasses 10 also includes a microphone, speaker, and battery.

[0073] Figure 6This is a block diagram illustrating the functional configuration of the smart glasses 10, server 70, work assistance terminal 80, and control unit 45 of the board processing device 40. The smart glasses 10 includes a camera unit 21, a communication unit 22, a display unit 23, a sound receiving unit 24, and a gaze tracking unit 25. The camera unit 21 includes a visible light camera mounted on the display lens 11. For example, the camera unit 21 includes four visible light cameras positioned in front and diagonally in front, capable of capturing the field of vision of the worker wearing the smart glasses 10.

[0074] The communication unit 22 includes the wireless communication mechanism of the aforementioned smart glasses 10. The communication unit 22 transmits and receives data with the work assistance terminal 80 and the server 70 via the information communication network 5. Furthermore, if the communication unit 22 is in a short-range position, it can directly transmit and receive data with the control unit 45 of the board processing device 40. That is, the communication unit 22 can directly or via the information communication network 5 send data or commands to the control unit 45 of the board processing device 40.

[0075] Display unit 23 includes a display with the aforementioned display lens 11. Display unit 23 has a holographic processing device that uses holographic technology to display a stereoscopic image at a predetermined spatial position. Furthermore, the stereoscopic image displayed by display unit 23 is not limited to a mapping of a three-dimensional shape, but may also be a two-dimensional mapping such as a document.

[0076] The sound receiving unit 24 includes the microphone of the smart glasses 10 described above. The sound receiving unit 24 converts the sound arriving at the smart glasses 10 into electrical signals. Therefore, the sound receiving unit 24 can collect the sound emitted by the operator wearing the smart glasses 10 and convert it into electrical signals. The electrical signals output from the sound receiving unit 24 can be stored in the storage unit of the smart glasses 10 (i.e., the sound can be recorded).

[0077] The eye-tracking unit 25 includes two infrared cameras that measure the movement of the operator's eyes. The eye-tracking unit 25 has an eye-tracking function that uses the two infrared cameras to track the operator's gaze. The operator wearing smart glasses 10 can use the eye-tracking function to perform gaze-based operations.

[0078] Furthermore, the smart glasses 10 includes a voice recognition unit 31 and a motion analysis unit 36. The voice recognition unit 31 and the motion analysis unit 36 ​​are functional processing units implemented by the CPU of the smart glasses 10 executing a predetermined processing program. The processing content of the voice recognition unit 31 and the motion analysis unit 36 ​​will be described further later.

[0079] The control unit 45 of the substrate processing apparatus 40 controls the operation of mechanisms such as the swing motor 63 provided in the processing unit 50. The control unit 45 of the substrate processing apparatus 40 can communicate with the communication unit 22 of the smart glasses 10 and can control the operation of various mechanisms provided in the processing unit 50 according to the operation instructions sent from the smart glasses 10.

[0080] The work assistance terminal 80 and server 70 are, for example, installed in the factory of the supplier that manufactures the substrate processing apparatus 40 and undertakes its maintenance and inspection. The work assistance terminal 80 and server 70 can communicate with the smart glasses 10 via the information communication network 5. In addition, the work assistance terminal 80 and server 70 can communicate with each other via the information communication network 5.

[0081] The work assistance terminal 80 and server 70 are general computer systems. That is, the work assistance terminal 80 and server 70 include: a CPU that performs various arithmetic operations, a read-only memory (ROM) that stores basic programs, a read-write memory (RAM) that stores various information, a storage unit (e.g., a disk or SSD) that stores control software or data in advance, and a communication unit that communicates with the information communication network 5.

[0082] The work assistance terminal 80 is, for example, a computer used by supplier-side work assistants to assist workers in cleanroom operations. Work assistants can send various messages from the work assistance terminal 80 to the smart glasses 10 worn by the workers in the cleanroom.

[0083] In the job assistance system of the present invention, server 70 is a computer that performs prescribed processing according to requests from smart glasses 10 and job assistance terminal 80. Server 70 has a storage unit 74 with a relatively large capacity. Large amounts of data generated by smart glasses 10 and job assistance terminal 80 can be stored in storage unit 74. Furthermore, server 70 and job assistance terminal 80 are not essential elements.

[0084] Next, a work assistance method using a work assistance system with the above-described configuration will be described. Figure 7 A flowchart illustrating the procedure of the work assistance method of the present invention is provided. In this embodiment, an operator wearing smart glasses 10 performs maintenance work on the substrate processing apparatus 40. Specific maintenance tasks for the substrate processing apparatus 40 include, for example, teaching the operator to adjust the processing position of the processing fluid nozzle 60, replacing the rotary chuck 57, and teaching the operator to adjust the arm of the main transport robot 48. Instead of operating the apparatus with a handheld remote control as in the past, the operator uses the smart glasses 10 to operate various parts of the substrate processing apparatus 40. In particular, in this embodiment, the operator operates the apparatus using voice commands.

[0085] First, the operator wearing smart glasses 10 issues voice-based operation instructions (step S1). For example, when teaching the processing fluid nozzle 60, the operator issues the voice command "inching plus 8 pixels". In addition, for example, when adjusting the rotary chuck 57 after replacement, the operator issues a voice command such as "chuck open".

[0086] The sound emitted by the operator is received by the sound receiver 24 of the smart glasses 10 (step S2). The received sound is converted into an electrical signal by the sound receiver 24.

[0087] Next, the voice recognition unit 31 converts the voice received by the voice receiving unit 24 into text using voice recognition (step S3). The voice recognition unit 31 is responsible for parsing the operator's voice received by the voice receiving unit 24 and converting it into text for operation instructions.

[0088] Figure 8 The diagram illustrates the sound recognition process of the sound recognition unit 31. The sound emitted by the operator is received by the sound receiving unit 24 and converted into an electrical signal. Then, feature quantities are extracted from the sound signal and converted into sound data (acoustic analysis) that is easily recognized by a computer. Since humans can understand even sounds mixed with some noise, but computers sometimes have difficulty directly recognizing sounds, the sound data is processed in a way that is easily recognized by a computer.

[0089] Next, the voice recognition unit 31 determines phonemes from the voice data. A phoneme is the smallest unit of phonology. The voice recognition unit 31 uses an acoustic model to determine the phonemes contained in the operator's voice received by the voice receiving unit 24. The acoustic model is a module that compares the features of the sound signal obtained using the above-described acoustic analysis with pre-learned data to obtain the phonemes closest to the input sound. For example, such as... Figure 8 As shown, when the operator makes the sound "open", the voice recognition unit 31 uses an acoustic model to determine the phonemes (phoneme strings) contained in the operator's voice as "opun".

[0090] Since the phoneme string determined using the acoustic model is merely a list of phonemes, the sound recognition unit 31 uses a language model to transform the determined phoneme arrangement into meaningful words. The language model is a module that extracts and outputs the most frequently occurring words from the determined phonemes based on pre-stored word information. For example, when determining... Figure 8 When the phoneme arrangement is shown, the voice recognition unit 31 uses a language model to determine that the sound produced by the operator is the word "OPEN". Furthermore, when transforming the phoneme arrangement into a word, a voice dictionary with pre-stored word information can be used in addition to the language model.

[0091] The voice recognition process described above converts the operator's voice into text. Return Figure 7 The display unit 23 of the smart glasses 10 displays the text of the obtained operation instructions (step S4). Figure 9 This diagram illustrates an example of text display performed via smart glasses 10. The display unit 23 of the smart glasses 10 displays the text of operation instructions obtained through voice recognition as a stereoscopic image. Figure 9 In the example, the "inchingplus 8 pixels" obtained by parsing the operator's voice is displayed as a stereoscopic image.

[0092] The operator wearing the smart glasses 10 visually confirms the displayed text of the operation instructions. Furthermore, in the first embodiment, the operator uses gestures (or fingertip movements) to indicate whether the confirmation result is correct or not, that is, whether the displayed text matches the speech. The gestures shown by the operator are captured and detected by the camera unit 21 of the smart glasses 10 (step S5). That is, in the first embodiment, the camera unit 21 functions as a detection unit that detects the confirmation result shown by the operator and obtains confirmation information (gestures). Figure 10 This diagram illustrates an example of a worker's gesture captured by camera unit 21.

[0093] Next, the motion analysis unit 36 ​​of the smart glasses 10 analyzes the confirmation information obtained to determine whether the confirmation result is correct or not. In the first embodiment, the motion analysis unit 36 ​​analyzes the operator's gestures captured by the camera unit 21 to determine whether the displayed text is correct or not. Specifically, for example, a model that has been pre-built and learned from data of a large number of gesture images captured when the confirmation result is correct and incorrect can be used to determine whether it is correct or not.

[0094] For example, such as Figure 10 The gesture of making a circle with the thumb and forefinger is a gesture indicating that the result is correct. The motion analysis unit 36 ​​analyzes... Figure 10 The gesture shown is considered a confirmation result, indicating that the operator agrees to display the text. In this case, the process proceeds from step S6 to step S7, where the smart glasses 10 generates an operation instruction command based on the text of the operation instruction obtained from voice recognition. Then, the communication unit 22 of the smart glasses 10 sends the operation instruction command to the control unit 45 of the substrate processing device 40. In the example described above, the operation instruction command containing "inching plus 8 pixels" is sent from the smart glasses 10 to the control unit 45 of the substrate processing device 40.

[0095] The control unit 45 executes the command received from the smart glasses 10. For example, upon receiving the command described above, the control unit 45 causes the swing motor 63 to move by 8 pixels, thereby inching the processing liquid nozzle 60.

[0096] On the other hand, for example, a gesture that crosses the index fingers of both hands to form an "X" symbol is considered an incorrect confirmation result. When the motion analysis unit 36 ​​analyzes such a gesture, it determines that the confirmation result is incorrect, meaning the operator does not agree to the displayed text. In this case, the process proceeds from step S6 to step S8, and the smart glasses 10 cancels the operation. Specifically, the smart glasses 10 temporarily deletes the text of the operation instructions obtained through voice recognition and does not execute the command sent from the smart glasses 10 to the board processing device 40. Therefore, the board processing device 40 remains inactive.

[0097] In the first embodiment, the smart glasses 10 receives operation instructions based on the voice of the operator wearing the smart glasses 10, transcribes them into text through voice recognition, and displays them. The operator visually confirms the displayed text, and if the content is correct, confirms the result by gesture. The smart glasses 10 captures and analyzes the operator's gestures, and only sends the operation instruction command to the board processing device 40 when the confirmation result is correct.

[0098] Because the smart glasses 10 displays the textualized content of the received sound through voice recognition, and the operator confirms this content, it prevents the board processing device 40 from receiving incorrect voice recognition operation instructions. Therefore, even if the smart glasses 10 misinterprets the operation instructions based on the operator's voice, it can still prevent the board processing device 40 from malfunctioning according to the incorrect interpretation.

[0099] Furthermore, sometimes multiple operators simultaneously perform maintenance work on multiple processing units 50 mounted on the board processing device 40. In such cases, there is a risk that the smart glasses 10 may receive and transcribe the voice of someone different from the operator wearing the smart glasses 10. In the first embodiment, since the smart glasses 10 displays transcribed content and the operator confirms the content, even if the smart glasses 10 receives and transcribes the voice of another person, the operator can detect the error and stop executing the operation instructions based on that other person. Therefore, malfunctions of the board processing device 40 caused by the voice of another person can be prevented. That is, according to the first embodiment, malfunctions of the board processing device 40 caused by incorrect voice recognition can be prevented, thereby ensuring the safety of the operator.

[0100] <Second Implementation>

[0101] Next, a second embodiment of the present invention will be described. The overall configuration of the job assistance system and the configuration of the substrate processing apparatus 40 in the second embodiment are the same as those in the first embodiment. Furthermore, the procedure of the job assistance method in the second embodiment is also largely the same as that in the first embodiment. In the first embodiment, the operator uses gestures to express the confirmation result of the textualized content, but in the second embodiment, the correctness of the confirmation result is determined by eye tracking.

[0102] Similar to the first embodiment, the smart glasses 10 receives operation instructions based on the voice of the operator wearing the smart glasses 10 and displays them as text through voice recognition. For example, when the operator says "chuck open," this content is displayed as "chuck open" through the text of the smart glasses 10. In the second embodiment, the operator visually confirms the text of the displayed operation instructions, and if the content is correct, shifts their gaze to the part that is to be operated. For example, if the content of the text displayed as "chuck open" is correct, the operator observes the part that is to be operated, i.e., rotating the chuck 57. The gaze tracking unit 25 of the smart glasses 10 tracks the operator's gaze and identifies where the gaze is directed, determining the part of the board processing device 40 that the operator visually confirms. Furthermore, the smart glasses 10 can determine the part by, for example, using the VPS (Visual Positioning System) function of the smart glasses 10 to pre-register the position information of each part of the board processing device 40, and then using this position information to perform the operation. In the second embodiment, the eye tracking unit 25 functions as a detection unit that detects and displays the confirmation results by the operator and obtains confirmation information (direction of eye movement).

[0103] Next, in the second embodiment, the motion analysis unit 36 ​​of the smart glasses 10 determines whether the confirmation result is correct based on the direction of the operator's gaze. Specifically, the motion analysis unit 36 ​​analyzes whether the object part of the displayed text that serves as the operation instruction matches the part visually confirmed by the operator, and determines that the confirmation result is correct (text correct) when they match. For example, if the object part of the operation instruction text "chuck open" is rotating chuck 57, and the part visually confirmed by the operator is also rotating chuck 57, then they match, and the motion analysis unit 36 ​​determines that the confirmation result is correct. In this case, the communication unit 22 of the smart glasses 10 sends an operation instruction command to the control unit 45 of the board processing device 40, and the control unit 45 executes the command.

[0104] On the other hand, if the object of the displayed operation instruction text does not match the area visually confirmed by the operator, the motion analysis unit 36 ​​determines that the confirmation result is incorrect (text error). In this case, the smart glasses 10 cancels the operation, and the part of the board processing device 40 does not move.

[0105] In the second embodiment, the smart glasses 10 receives operation instructions based on the voice of the operator wearing the smart glasses 10, transcribes them into text using voice recognition, and displays them. The operator visually confirms the displayed text; if the content is correct, the operator confirms the result by observing the direction of their gaze. The smart glasses 10 tracks and analyzes the direction of the operator's gaze, and only sends the operation instruction command to the board processing device 40 when the confirmation result is correct.

[0106] Similar to the first embodiment, since the smart glasses 10 displays the textualized content of the received sound through voice recognition, and the operator confirms the content, it prevents the erroneous voice recognition operation instructions from being given to the board processing device 40. Therefore, according to the second embodiment, malfunctions of the board processing device 40 due to erroneous voice recognition can also be prevented.

[0107] <Third Implementation>

[0108] Next, a third embodiment of the present invention will be described. The overall configuration of the work assistance system and the configuration of the substrate processing apparatus 40 in the third embodiment are the same as those in the first embodiment. Furthermore, the procedure of the work assistance method in the third embodiment is also largely the same as that in the first embodiment.

[0109] In the third embodiment, when the smart glasses 10 receives the voice emitted by the operator and transcribes it through voice recognition, the voice recognition procedure is executed only if the voiceprint of the received voice matches the voiceprint of a pre-registered operator. The voiceprint is a spectrogram representing the result of frequency analysis of the sound. Specifically, for example, the acoustic model used in the voice recognition procedure to determine phonemes (…). Figure 8 The voice recognition unit 31 pre-registers the voiceprint of the operator. It compares the voiceprint obtained by frequency analysis of the received sound with the voiceprint pre-registered in the acoustic model. Then, only when the difference between the two is below a certain threshold, the voice recognition unit 31 determines that the operator emitting the voice is the same person as the pre-registered operator and transcribes the received sound into text through voice recognition. Except for the point of voiceprint-based confirmation, everything else is the same as in the first embodiment.

[0110] In the third embodiment, sound analysis is performed only when the voiceprint of the sound received by the sound receiving unit matches a pre-registered voiceprint. Therefore, the possibility of the smart glasses 10 receiving and translating a voice from someone different from the operator is reduced, and the execution of operation instructions based on others can be more reliably prevented. As a result, according to the third embodiment, malfunctions of the board processing device 40 due to incorrect voice recognition can be more reliably prevented.

[0111] <Fourth Implementation>

[0112] Next, the fourth embodiment of the present invention will be described. The overall configuration of the work assistance system and the configuration of the substrate processing apparatus 40 in the fourth embodiment are the same as those in the first embodiment. Furthermore, the procedure of the work assistance method in the fourth embodiment is also largely the same as that in the first embodiment.

[0113] In the fourth embodiment, the language model used when acquiring words in the voice recognition program ( Figure 8 In this language model, only terms associated with maintenance work on the substrate processing apparatus 40 are pre-registered. For example, only terms such as "inching" associated with teaching the processing fluid nozzle 60 and "chuck" associated with replacing the rotary chuck 57 are pre-registered in the language model. Using this language model, the voice recognition unit 31 identifies only terms associated with maintenance work on the substrate processing apparatus 40 as words. As a result, the voices emitted by the operator during maintenance of the substrate processing apparatus 40 can be accurately transcribed with higher precision. Except for the point that only terms associated with specific tasks are pre-registered in the language model, everything else is the same as in the first embodiment.

[0114] Typically, a large number of words are learned using a general language model for voice recognition so that the voice can be transformed into sentences that best suit various situations. In the fourth embodiment, only terms associated with specific operations for the board processing apparatus 40 are registered in the language model. That is, instead of pre-registering a large number of words in the language model, the fourth embodiment specializes and simplifies the words registered in the language model. Therefore, in limited situations such as maintenance operations of the board processing apparatus 40, operation instructions based on the operator's voice can be transcribed more accurately and reliably. As a result, malfunctions of the board processing apparatus 40 due to incorrect voice recognition can be prevented more reliably. Furthermore, because the language model is miniaturized, the processing efficiency of voice recognition can be improved, and transcription can be completed in a shorter time.

[0115] <Example of Variation>

[0116] The embodiments of the present invention have been described above, but various modifications can be made to the present invention in addition to the above description without departing from its spirit. For example, the first embodiment analyzes the operator's gestures, and the second embodiment uses eye tracking, but both can also be used to determine the correctness of the confirmation result of the textualized content. That is, in addition to analyzing the operator's gestures, the direction of the operator's gaze is also tracked to determine the correctness of the confirmation result.

[0117] Furthermore, for the confirmation results of the displayed text, the operator can use gestures or methods other than eye contact to indicate whether it is correct or not. For example, the operator can use the movement of their head while wearing the smart glasses 10 to indicate the confirmation result. That is, it is sufficient for the operator to use a part of their body to represent the form of the confirmation result.

[0118] Furthermore, in the fourth embodiment, only terms associated with specific tasks are registered in the language model, but a language model specifically for each task can also be prepared. For example, a first language model can be prepared in advance, registering only terms associated with teaching the processing fluid nozzle 60; a second language model can be prepared, registering only terms associated with replacing the rotary chuck 57; and a third language model can be prepared, registering only terms associated with teaching the arm of the main transport robot 48. Then, the operator selects the optimal language model based on the maintenance task to be performed. For example, an operator who wants to replace the rotary chuck 57 selects the second language model described above. The selection of the language model can be done, for example, simply by using the smart glasses 10. Therefore, since the language model becomes a language model specifically for the task to be performed by the operator, the operation instructions based on the operator's voice can be more accurately and precisely textualized.

[0119] Furthermore, in the above embodiments, the operator uses smart glasses 10, but is not limited to this; a portable terminal such as a tablet or smartphone may also be used instead of smart glasses 10. That is, any portable terminal equipped with a camera and communication unit is acceptable. However, using a tablet or similar device would occupy the operator's hand, so a wearable terminal such as smart glasses 10 is preferred.

[0120] Furthermore, the substrate processing apparatus 40 is not limited to a substrate cleaning apparatus, but can be an apparatus that performs a prescribed treatment on the substrate, such as a heat treatment apparatus, an exposure apparatus, a coating and developing apparatus, a measuring apparatus, or an inspection apparatus. When the substrate processing apparatus 40 is a substrate cleaning apparatus, it can be a single-piece cleaning apparatus that cleans the substrate one by one, or a batch cleaning apparatus that cleans multiple substrates in batches.

[0121] Furthermore, the target of the work assistance technology of the present invention is not limited to substrate processing apparatus, but can be any industrial machine that has an actuating part that performs certain actions. Examples of such industrial machines include printing processing apparatus, film forming apparatus, medical apparatus, and appearance inspection apparatus.

[0122] Explanation of reference numerals in the attached figures

[0123] 5: Information and Communication Network

[0124] 10: Smart Glasses

[0125] 21: Camera Department

[0126] 22: Ministry of Communications

[0127] 23: Display Section

[0128] 24: Sound receiving unit

[0129] 25: Eye Tracking Department

[0130] 31: Voice Recognition Department

[0131] 36: Motion Analysis Department

[0132] 40: Substrate processing apparatus

[0133] 45: Control Department

[0134] 48: Main transport robot

[0135] 50: Processing Unit

[0136] 56: Rotational retaining part

[0137] 57: Rotary chuck

[0138] 60: Processing fluid nozzle

[0139] 63: Oscillating Motor

[0140] 65: Spray nozzle

[0141] 70: Server

[0142] 80: Operation Assist Terminal

[0143] W: substrate

Claims

1. A work assisting method for a work assisting method when a prescribed work is performed on an industrial machine, wherein, comprising: a sound receiving step of receiving an operation instruction based on a sound emitted from an operator who wears a portable terminal including a sound receiving section, a display section, a camera section, and a communication section; a sound recognition step of analyzing the sound of the operator received by the sound receiving section and converting into a text of the operation instruction; a display step of the display section displaying the text of the operation instruction; a detection step of detecting a confirmation result and acquiring confirmation information, the confirmation result being shown by the operator who confirms the displayed text of the operation instruction by a part of the body; a motion analysis step of analyzing the confirmation information and generating instruction information of the operation instruction when the confirmation result is correct; and a transmission step of the communication section transmitting the instruction information of the operation instruction to the industrial machine.

2. The work assistance method according to claim 1, wherein in the detection step, the camera section photographs a gesture of the operator.

3. The work assistance method according to claim 1, wherein the portable terminal further includes a line-of-sight tracking section that tracks a line of sight of the operator who wears the portable terminal, in the detection step, a part of the industrial machine visually confirmed by the operator is determined by the line-of-sight tracking section, in the motion analysis step, when a target part that becomes a target of the text of the operation instruction coincides with the part visually confirmed by the operator, it is determined that the confirmation result is correct.

4. The work assistance method according to claim 1, wherein in the sound recognition step, sound analysis is performed only when a voiceprint of the sound received by the sound receiving section coincides with a voiceprint registered in advance.

5. The work assistance method according to claim 1, wherein the sound recognition step includes: a phoneme determination step of determining a phoneme included in the sound of the operator received by the sound receiving section using an acoustic model; and a word conversion step of converting an arrangement of the phoneme determined in the phoneme determination step into a word using a language model, in the language model, only a term associated with a specific work on the industrial machine is registered.

6. The work assistance method according to claim 1, wherein the industrial machine is a substrate processing device that performs a predetermined process on a substrate.

7. The work assistance method according to any one of claims 1 to 6, wherein the portable terminal is smart glasses.

8. A work assistance system that is a work assistance system when a work on an industrial machine using a portable terminal is performed, wherein the portable terminal includes: a sound receiving section that receives an operation instruction based on a sound emitted from an operator who wears the portable terminal; a sound recognition section that analyzes the sound of the operator received by the sound receiving section and converts into a text of the operation instruction; a display section that displays the text of the operation instruction; a detection section that detects a confirmation result and acquires confirmation information, the confirmation result being shown by the operator who confirms the displayed text of the operation instruction by a part of the body; ​ The action analysis section analyzes the confirmation information and generates instruction information of the operation instruction when the confirmation result is correct. And The communication section transmits the instruction information of the operation instruction to the industrial machine.

9. The work support system according to claim 8, wherein The detection section includes an imaging section that images the gesture of the worker.

10. The work support system according to claim 8, wherein The detection section includes a line-of-sight tracking section that tracks the line of sight of the worker wearing the portable terminal and determines the part of the industrial machine that the worker visually confirmed, The action analysis section determines that the confirmation result is correct when the object part that is the object of the text of the operation instruction coincides with the part that the worker visually confirmed.

11. The work support system according to claim 8, wherein The sound recognition section performs sound analysis only when the voiceprint of the sound received by the sound receiving section coincides with the voiceprint registered in advance.

12. The work support system according to claim 8, wherein The sound recognition section determines the phonemes contained in the sound of the worker received by the sound receiving section using an acoustic model and transforms the arrangement of the determined phonemes into words using a language model, In the language model, only expressions associated with a specific work on the industrial machine are registered.

13. The work support system according to claim 8, wherein The industrial machine is a substrate processing device that performs a predetermined process on a substrate.

14. The work support system according to any one of claims 8 to 13, wherein The portable terminal is smart glasses.

Images