Work support method and work support system

Abstract

To provide a work support method and a work support system that enable a worker to recognize a change in a surrounding situation even in a dark environment.SOLUTION: A space mesh acquired by scanning a space including a target place in a normal state by a space recognition unit of smart glasses is registered in advance as a reference space mesh. The two-dimension image of the flat surface in the normal state is registered in advance as a reference image. When an abnormality is detected, a worker wearing smart glasses enters the work area 101 and acquires a target spatial mesh by scanning. When the target spatial mesh satisfies the detection condition, the target spatial mesh is compared with the reference spatial mesh. On the other hand, when the target spatial mesh does not satisfy the detection condition, the target spatial mesh is converted into a two-dimension image to acquire a target image, and the target image is compared with the reference image.SELECTED DRAWING: Figure 15

Description

[Technical Field]

[0001] The present invention relates to a method and system for supporting operations when performing predetermined operations such as maintenance on industrial equipment such as substrate processing apparatuses that perform predetermined processes on substrates. Substrates to be processed by the substrate processing apparatuses include, for example, semiconductor substrates, substrates for liquid crystal display devices, substrates for flat panel displays (FPDs), substrates for optical disks, substrates for magnetic disks, and substrates for solar cells. [Background technology]

[0002] Conventionally, in the manufacturing process of semiconductor devices, substrate processing apparatuses are used to perform various processes on substrates such as semiconductor substrates. Examples of substrate processing apparatuses include substrate cleaning apparatuses, heat treatment apparatuses, and inspection apparatuses. Typically, many substrate processing apparatuses are arranged in an orderly fashion in a large clean room. Maintenance is performed on these substrate processing apparatuses at appropriate intervals. Patent Document 1 describes arranging many substrate processing apparatuses at a relatively high density in a clean room and performing maintenance on the substrate processing apparatuses. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-4866 Summary of the Invention [Problem to be solved by the invention]

[0004] In a clean room where a substrate processing apparatus is installed, foreign objects such as tools and parts may be placed. Also, the grating on the floor of the clean room may be removed to form openings. Generally, workers working in the clean room visually check for foreign objects and openings to avoid contact with them.

[0005] However, when performing maintenance on a substrate processing apparatus, workers may have to work in a dark environment. For example, an area known as a yellow room, where ultraviolet light is blocked, is a particularly dark environment. Even if foreign objects or openings are present in such a dark environment, it may be difficult for the workers to see them with the naked eye. This may result in the risk of the workers colliding with foreign objects or falling through openings during work.

[0006] The present invention has been made in consideration of the above-mentioned problems, and aims to provide a work assistance method and a work assistance system that enable a worker to recognize changes in the surrounding situation even in a dark environment. [Means for solving the problem]

[0007] In order to solve the above problem, a first aspect of the present invention is a work support method for performing specified work on industrial equipment, comprising: a pre-registration step of registering a reference spatial mesh, which is mesh data obtained by scanning a space including a specified location in a normal state using a portable terminal equipped with a display unit, a communication unit, and a spatial recognition unit, and reference image data, which is two-dimensional image data of a flat surface in a normal state; a spatial recognition step of scanning the space including the specified location and / or the flat surface using the portable terminal at a specified timing to obtain a target spatial mesh; an image conversion step of converting the target spatial mesh into a two-dimensional image to obtain target image data when the target spatial mesh does not satisfy a detection condition; a comparison step of comparing the target spatial mesh with the reference spatial mesh if the target spatial mesh satisfies the detection condition, and comparing the target image data with the reference image data if the target spatial mesh does not satisfy the detection condition; and an alert step of issuing an alert if the difference between the target spatial mesh and the reference spatial mesh is equal to or greater than a first threshold value, or if the difference between the target image data and the reference image data is equal to or greater than a second threshold value.

[0008] In addition, a second aspect is the work supporting method according to the first aspect, further comprising a position registration step of registering position information of the difference portion when a warning is issued in the warning issuing step.

[0009] In addition, in a third aspect, in the work support method according to the first or second aspect, it is determined that the target space mesh does not satisfy the detection condition if the density of vertices in the target space mesh is less than a predetermined threshold value.

[0010] In addition, in a fourth aspect, in the work support method according to any one of the first to third aspects, the reference image data is obtained by converting mesh data obtained by scanning a flat surface in a normal state using the mobile terminal into two-dimensional image data.

[0011] In addition, a fifth aspect is the work support method according to any one of the first to fourth aspects, wherein the industrial equipment is a substrate processing apparatus that performs a predetermined process on a substrate.

[0012] In addition, a sixth aspect is the task assistance method according to any one of the first to fifth aspects, wherein the mobile terminal is a pair of smart glasses.

[0013] Further, a seventh aspect is a work support system for performing specified tasks on industrial equipment using a portable terminal, the system comprising: a portable terminal having a display unit and a communication unit; a memory unit that stores a reference spatial mesh, which is mesh data obtained by scanning a space including a specified location in a normal state using the portable terminal, and reference image data, which is two-dimensional image data of a flat surface in a normal state; a spatial recognition unit that scans the space including the specified location and / or the flat surface using the portable terminal at a specified timing to obtain a target spatial mesh; an image conversion unit that converts the target spatial mesh into a two-dimensional image to obtain target image data when the target spatial mesh does not satisfy a detection condition; a comparison unit that compares the target spatial mesh with the reference spatial mesh if the target spatial mesh satisfies the detection condition, and compares the target image data with the reference image data if the target spatial mesh does not satisfy the detection condition; and a warning issuance unit that issues a warning if a difference between the target spatial mesh and the reference spatial mesh is equal to or greater than a first threshold value, or if a difference between the target image data and the reference image data is equal to or greater than a second threshold value.

[0014] According to an eighth aspect, in the work assistance system according to the seventh aspect, the system further comprises a position registration unit that registers position information of the difference portion when the warning issuing unit issues a warning.

[0015] In addition, in a ninth aspect, in the work support system according to the seventh or eighth aspect, it is determined that the target space mesh does not satisfy the detection condition if the density of vertices in the target space mesh is less than a predetermined threshold value.

[0016] In addition, in a tenth aspect, in the work support system according to any one of the seventh to ninth aspects, the reference image data is obtained by converting mesh data acquired by scanning a flat surface in a normal state using the mobile terminal into two-dimensional image data.

[0017] In addition, an eleventh aspect is the work support system according to any one of the seventh to tenth aspects, wherein the industrial equipment is a substrate processing apparatus that performs a predetermined process on a substrate.

[0018] In addition, a twelfth aspect is the task assistance system according to any one of the seventh to eleventh aspects, wherein the mobile terminal is a pair of smart glasses. [Effects of the Invention]

[0019] According to the work support methods of the first to sixth aspects, if the difference between the target space mesh obtained by scanning a space including a specified location at a specified timing and the reference space mesh for the specified location in a normal state is equal to or greater than a first threshold value, or if the difference between the target image data obtained by converting the target space mesh obtained at a specified timing into a two-dimensional image and the reference image data is equal to or greater than a second threshold value, an alert is issued when there is a change in the surroundings, allowing the worker to recognize changes in the surrounding situation even in a dark environment.

[0020] According to the work support systems of the seventh to twelfth aspects, if the difference between the target space mesh obtained by scanning a space including a specified location at a specified timing and the reference space mesh for the specified location in a normal state is equal to or greater than a first threshold value, or if the difference between the target image data obtained by converting the target space mesh obtained at a specified timing into a two-dimensional image and the reference image data is equal to or greater than a second threshold value, an alert is issued when there is a change in the surroundings, allowing the worker to recognize changes in the surrounding situation even in a dark environment. [Brief explanation of the drawings]

[0021] [Figure 1] 1 is a diagram showing a schematic configuration of a work support system according to the present invention; [Figure 2] FIG. 2 is a plan view showing an example of a layout of a plurality of substrate processing apparatuses. [Figure 3] FIG. 2 is a side view showing the configuration of the substrate processing apparatus. [Figure 4] FIG. 2 is a plan view of the substrate processing apparatus. [Figure 5] FIG. 2 is a diagram showing a schematic configuration of a processing unit. [Figure 6] FIG. 1 is a perspective view showing the appearance of smart glasses. [Figure 7] FIG. 2 is a block diagram showing functional configurations of a smart glass, a server, a work support terminal, and a control unit of a substrate processing apparatus. [Figure 8] 1 is a flowchart showing a procedure of advance preparation in a work assistance method according to the present invention. [Figure 9] FIG. 10 is a diagram showing an example of a space mesh created by scanning a space containing a foreign object having irregularities. [Figure 10] FIG. 10 is a diagram showing an example of a spatial mesh created by scanning a flat surface. [Figure 11] FIG. 10 is a diagram showing an example of an image obtained by converting a spatial mesh created by scanning the floor surface of a clean room. [Figure 12] 1 is a flowchart showing a procedure of an initial stage of abnormality detection in a work assistance method according to the present invention. [Figure 13] 10 is a flowchart showing a procedure when a target space mesh satisfies a detection condition. [Figure 14] 10 is a flowchart showing the procedure when the target space mesh does not satisfy the detection condition. [Figure 15] FIG. 10 is a diagram illustrating a situation in which a worker wearing smart glasses performs work for detecting anomalies. [Figure 16] FIG. 10 is a diagram showing an example of an image obtained by converting a spatial mesh created by scanning the floor surface of a clean room when an abnormality is detected. [Figure 17] FIG. 10 is a diagram showing an example of a warning message displayed by a warning issuing unit. DETAILED DESCRIPTION OF THE INVENTION

[0022] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Hereinafter, expressions indicating relative or absolute positional relationships (e.g., "in one direction," "along one direction," "parallel," "orthogonal," "center," "concentric," "coaxial," etc.) not only strictly represent the positional relationship but also represent a state of relative angular or distance displacement within a tolerance or a range that provides equivalent functionality, unless otherwise specified. Furthermore, expressions indicating an equal state (e.g., "identical," "equal," "homogeneous," etc.) not only represent a state of strict quantitative equality but also represent a state of difference that provides a tolerance or equivalent functionality, unless otherwise specified. Furthermore, expressions indicating a shape (e.g., "circular," "square," "cylindrical," etc.) not only represent a geometrically strict shape but also represent a shape within a range that provides equivalent functionality, such as irregularities or chamfers, unless otherwise specified. Furthermore, expressions such as "comprise," "comprise," "include," "have," etc., regarding components, are not exclusive expressions that exclude the presence of other components. Furthermore, the expression "at least one of A, B, and C" includes "A only," "B only," "C only," "any two of A, B, and C," and "all of A, B, and C."

[0023] FIG. 1 is a diagram showing a schematic configuration of a work support system according to the present invention. The work support system according to the present invention includes a plurality of substrate processing apparatuses 50, smart glasses 10, a server 70, and a work support terminal 80. The controllers of the smart glasses 10 and the substrate processing apparatuses 50 are connected to an information and communication network 5 (e.g., the Internet) via wireless communication. The work support terminal 80 and the server 70 are connected to the information and communication network 5 via a wired connection. Information can be transmitted and received between devices connected to the information and communication network 5; for example, information can be exchanged between the smart glasses 10 and the work support terminal 80. Note that whether each device is connected to the information and communication network 5 wirelessly or via a wired connection is not limited to the above example, and any appropriate form can be used (for example, the work support terminal 80 may be connected to the information and communication network 5 wirelessly).

[0024] FIG. 2 is a plan view showing an example of the layout of a plurality of substrate processing apparatuses 50. As shown in FIG. 2, a plurality of substrate processing apparatuses 50 are arranged in a clean room 40 in a regular line at regular intervals. The clean room 40 is provided, for example, in a semiconductor device manufacturing factory, and is a room in which a certain level of air cleanliness is ensured and temperature and humidity are controlled. In this embodiment, a plurality of substrate processing apparatuses 50 of the same type and model are arranged in the relatively large clean room 40. Therefore, it is difficult for an operator to distinguish the substrate processing apparatuses 50 arranged in the clean room 40 from their appearance alone.

[0025] Fig. 3 is a side view showing the configuration of one substrate processing apparatus 50. Fig. 4 is a plan view of the substrate processing apparatus 50. The substrate processing apparatus 50 is, for example, a single-wafer substrate cleaning apparatus that cleans substrates one by one. The substrate processing apparatus 50 has an indexer 51, multiple processing units 52, a transfer robot 56, and a main transport robot 57.

[0026] Carriers C each accommodating a plurality of substrates W are placed on the indexer 51. For example, three carriers C can be placed on the indexer 51. The transfer robot 56 is configured to be able to slide along the arrangement direction of the plurality of carriers C, to move up and down, to rotate, and to move its hand forward and backward. Unprocessed substrates W are removed from the carriers C placed on the indexer 51 by the transfer robot 56. Processed substrates W are stored in the carriers C placed on the indexer 51 by the transfer robot 56. The carriers C are, for example, FOUPs (front opening unified pods) that store substrates W in an enclosed space.

[0027] In the substrate processing apparatus 50, for example, three processing units 52 are stacked to form one stack. Then, for example, four stacks are arranged around the main transport robot 57 of the substrate processing apparatus 50. That is, one substrate processing apparatus 50 includes, for example, 12 (=3×4) processing units 52.

[0028] The main transport robot 57, which is disposed at the center of the four stacks, is configured to be capable of lifting and lowering, rotating, and moving its transport arm AM forward and backward. The main transport robot 57 can deliver substrates W to and from all 12 processing units 52. The main transport robot 57 receives unprocessed substrates W from the transfer robot 56 and transports them into any of the 12 processing units 52. The main transport robot 57 also removes processed substrates W from the processing units 52 and delivers them to the transfer robot 56.

[0029] The substrate processing apparatus 50 also includes a control unit 55. The control unit 55 is a general-purpose computer that controls the operations of the transfer robot 56, the main transport robot 57, and each of the processing units 52 provided within the apparatus. The control unit 55 has a touch panel serving as an input / output interface provided on a wall surface of the apparatus, and a communication unit that communicates with the outside of the apparatus. While the control unit 55 is shown in the indexer 51 in FIGS. 3 and 4 for convenience of illustration, the present invention is not limited to this, and the control unit 55 may be provided at an appropriate position within the substrate processing apparatus 50.

[0030] FIG. 5 is a diagram showing a schematic configuration of the processing unit 52. The processing unit 52 includes a processing chamber 60, a spin holder 61, and a discharge nozzle 65. The processing chamber 60 is a hollow housing. The spin holder 61, the discharge nozzle 65, and the like are provided inside the processing chamber 60. The processing chamber 60 also has a loading / unloading port (not shown). The loading / unloading port is opened and closed by a shutter. With the loading / unloading port open, the main transport robot 57 loads and unloads the substrate W into and from the processing chamber 60. The loading / unloading port is closed during processing of the substrate W. The processing chamber 60 also has an air supply mechanism and an exhaust mechanism (not shown).

[0031] The rotation holder 61 includes a spin chuck 62 and a spin motor 63. The spin chuck 62 is a substrate holder that holds the substrate W in a horizontal position (a position in which the normal to the main surface of the substrate W is aligned vertically). The spin chuck 62 is, for example, a vacuum suction-type chuck. The spin chuck 62 has a disk shape with a diameter smaller than the diameter of the substrate W. The spin chuck 62 suction-holds the central portion of the lower surface of the substrate W. When the lower surface of the substrate W is suction-held by the spin chuck 62, the peripheral edge of the substrate W extends beyond the outer circumferential edge of the spin chuck 62. The spin chuck 62 may be another type of chuck, such as a clamping-type mechanical chuck.

[0032] The spin chuck 62 is connected to the spin motor 63 via a motor shaft. That is, the upper end of the motor shaft of the spin motor 63 is connected to the center of the lower surface of the spin chuck 62. When the spin motor 63 rotates the motor shaft while the substrate W is held by suction on the spin chuck 62, the substrate W and the spin chuck 62 rotate in a horizontal plane around a rotation axis along the vertical direction.

[0033] A cup 64 is provided to surround the periphery of the spin chuck 62. The cup 64 can be raised and lowered by a lifting mechanism (not shown). The cup 64 has a cylindrical shape, and the upper part of the cup 64 is inclined so that it approaches the spin chuck 62 as it goes up. However, the inner diameter of the upper end part of the cup 64 is larger than the diameter of the substrate W. During processing of the substrate W, the upper end of the cup 64 is higher than the height position of the substrate W held by the spin chuck 62. Therefore, liquid scattered by centrifugal force from the substrate W rotated by the spin motor 63 is received and recovered by the cup 64. The liquid recovered by the cup 64 is discharged from a drain pipe provided at the bottom of the cup 64. The cup 64 may have a multi-stage structure with multiple recovery ports provided for different purposes.

[0034] The discharge nozzle 65 discharges a processing liquid onto the substrate W held on the spin chuck 62. The processing liquid is a term that encompasses various chemical liquids and pure water. Examples of the chemical liquid include a liquid for etching or a liquid for removing particles. Specifically, an SC-1 liquid (a mixed solution of ammonium hydroxide, hydrogen peroxide, and pure water), an SC-2 liquid (a mixed solution of hydrochloric acid, hydrogen peroxide, and pure water), or hydrofluoric acid is used. The discharge nozzle 65 is moved by a drive mechanism (not shown) between a processing position above the spin chuck 62 and a standby position outside the cup 64. At the processing position, the discharge nozzle 65 discharges the chemical liquid onto the substrate W held on the spin chuck 62, thereby performing, for example, an etching process on the substrate W. Furthermore, the discharge nozzle 65 discharges pure water onto the substrate W, thereby performing a pure water rinse process on the substrate W.

[0035] A worker who performs maintenance and inspection work on a plurality of substrate processing apparatuses 50 arranged in a clean room 40 wears smart glasses 10. The smart glasses 10 are a type of wearable terminal that uses a head-mounted display (HMD). The smart glasses 10 are also devices that realize AR (Augmented Reality) or MR (Mixed Reality). For example, Microsoft's "HoloLens" (registered trademark) can be used as the smart glasses 10.

[0036] FIG. 6 is a perspective view showing the appearance of the smart glasses 10. The smart glasses 10 include a visor 11 and a headband 12. A worker wears the smart glasses 10 by placing the headband 12 on their head. The worker can adjust the length of the headband 12 to fit the size of their head. The headband 12 also includes a power button, a brightness button, a volume button, and the like.

[0037] The visor 11 includes various sensors and a display. The display is a see-through holographic lens. That is, the display can display a three-dimensional image in the worker's field of view using a hologram, and transmits light from real objects in the same way as regular eyeglass lenses. Therefore, a worker wearing the smart glasses 10 can view the displayed three-dimensional image while viewing real objects through the display.

[0038] The sensors of the visor 11 include, for example, multiple visible light cameras that mainly capture images in front of the visor 11, an infrared camera that tracks the worker's line of sight, a depth sensor that measures the distance to an object, and an inertial measurement sensor. The infrared camera tracks the line of sight by measuring the movement of the eyeballs of the person wearing the smart glasses 10. The depth sensor measures the distance to an object using, for example, the ToF (Time of Flight) method. The inertial measurement sensor is composed of an accelerometer, a gyroscope, a magnetometer, etc.

[0039] The smart glasses 10 also have a built-in computer including a CPU, memory, etc. The smart glasses 10 are also provided with a wireless communication mechanism, and the computer in the smart glasses 10 connects to the information and communication network 5 using the wireless communication mechanism. The smart glasses 10 are also provided with a microphone, a speaker, a battery, etc.

[0040] 7 is a block diagram showing the functional configuration of the smart glasses 10, the server 70, the work support terminal 80, and the control unit 55 of the substrate processing apparatus 50. The smart glasses 10 include, as hardware elements, an imaging unit 21, a communication unit 22, a display unit 23, a storage unit 24, and a spatial recognition unit 25. The imaging unit 21 includes a visible light camera provided in the visor 11 described above. The imaging unit 21 includes, for example, four visible light cameras that capture images of the front and diagonally forward, and can capture images of the field of view of the worker wearing the smart glasses 10.

[0041] The communication unit 22 includes the wireless communication mechanism of the smart glasses 10 described above. The communication unit 22 transmits and receives data to and from the work support terminal 80 and the server 70 via the information and communication network 5. The communication unit 22 can also transmit and receive data directly to and from the control unit 55 of the substrate processing apparatus 50 if the distance is short. That is, the communication unit 22 can transmit data and commands to the control unit 55 of the substrate processing apparatus 50 directly or via the information and communication network 5.

[0042] The display unit 23 includes the display of the visor 11. The display unit 23 has a holographic processing device and displays a 3D image at a predetermined spatial position using hologram technology. Note that the 3D image displayed by the display unit 23 is not limited to a 3D shape, and may be a 2D image such as a document.

[0043] The storage unit 24 includes a memory and storage installed in the smart glasses 10. The memory and storage provided in the smart glasses 10 are, for example, a dynamic random access memory (DRAM) and a universal flash storage (UFS), respectively. The storage unit 24 stores applications and data used by the computer of the smart glasses 10.

[0044] The spatial recognition unit 25 includes the depth sensor of the smart glasses 10 described above. The depth sensor can measure the distance to an object. The spatial recognition unit 25 can perform spatial mapping by scanning a space using the depth sensor. Specifically, the spatial recognition unit 25 creates a spatial mesh, which is mesh data, by scanning the space. Because the spatial recognition unit 25 uses the depth sensor, it can scan a space even in a dark place.

[0045] The smart glasses 10 also include an image conversion unit 31, a comparison unit 32, a warning issuance unit 33, and a position registration unit 34. The image conversion unit 31, the comparison unit 32, the warning issuance unit 33, and the position registration unit 34 are function processing units realized by the CPU of the smart glasses 10 executing a predetermined processing program. The processing contents of the image conversion unit 31, the comparison unit 32, the warning issuance unit 33, and the position registration unit 34 will be described in further detail below.

[0046] The control unit 55 of the substrate processing apparatus 50 controls the operation of mechanisms provided in the processing unit 52, such as the discharge nozzle 65. The control unit 55 of the substrate processing apparatus 50 can communicate with the communication unit 22 of the smart glasses 10, and can also control the operation of various mechanisms provided in the processing unit 52 in accordance with operation instruction commands transmitted from the smart glasses 10.

[0047] The work support terminal 80 is installed, for example, in a factory of a vendor that manufactures and maintains the substrate processing apparatus 50. The server 70 is installed in a clean room 40 in which a plurality of substrate processing apparatuses 50 are installed. The work support terminal 80 and the server 70 are capable of communicating with the smart glasses 10 via an information and communication network 5. The work support terminal 80 and the server 70 are also capable of communicating with each other via the information and communication network 5.

[0048] The work support terminal 80 and the server 70 are general computer systems. That is, the work support terminal 80 and the server 70 are equipped with a CPU which is a circuit that performs various arithmetic processing, a ROM which is a read-only memory that stores basic programs, a RAM which is a readable and writable memory that stores various information, a storage unit (for example, a magnetic disk or SSD) that stores control software and data, and a communication unit that communicates with the information communication network 5.

[0049] The work support terminal 80 is a computer that, for example, allows a work supporter on the vendor side to support the work of a worker in the clean room 40. The work supporter can send various information from the work support terminal 80 to the smart glasses 10 worn by the worker in the clean room 40.

[0050] In the work support system according to the present invention, the server 70 is a computer that executes predetermined processes in response to requests from the smart glasses 10 and the work support terminal 80. The server 70 includes a memory unit 74 with a relatively large capacity. Large-sized data created by the smart glasses 10 and the work support terminal 80 may be stored in the memory unit 74.

[0051] Next, a work assistance method using the work assistance system having the above-described configuration will be described. The work assistance method according to the present invention includes a preparation step of registering normal state data as reference data, and a step of a worker wearing the smart glasses 10 detecting an abnormality in the work area. First, the preparation step will be described.

[0052] 8 is a flowchart showing the steps of advance preparation in the work support method according to the present invention. First, a spatial mesh in a normal state is created for any location in the clean room 40 using the smart glasses 10 (step S11). The locations to be scanned to create the spatial mesh include, for example, the outer wall of the substrate processing apparatus 50 and the floor of the clean room 40. For example, the space recognition unit 25 of the smart glasses 10 scans the space including the outer wall of the substrate processing apparatus 50 and the floor of the clean room 40 to create the spatial mesh, which is mesh data.

[0053] To operate the smart glasses 10 to create a spatial mesh, the worker wearing the smart glasses 10 simply turns on the scan mode. The worker selects the scan mode using a hand gesture from a menu screen displayed as a 3D image on the display unit 23, for example. The hand gesture is captured and detected by the imaging unit 21, and the computer in the smart glasses 10 recognizes from the detection result that the scan mode has been selected and turns on the scan mode. Alternatively, the worker may turn on the scan mode by pressing a predetermined button provided on the smart glasses 10. When the scan mode is turned on, the space recognition unit 25 of the smart glasses 10 begins scanning the space.

[0054] In step S11, the space recognition unit 25 of the smart glasses 10 scans the space including the target location in a normal state and creates a space mesh. The normal state is a safe and constant state in which no foreign objects, openings, etc. exist.

[0055] The space mesh created by the space recognition unit 25 by scanning the space is mesh data represented by a large number of triangular meshes. Various shapes, including curved and flat surfaces, are represented by a set of many connected triangles.

[0056] A fundamental characteristic of spatial meshes is that complex shapes including irregularities are represented by a high density of triangles, while flat shapes are represented by a relatively low density of triangles. Figure 9 shows an example of a spatial mesh created by scanning a space containing irregular foreign objects. In the example of Figure 9, the space around a triangular cone installed in a clean room 40 is scanned. Meanwhile, Figure 10 shows an example of a spatial mesh created by scanning a flat surface. In the example of Figure 10, the floor surface of the clean room 40 is scanned.

[0057] As shown in Figure 9, a spatial mesh created by scanning a space containing foreign objects such as triangular cones placed in a clean room 40 is represented by a collection of many relatively small triangles. Therefore, the density of triangles in the spatial mesh is high, and the number of vertices of triangles contained within a unit area is also large (vertices are contained at a high density). In contrast, as shown in Figure 10, a spatial mesh created by scanning the floor, which is a flat surface within the clean room 40, is represented by a collection of few relatively large triangles. Therefore, the density of triangles in the spatial mesh is low, and the number of vertices of triangles contained within a unit area is also small (vertices are contained at a low density).

[0058] Another fundamental characteristic of creating a spatial mesh is that the positions of the vertices of triangles are different each time the same object is scanned (i.e., the vertices are not reproducible). As in the example shown in Figure 9, a spatial mesh created by scanning a space containing a foreign object has a high density of triangular vertices, so even if the positions of the individual vertices differ with each scan, the shape of the foreign object is captured in all spatial meshes from a macroscopic perspective. On the other hand, as in the example shown in Figure 10, a spatial mesh created by scanning a flat surface has a low density of triangular vertices, so if the positions of the individual vertices differ with each scan, a different spatial mesh will be created with each scan.

[0059] In this embodiment, it is determined whether the spatial mesh created in step S11 satisfies the detection condition (step S12). This process may be performed by the smart glasses 10 or a separate computer (e.g., the work support terminal 80). Specifically, if the density of triangle vertices in the spatial mesh created in step S11 is less than a predetermined threshold, the spatial mesh is determined not to satisfy the detection condition. For example, the spatial mesh (FIG. 9) created by scanning the space containing the substrate processing apparatus 50 and foreign matter in the clean room 40 is determined to satisfy the detection condition because the density of triangle vertices is relatively high. On the other hand, the spatial mesh (FIG. 10) created by scanning the floor of the clean room 40 is determined not to satisfy the detection condition because the density of triangle vertices is relatively low.

[0060] If it is determined that the spatial mesh created in step S11 satisfies the detection conditions, the process proceeds from step S12 to step S13, where the created spatial mesh is registered as a reference spatial mesh. For example, the created spatial mesh may be stored in the memory unit 24 of the smart glasses 10 as the reference spatial mesh. This registers the reference spatial mesh, which is mesh data acquired by scanning the space in the clean room 40, including uneven areas in a normal state. Note that if the data volume of the reference spatial mesh is large, it may be stored in the memory unit 74 of the server 70, for example.

[0061] On the other hand, if it is determined that the spatial mesh created in step S11 does not satisfy the detection conditions, the process proceeds from step S12 to step S14, where the created spatial mesh is converted into a two-dimensional image.

[0062] The spatial mesh, which is mesh data, can be converted into a two-dimensional image using a 3D object library such as "Open GL" (registered trademark) manufactured by Silicon Graphics, Inc. FIG. 11 is a diagram showing an example of an image obtained by converting a spatial mesh created by scanning the floor of a clean room 40. When the spatial mesh obtained by scanning a flat surface such as a floor is converted into a two-dimensional image, the flat surface is represented by being filled in. In the image of FIG. 11, the floor of the clean room 40 is represented as a filled-in image object AJ1. Note that the hole (not filled in) near the center of the image object AJ1 in the example of FIG. 11 corresponds to the installation location of the substrate processing apparatus 50 on the floor.

[0063] Thereafter, the data of the two-dimensional image obtained by the conversion is registered as reference image data (step S15). For example, the obtained two-dimensional image data may be stored as reference image data in the storage unit 24 of the smart glasses 10. This results in the registration of reference image data, which is two-dimensional image data of a flat surface in a normal state within the clean room 40. Note that if the amount of data of the reference image data is large, it may be stored in the storage unit 74 of the server 70, for example.

[0064] Next, anomaly detection using the smart glasses 10 will be described with reference to Figs. 12 to 14. Fig. 12 is a flowchart showing the procedure for the initial stage of anomaly detection in the work assistance method according to the present invention. Fig. 13 is a flowchart showing the procedure when the target space mesh satisfies the detection conditions. Fig. 14 is a flowchart showing the procedure when the target space mesh does not satisfy the detection conditions.

[0065] First, a worker wearing the smart glasses 10 enters the work area in the clean room 40 at an appropriate time and creates a space mesh (step S21). FIG. 15 is a diagram for explaining how a worker wearing the smart glasses 10 performs work for anomaly detection. A substrate processing apparatus 50 is arranged in a work area 101 in the clean room 40. A worker performing anomaly inspection wears the smart glasses 10 and walks around the work area 101 with the scan mode turned on. In the example of FIG. 15, the worker wearing the smart glasses 10 walks around the periphery of the substrate processing apparatus 50 along the path indicated by the arrow AR15. As the worker walks around the work area 101 with the scan mode of the smart glasses 10 turned on, the space recognition unit 25 sequentially scans the space near the smart glasses 10 and acquires a space mesh (target space mesh). When the worker wearing the smart glasses 10 faces the substrate processing apparatus 50, the space recognition unit 25 scans the space including the substrate processing apparatus 50 and acquires a space mesh. Furthermore, when the worker is looking at the floor surface of the clean room 40, the space recognition unit 25 scans the floor surface, which is a flat surface, to obtain a space mesh.

[0066] Next, it is determined whether the spatial mesh created in step S21 satisfies the detection condition (step S22). This determination process may be performed by, for example, the smart glasses 10. Specifically, if the density of triangle vertices in the spatial mesh created in step S21 is less than a predetermined threshold, it is determined that the spatial mesh does not satisfy the detection condition. The threshold value at this time is preferably the same as the threshold value used in the determination in step S12. That is, it is preferable to align the determination criteria used in the advance preparation stage with the determination criteria used when detecting an abnormality. The spatial mesh created in step S21 by scanning the space containing the substrate processing apparatus 50 and foreign matter is determined to satisfy the detection condition because the density of triangle vertices is relatively high. On the other hand, the spatial mesh created in step S21 by scanning the floor of the clean room 40 is determined to not satisfy the detection condition because the density of triangle vertices is relatively low.

[0067] If it is determined that the spatial mesh created in step S21 satisfies the detection condition, the process proceeds to step S23 in Fig. 13, where the target spatial mesh is compared with the reference spatial mesh. Specifically, the comparison unit 32 of the smart glasses 10 compares the spatial mesh created in step S21 (target spatial mesh) with the reference spatial mesh that was created and registered in the advance preparation step for the same space. The comparison unit 32 compares the target spatial mesh with the reference spatial mesh and extracts the difference between them.

[0068] Next, the comparison unit 32 determines whether the difference between the target space mesh and the reference space mesh is equal to or greater than a predetermined threshold (first threshold) (step S24). More specifically, the comparison unit 32 determines whether the difference value obtained by quantifying the difference between the target space mesh and the reference space mesh is equal to or greater than the threshold.

[0069] If the result of the judgment is that the difference between the target space mesh and the reference space mesh is equal to or greater than the threshold, this means that some change, such as a foreign object, has occurred in the target location when the target space mesh was created compared to the normal state when the reference space mesh was registered. For example, in the example of Figure 15, if a foreign object 105, such as a tool, is placed in the work area 101 after the reference space mesh is registered, the difference between the target space mesh created by scanning the space including the foreign object 105 and the reference space mesh will be equal to or greater than the threshold.

[0070] If the difference between the target space mesh and the reference space mesh is equal to or greater than the threshold, the process proceeds from step S24 to step S25, where the warning unit 33 of the smart glasses 10 issues a warning. Specifically, for example, the warning unit 33 displays a warning message as a 3D image on the display unit 23. FIG. 17 is a diagram showing an example of a warning message displayed by the warning unit 33. By displaying the warning message, the worker can recognize changes in the surrounding situation and pay attention to those changes, thereby ensuring the safety of the worker. In the example of FIG. 15, the worker can pay attention to the presence of the newly placed foreign object 105 and can work safely.

[0071] Next, the position registration unit 34 of the smart glasses 10 registers the position information of the difference part between the target space mesh and the reference space mesh in the storage unit 74 of the server 70 (step S26). In the example of FIG. 15, the position registration unit 34 registers the position information of the foreign matter 105, which is the difference part, in the server 70. The position information to be registered can be any appropriate information. For example, the coordinate position of the foreign matter 105 in the clean room 40 may be registered, or the relative position of the foreign matter 105 from the substrate processing apparatus 50 in the work area 101 may be registered.

[0072] On the other hand, if the difference between the target space mesh and the reference space mesh is less than the threshold, it means that no significant changes have occurred in the target location when the target space mesh was created compared to the normal state when the reference space mesh was registered. In this case, the process proceeds from step S24 to step S27, and the worker continues with the desired work. Since no significant changes have occurred in the environment around the worker from the normal state, the worker can work safely.

[0073] Furthermore, if it is determined in step S22 that the spatial mesh created in step S21 does not satisfy the detection conditions, the process proceeds to step S32 in FIG. 14, where the image conversion unit 31 of the smart glasses 10 converts the created target spatial mesh into a two-dimensional image to acquire image data (target image data). The method by which the image conversion unit 31 converts the spatial mesh into a two-dimensional image can be performed using a 3D object library, as in step S14 described above. FIG. 16 is a diagram showing an example of an image obtained by converting a spatial mesh created by scanning the floor of the clean room 40 when an abnormality is detected. In the image in FIG. 16, the floor of the clean room 40 is represented as image object AJ2.

[0074] Next, the target image data and the reference image data are compared (step S33). Specifically, the comparison unit 32 of the smart glasses 10 compares the target image data acquired in step S32 with the reference image data that has been created and registered in the advance preparation process for the same space. In the above example, the comparison unit 32 compares the data of the target image shown in FIG. 16 with the data of the reference image shown in FIG. 11. The comparison unit 32 compares the target image data and the reference image data to extract the difference between them.

[0075] Next, the comparison unit 32 determines whether the difference between the target image data and the reference image data is equal to or greater than a predetermined threshold (second threshold) (step S34). More specifically, the comparison unit 32 determines whether the difference value (for example, the total value of the gradation difference) obtained by quantifying the difference between the target image data and the reference image data is equal to or greater than a threshold.

[0076] If the result of the determination is that the difference between the target image data and the reference image data is greater than or equal to the threshold, this indicates that some change, such as an opening, occurred in the target location when the target image data was created compared to the normal state when the reference image data was registered. For example, in the example of FIG. 15 , assume that after the reference image data was registered, a grating was removed from the floor in the work area 101, forming an opening 108. A two-dimensional image obtained by converting the spatial mesh created by scanning the floor surface including the opening 108 is shown in FIG. 16 . The comparison unit 32 compares the data of the target image shown in FIG. 16 with the data of the reference image shown in FIG. 11 to extract the difference 109 and determine that the difference 109 is greater than or equal to the threshold. This difference 109 is caused by the opening 108 being formed in the floor surface in a normal state. In other words, the formation of the opening 108 compared to the normal state when the reference image data was registered results in the difference 109 between the target image data and the reference image data, which is greater than or equal to the threshold.

[0077] The processing contents of steps S35 to S37 are generally the same as the processing contents of steps S25 to S27. That is, if the difference between the target image data and the reference image data is equal to or greater than the threshold, the process proceeds from step S34 to step S35, and the warning issuing unit 33 of the smart glasses 10 issues a warning, for example, as shown in FIG. 17. This allows the worker to recognize changes in the surrounding situation and pay attention to those changes, thereby ensuring the worker's safety. In the example of FIG. 15, the worker can pay attention to the presence of the opening 108 newly formed in the floor surface, and can perform the work safely.

[0078] Next, the position registration unit 34 of the smart glasses 10 registers the position information of the difference part between the target image data and the reference image data in the storage unit 74 of the server 70 (step S36). In the example of FIG. 15 , the position registration unit 34 registers the position information of the opening 108, which is the difference part, in the server 70.

[0079] On the other hand, if the difference between the target image data and the reference image data is less than the threshold, it means that there has been no significant change in the floor surface of the target area when the target image data was created compared to the normal state when the reference image data was registered. In this case, the process proceeds from step S34 to step S37, and the worker continues the desired work. Since there has been no significant change in the environment around the worker from the normal state, the worker can work safely.

[0080] For example, if the spatial mesh created for the same target location in the advance preparation process does not satisfy the detection conditions, but the spatial mesh satisfies the detection conditions when an abnormality is detected, it is assumed that some kind of change has occurred in the target location. Therefore, in such a case, the warning issuing unit 33 can issue a warning without performing a comparison. The same applies to the reverse case.

[0081] In this embodiment, in the preparation step, a space including a predetermined location in a normal state is scanned to create a spatial mesh. If the density of triangle vertices in the spatial mesh is equal to or greater than a predetermined threshold (i.e., the detection condition is satisfied), the created spatial mesh is registered as a reference spatial mesh. On the other hand, if the density of triangle vertices in the spatial mesh is less than the predetermined threshold (i.e., the detection condition is not satisfied), the created spatial mesh is converted into a two-dimensional image and the acquired two-dimensional image data is registered as reference image data. The density of triangle vertices in the spatial mesh depends on the degree of unevenness in the scanned space. The greater the degree of unevenness, the higher the density of triangle vertices in the spatial mesh. The closer to a flat surface, the lower the density of triangle vertices. Therefore, for a substrate processing apparatus 50 or the like in a normal state that has a relatively large degree of unevenness in the clean room 40, a reference spatial mesh is registered as mesh data. For a flat, normal floor surface, the reference image data is registered as two-dimensional image data obtained by converting the mesh data.

[0082] Next, when an abnormality is detected, if the target space mesh satisfies the detection conditions, the target space mesh is compared with the reference space mesh. On the other hand, if the target space mesh does not satisfy the detection conditions, the target space mesh is converted into a two-dimensional image to obtain target image data, and this target image data is compared with the reference image data. If the comparison results in a difference in either data exceeding a threshold, an alarm is issued and the location information of the difference is registered.

[0083] As described above, in this embodiment, for areas with relatively large irregularities, a comparison is made using a spatial mesh, while for flat surfaces such as floors, a comparison is made using image data obtained by converting the spatial mesh into a two-dimensional image. That is, for areas including the substrate processing apparatus 50, etc., with relatively large irregularities, a reference spatial mesh that has been registered in advance as a normal state is compared with a target spatial mesh created when an abnormality is detected. On the other hand, for flat surfaces such as floors, a reference image data that has been registered in advance as a normal state is compared with target image data created when an abnormality is detected.

[0084] As mentioned above, even if the same object is scanned to create a spatial mesh, the positions of the triangle vertices will be different for each scan. In areas with relatively large irregularities, the density of the triangle vertices in the created spatial mesh is high, so even if the positions of the individual vertices change with each scan, the shape of foreign objects, etc. will be captured in all spatial meshes from a macro perspective. Therefore, it is possible to detect foreign objects, etc. by performing a comparison and judgment using the spatial mesh as is.

[0085] In contrast, for flat surfaces such as the floor of a clean room 40, the density of triangle vertices in the generated spatial mesh is low. Therefore, if the positions of individual vertices change with each scan, a completely different spatial mesh will be generated each time the same object is scanned. Therefore, even if the floor remains unchanged from its normal state, the spatial mesh generated during the preparatory process will be different from the spatial mesh generated during the anomaly detection process. As a result, performing a comparison using the spatial mesh may result in false detection of foreign objects, even if there is no change from the normal state. Conversely, even if an opening or other feature is present in the floor, the opening is unlikely to be reflected as a vertex of a triangle in the spatial mesh, and the comparison using the spatial mesh may fail to detect the opening or other feature.

[0086] For this reason, in this embodiment, for flat surfaces such as the floor of the clean room 40, a comparison is made using image data obtained by converting a spatial mesh into a two-dimensional image. Even if the density of the vertices of the triangles in the spatial mesh is low, by converting the spatial mesh into a two-dimensional image, the condition of the floor and other surfaces is accurately reflected in the two-dimensional image, and openings and the like also appear. Therefore, for flat surfaces such as the floor, a comparison is made using image data obtained by converting the spatial mesh into a two-dimensional image, making it possible to accurately detect newly formed openings and the like.

[0087] Furthermore, the spatial recognition unit 25 of the smart glasses 10 that creates the spatial mesh uses a depth sensor to scan the space, making it possible to process even in dark environments. In other words, according to this embodiment, even in dark environments, foreign objects, openings, etc. can be accurately detected, allowing the worker to recognize changes in the surrounding situation, regardless of whether the target is an area with a large degree of unevenness or a flat surface such as a floor.

[0088] Although the above describes an embodiment of the present invention, various modifications can be made to the present invention without departing from the spirit and scope of the present invention. For example, in the above embodiment, in the preparation step, if the created spatial mesh does not satisfy the detection conditions, the spatial mesh is converted into a two-dimensional image and the acquired two-dimensional image data is registered as reference image data. However, this is not limited to this. For example, a design drawing of the floor surface surrounding the substrate processing apparatus 50 in the clean room 40 may be converted into a predetermined data format and the acquired image data may be registered as reference image data. In other words, the determination in step S12 is not a required process, and the reference image data may be two-dimensional image data of a flat surface in a normal state, regardless of the format in which it was created.

[0089] In addition to or instead of displaying a warning message on the display unit 23, the warning unit 36 ​​may also emit a warning sound from the speaker of the smart glasses 10.

[0090] Furthermore, in the above embodiment, the worker uses smart glasses 10, but this is not limited thereto, and a portable terminal such as a tablet terminal or a smartphone may be used instead of the smart glasses 10. That is, any portable terminal equipped with an imaging unit, a display unit, a communication unit, etc. is sufficient. However, since using a tablet terminal or the like would occupy the worker's hands while holding it, it is preferable to use a wearable terminal such as the smart glasses 10.

[0091] Furthermore, the substrate processing apparatus 50 installed in the clean room 40 is not limited to a substrate cleaning apparatus, but may be any apparatus that performs a predetermined process on a substrate, such as a heat treatment apparatus, an exposure apparatus, a coating and developing apparatus, a measuring apparatus, or an inspection apparatus. When the substrate processing apparatus 50 is a substrate cleaning apparatus, it may be a single-wafer type cleaning apparatus that cleans substrates one by one, or a batch type cleaning apparatus that cleans multiple substrates at once.

[0092] Furthermore, the work support technology according to the present invention is not limited to substrate processing equipment, but may be applied to any industrial equipment that performs any type of processing, such as a printing processing equipment, a film forming equipment, a medical equipment, and an appearance inspection equipment. [Explanation of symbols]

[0093] 5. Information and Communications Network 10. Smart Glasses 21 Imaging unit 22 Communications Department 23 Display section 24 Memory section 25 Spatial recognition section 31 Image conversion unit 32 Comparison section 33 Warning and Notification Department 34 Location registration unit 40 Clean Room 50 Substrate processing equipment 52 Processing Unit 55 Control Unit 60 processing chambers 61 Rotation holding part 65 Discharge nozzle 70 servers 80 Work support terminal 101 Work Area 105 Foreign object 108 Opening 109 Difference AJ1,AJ2 image objects W substrate

Claims

1. A work support method for performing a predetermined work on industrial equipment, comprising: a pre-registration step of registering a reference space mesh, which is mesh data acquired by scanning a space including a predetermined location in a normal state using a mobile terminal equipped with a display unit, a communication unit, and a space recognition unit, and reference image data, which is two-dimensional image data of a flat surface in a normal state; a space recognition step of scanning a space including the predetermined location and / or the flat surface with the mobile terminal at a predetermined timing to acquire a target space mesh; an image conversion step of converting the target space mesh into a two-dimensional image to acquire target image data when the target space mesh does not satisfy the detection condition; a comparison step of comparing the target spatial mesh with the reference spatial mesh if the target spatial mesh satisfies the detection condition, and comparing the target image data with the reference image data if the target spatial mesh does not satisfy the detection condition; a warning issuing step of issuing a warning when a difference between the target spatial mesh and the reference spatial mesh is equal to or greater than a first threshold value, or when a difference between the target image data and the reference image data is equal to or greater than a second threshold value; A work support method comprising:

2. 2. The work support method according to claim 1, The work support method further comprises a position registration step of registering position information of the difference portion when the warning is issued in the warning issuing step.

3. 2. The work support method according to claim 1, A work support method in which the target space mesh is determined not to satisfy the detection condition if the density of vertices in the target space mesh is less than a predetermined threshold.

4. 2. The work support method according to claim 1, The reference image data is obtained by converting mesh data acquired by scanning a flat surface in a normal state using the mobile terminal into two-dimensional image data.

5. 2. The work support method according to claim 1, The work support method, wherein the industrial equipment is a substrate processing apparatus that performs a predetermined process on a substrate.

6. 6. The work support method according to claim 1, A work assistance method in which the mobile terminal is smart glasses.

7. A work support system for performing a predetermined work on industrial equipment using a mobile terminal, a mobile terminal having a display unit and a communication unit; a storage unit that stores a reference space mesh, which is mesh data acquired by scanning a space including a predetermined location in a normal state using the mobile terminal, and reference image data, which is two-dimensional image data of a flat surface in a normal state; a space recognition unit that scans a space including the predetermined location and / or the flat surface using the mobile terminal at a predetermined timing to acquire a target space mesh; an image conversion unit that converts the target space mesh into a two-dimensional image and acquires target image data when the target space mesh does not satisfy the detection condition; a comparison unit that compares the target spatial mesh with the reference spatial mesh if the target spatial mesh satisfies the detection condition, and compares the target image data with the reference image data if the target spatial mesh does not satisfy the detection condition; a warning issuing unit that issues a warning when a difference between the target space mesh and the reference space mesh is equal to or greater than a first threshold value, or when a difference between the target image data and the reference image data is equal to or greater than a second threshold value; A work support system comprising:

8. 8. The work support system according to claim 7, The work support system further comprises a position registration unit that registers position information of the difference portion when the warning issuance unit issues a warning.

9. 8. The work support system according to claim 7, A work support system in which the target space mesh is determined not to satisfy the detection condition when the density of vertices in the target space mesh is less than a predetermined threshold.

10. 8. The work support system according to claim 7, The reference image data is obtained by converting mesh data acquired by scanning a flat surface in a normal state using the mobile terminal into two-dimensional image data.

11. 8. The work support system according to claim 7, The industrial equipment is a work support system that is a substrate processing apparatus that performs a predetermined process on a substrate.

12. The work support system according to any one of claims 7 to 11, The work support system, wherein the mobile terminal is a smart glass.

Images