Work assistance method and work assistance system

Abstract

According to the present invention, a spatial mesh acquired by using a spatial recognition unit of smart glasses to scan a space including a target location in a normal state is pre-registered as a reference spatial mesh. Furthermore, two-dimensional image data of a flat surface in a normal state are pre-registered as reference image data. During abnormality detection, a worker wearing the smart glasses enters a work area and performs a scan to acquire a target spatial mesh. If the target spatial mesh satisfies a detection condition, the target spatial mesh is compared with the reference spatial mesh. Meanwhile, if the target spatial mesh does not satisfy the detection condition, the target spatial mesh is converted into a two-dimensional image to acquire target image data, and the target image data are compared with the reference image data.

Description

Work support method and work support system

[0001] The present invention relates to a method and system for supporting operations when performing predetermined operations such as maintenance operations on industrial equipment such as substrate processing apparatuses that perform predetermined operations on substrates. Substrates to be processed by the substrate processing apparatuses include, for example, semiconductor substrates, substrates for liquid crystal display devices, substrates for flat panel displays (FPDs), substrates for optical disks, substrates for magnetic disks, and substrates for solar cells.

[0002] Conventionally, in the manufacturing process of semiconductor devices, substrate processing apparatuses are used to perform various processes on substrates such as semiconductor substrates. Examples of substrate processing apparatuses include substrate cleaning apparatuses, heat treatment apparatuses, and inspection apparatuses. Typically, a large number of substrate processing apparatuses are arranged in an orderly fashion in a large clean room. Maintenance is performed on these substrate processing apparatuses at appropriate intervals. Patent Document 1 describes arranging a large number of substrate processing apparatuses at a relatively high density in a clean room and performing maintenance on the substrate processing apparatuses.

[0003] Japanese Patent Application Laid-Open No. 2020-4866

[0004] In a clean room where a substrate processing apparatus is installed, foreign objects such as tools and parts may be placed. Also, the grating on the floor of the clean room may be removed to form openings. Generally, workers working in the clean room visually check for foreign objects and openings to avoid contact with them.

[0005] However, when performing maintenance on a substrate processing apparatus, workers may have to work in a dark environment. For example, an area known as a yellow room, where ultraviolet light is blocked, is a particularly dark environment. Even if foreign objects or openings are present in such a dark environment, it may be difficult for the workers to see them with the naked eye. This may result in the risk of the workers colliding with foreign objects or falling through openings during work.

[0006] The present invention has been made in consideration of the above-mentioned problems, and aims to provide a work assistance method and a work assistance system that enable a worker to recognize changes in the surrounding situation even in a dark environment.

[0007] In order to solve the above problem, a first aspect of the present invention is a work support method for performing specified work on industrial equipment, comprising: a pre-registration step of registering a reference spatial mesh, which is mesh data obtained by scanning a space including a specified location in a normal state using a portable terminal equipped with a display unit, a communication unit, and a spatial recognition unit, and reference image data, which is two-dimensional image data of a flat surface in a normal state; a spatial recognition step of scanning the space including the specified location and / or the flat surface using the portable terminal at a specified timing to obtain a target spatial mesh; an image conversion step of converting the target spatial mesh into a two-dimensional image to obtain target image data when the target spatial mesh does not satisfy a detection condition; a comparison step of comparing the target spatial mesh with the reference spatial mesh if the target spatial mesh satisfies the detection condition, and comparing the target image data with the reference image data if the target spatial mesh does not satisfy the detection condition; and an alert step of issuing an alert if a difference between the target spatial mesh and the reference spatial mesh is equal to or greater than a first threshold value or if a difference between the target image data and the reference image data is equal to or greater than a second threshold value.

[0008] In addition, a second aspect is the work support method according to the first aspect, further comprising a position registration step of registering position information of the difference portion when the warning is issued in the warning issuing step.

[0009] In addition, in a third aspect, in the work support method according to the first or second aspect, it is determined that the target space mesh does not satisfy the detection condition if the density of vertices in the target space mesh is less than a predetermined threshold value.

[0010] In addition, in a fourth aspect, in the work support method according to any one of the first to third aspects, the reference image data is obtained by converting mesh data obtained by scanning a flat surface in a normal state using the mobile terminal into two-dimensional image data.

[0011] A fifth aspect is the work support method according to any one of the first to fourth aspects, wherein the industrial equipment is a substrate processing apparatus that performs a predetermined process on a substrate.

[0012] In addition, a sixth aspect is the task assistance method according to any one of the first to fifth aspects, wherein the mobile terminal is a pair of smart glasses.

[0013] a space recognition unit that scans the space including the predetermined location and / or the flat surface using the portable terminal at a predetermined timing to acquire a target space mesh; an image conversion unit that converts the target space mesh into a two-dimensional image to acquire target image data when the target space mesh does not satisfy a detection condition; a comparison unit that compares the target space mesh with the reference space mesh if the target space mesh satisfies the detection condition, and compares the target image data with the reference image data if the target space mesh does not satisfy the detection condition; and a warning issuance unit that issues a warning if a difference between the target space mesh and the reference space mesh is equal to or greater than a first threshold value or if a difference between the target image data and the reference image data is equal to or greater than a second threshold value.

[0014] In an eighth aspect, the work assistance system according to the seventh aspect further comprises a position registration unit that registers position information of the difference portion when the warning issuing unit issues a warning.

[0015] In addition, in a ninth aspect, in the work support system according to the seventh or eighth aspect, it is determined that the target space mesh does not satisfy the detection condition if the density of vertices in the target space mesh is less than a predetermined threshold value.

[0016] In addition, in a tenth aspect, in the work support system according to any one of the seventh to ninth aspects, the reference image data is obtained by converting mesh data acquired by scanning a flat surface in a normal state using the mobile terminal into two-dimensional image data.

[0017] An eleventh aspect is the work support system according to any one of the seventh to tenth aspects, wherein the industrial equipment is a substrate processing apparatus that performs a predetermined process on a substrate.

[0018] In addition, a twelfth aspect is the work assistance system according to any one of the seventh to eleventh aspects, wherein the mobile terminal is a pair of smart glasses.

[0019] According to the work support methods of the first to sixth aspects, if the difference between the target space mesh obtained by scanning a space including a specified location at a specified timing and the reference space mesh for the specified location in a normal state is equal to or greater than a first threshold value, or if the difference between the target image data obtained by converting the target space mesh obtained at a specified timing into a two-dimensional image and the reference image data is equal to or greater than a second threshold value, an alert is issued when a change occurs in the surroundings, allowing the worker to recognize changes in the surrounding situation even in a dark environment.

[0020] According to the work support systems of the seventh to twelfth aspects, if the difference between the target space mesh obtained by scanning a space including a specified location at a specified timing and the reference space mesh for the specified location in a normal state is equal to or greater than a first threshold value, or if the difference between the target image data obtained by converting the target space mesh obtained at a specified timing into a two-dimensional image and the reference image data is equal to or greater than a second threshold value, an alert is issued when there is a change in the surroundings, allowing the worker to recognize changes in the surrounding situation even in a dark environment.

[0021] FIG. 1 is a diagram schematically illustrating the configuration of a work support system according to the present invention. FIG. 2 is a plan view showing an example of the layout of multiple substrate processing apparatuses. FIG. 3 is a side view showing the configuration of the substrate processing apparatus. FIG. 4 is a plan view of the substrate processing apparatus. FIG. 5 is a diagram showing the schematic configuration of a processing unit. FIG. 6 is a perspective view showing the appearance of smart glasses. FIG. 7 is a block diagram showing the functional configuration of a control unit of the smart glasses, a server, a work support terminal, and a substrate processing apparatus. FIG. 8 is a flowchart showing the advance preparation procedure in the work support method according to the present invention. FIG. 9 is a diagram showing an example of a spatial mesh created by scanning a space containing an uneven foreign object. FIG. 10 is a diagram showing an example of a spatial mesh created by scanning a flat surface. FIG. 11 is a diagram showing an example of an image obtained by converting a spatial mesh created by scanning the floor of a clean room. FIG. 12 is a flowchart showing the procedure in the initial stage of anomaly detection in the work support method according to the present invention. FIG. 13 is a flowchart showing the procedure when the target spatial mesh satisfies the detection condition. FIG. 14 is a flowchart showing the procedure when the target spatial mesh does not satisfy the detection condition. FIG. 15 is a diagram for explaining how a worker wearing smart glasses performs work for anomaly detection. Fig. 16 is a diagram showing an example of an image obtained by converting a spatial mesh created by scanning the floor of a clean room when an abnormality is detected. Fig. 17 is a diagram showing an example of a warning message displayed by the warning issuing unit.

[0022] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Hereinafter, expressions indicating relative or absolute positional relationships (e.g., "in one direction," "along one direction," "parallel," "orthogonal," "center," "concentric," "coaxial," etc.) not only strictly represent the positional relationship but also represent a state of relative angular or distance displacement within a tolerance or a range that provides equivalent functionality, unless otherwise specified. Furthermore, expressions indicating an equal state (e.g., "identical," "equal," "homogeneous," etc.) not only represent a state of strict quantitative equality but also represent a state of difference that provides a tolerance or equivalent functionality, unless otherwise specified. Furthermore, expressions indicating a shape (e.g., "circular," "square," "cylindrical," etc.) not only represent a geometrically strict shape but also represent a shape within a range that provides equivalent functionality, such as irregularities or chamfers, unless otherwise specified. Furthermore, expressions such as "comprise," "comprise," "include," "have," etc., regarding components, are not exclusive expressions that exclude the presence of other components. Furthermore, the expression "at least one of A, B, and C" includes "A only," "B only," "C only," "any two of A, B, and C," and "all of A, B, and C."

[0023] FIG. 1 is a diagram illustrating a schematic configuration of a work support system according to the present invention. The work support system according to the present invention includes a plurality of substrate processing apparatuses 50, smart glasses 10, a server 70, and a work support terminal 80. Controllers of the smart glasses 10 and the substrate processing apparatuses 50 are connected to an information and communication network 5 (e.g., the Internet) via wireless communication. The work support terminal 80 and the server 70 are connected to the information and communication network 5 via a wired connection. Information can be transmitted and received between devices connected to the information and communication network 5, for example, information can be exchanged between the smart glasses 10 and the work support terminal 80. Note that whether each device is connected to the information and communication network 5 wirelessly or via a wired connection is not limited to the above example and can be any suitable connection (e.g., the work support terminal 80 may be connected to the information and communication network 5 wirelessly).

[0024] 2 is a plan view showing an example of the layout of a plurality of substrate processing apparatuses 50. As shown in FIG. 2, a plurality of substrate processing apparatuses 50 are arranged in a clean room 40 in a regular line at regular intervals. The clean room 40 is provided, for example, in a semiconductor device manufacturing factory, and is a room in which a certain level of air cleanliness is ensured and temperature and humidity are controlled. In this embodiment, a plurality of substrate processing apparatuses 50 of the same type and model are arranged in the relatively large clean room 40. Therefore, it is difficult for an operator to distinguish the substrate processing apparatuses 50 arranged in the clean room 40 from their appearance alone.

[0025] Fig. 3 is a side view showing the configuration of one substrate processing apparatus 50. Fig. 4 is a plan view of the substrate processing apparatus 50. The substrate processing apparatus 50 is, for example, a single-wafer substrate cleaning apparatus that cleans substrates one by one. The substrate processing apparatus 50 has an indexer 51, a plurality of processing units 52, a transfer robot 56, and a main transport robot 57.

[0026] Carriers C each accommodating a plurality of substrates W are placed on the indexer 51. For example, three carriers C can be placed on the indexer 51. The transfer robot 56 is configured to be able to slide along the arrangement direction of the plurality of carriers C, to move up and down, to rotate, and to move its hand forward and backward. Unprocessed substrates W are removed from the carriers C placed on the indexer 51 by the transfer robot 56. Processed substrates W are stored in the carriers C placed on the indexer 51 by the transfer robot 56. The carriers C are, for example, FOUPs (front opening unified pods) that store substrates W in an enclosed space.

[0027] In the substrate processing apparatus 50, for example, three processing units 52 are stacked to form one stack. Then, for example, four stacks are arranged around the main transport robot 57 of the substrate processing apparatus 50. That is, one substrate processing apparatus 50 includes, for example, 12 (=3×4) processing units 52.

[0028] The main transport robot 57, which is disposed at the center of the four stacks, is configured to be capable of lifting and lowering, rotating, and moving its transport arm AM back and forth. The main transport robot 57 can transfer substrates W to and from all 12 processing units 52. The main transport robot 57 receives unprocessed substrates W from the transfer robot 56 and transfers them into any of the 12 processing units 52. The main transport robot 57 also removes processed substrates W from the processing units 52 and transfers them to the transfer robot 56.

[0029] The substrate processing apparatus 50 also includes a control unit 55. The control unit 55 is a general-purpose computer that controls the operations of the transfer robot 56, the main transport robot 57, and each of the processing units 52 provided within the apparatus. The control unit 55 has a touch panel that serves as an input / output interface provided on a wall surface of the apparatus, and a communication unit that communicates with the outside of the apparatus. While the control unit 55 is shown in the indexer 51 in Figures 3 and 4 for convenience of illustration, the present invention is not limited to this, and the control unit 55 may be provided at an appropriate position within the substrate processing apparatus 50.

[0030] 5 is a diagram showing a schematic configuration of the processing unit 52. The processing unit 52 includes a processing chamber 60, a spin holder 61, and a discharge nozzle 65. The processing chamber 60 is a hollow housing. The spin holder 61, the discharge nozzle 65, and the like are provided inside the processing chamber 60. The processing chamber 60 also has a loading / unloading port (not shown). The loading / unloading port is opened and closed by a shutter. With the loading / unloading port open, the main transport robot 57 loads and unloads a substrate W into and from the processing chamber 60. The loading / unloading port is closed during processing of the substrate W. The processing chamber 60 also has an air supply mechanism and an exhaust mechanism (not shown).

[0031] The rotation holder 61 includes a spin chuck 62 and a spin motor 63. The spin chuck 62 is a substrate holder that holds the substrate W in a horizontal position (a position in which the normal to the main surface of the substrate W is aligned vertically). The spin chuck 62 is, for example, a vacuum suction-type chuck. The spin chuck 62 has a disk shape with a diameter smaller than the diameter of the substrate W. The spin chuck 62 suction-holds the central portion of the lower surface of the substrate W. When the lower surface of the substrate W is suction-held by the spin chuck 62, the peripheral edge of the substrate W extends beyond the outer circumferential edge of the spin chuck 62. The spin chuck 62 may be another type of chuck, such as a clamping-type mechanical chuck.

[0032] The spin chuck 62 is connected to the spin motor 63 via a motor shaft. That is, the upper end of the motor shaft of the spin motor 63 is connected to the center of the lower surface of the spin chuck 62. When the spin motor 63 rotates the motor shaft while the substrate W is held by suction on the spin chuck 62, the substrate W and the spin chuck 62 rotate in a horizontal plane around a rotation axis extending in the vertical direction.

[0033] A cup 64 is provided to surround the spin chuck 62. The cup 64 can be raised and lowered by a lifting mechanism (not shown). The cup 64 has a cylindrical shape, and the upper part of the cup 64 is inclined so that it approaches the spin chuck 62 as it goes up. However, the inner diameter of the upper end part of the cup 64 is larger than the diameter of the substrate W. During processing of the substrate W, the upper end of the cup 64 is higher than the height position of the substrate W held by the spin chuck 62. Therefore, liquid scattered by centrifugal force from the substrate W rotated by the spin motor 63 is received and recovered by the cup 64. The liquid recovered by the cup 64 is discharged from a drain pipe provided at the bottom of the cup 64. The cup 64 may have a multi-stage structure in which multiple recovery ports are provided for different purposes.

[0034] The discharge nozzle 65 discharges a processing liquid onto the substrate W held on the spin chuck 62. The processing liquid is a term that encompasses various chemical liquids and pure water. Examples of chemical liquids include liquids for etching or liquids for removing particles. Specifically, SC-1 liquid (a mixed solution of ammonium hydroxide, hydrogen peroxide, and pure water), SC-2 liquid (a mixed solution of hydrochloric acid, hydrogen peroxide, and pure water), or hydrofluoric acid are used. The discharge nozzle 65 is moved by a drive mechanism (not shown) between a processing position above the spin chuck 62 and a standby position outside the cup 64. At the processing position, the discharge nozzle 65 discharges a chemical liquid onto the substrate W held on the spin chuck 62, thereby, for example, etching the substrate W. Furthermore, the discharge nozzle 65 discharges pure water onto the substrate W, thereby performing a pure water rinse process on the substrate W.

[0035] A worker who performs maintenance and inspection work on multiple substrate processing apparatuses 50 arranged in a clean room 40 wears smart glasses 10. The smart glasses 10 are a type of wearable device that uses a head-mounted display (HMD). The smart glasses 10 are also devices for realizing AR (Augmented Reality) or MR (Mixed Reality). For example, Microsoft's "HoloLens" (registered trademark) can be used as the smart glasses 10.

[0036] 6 is a perspective view showing the appearance of the smart glasses 10. The smart glasses 10 include a visor 11 and a headband 12. A worker wears the smart glasses 10 by placing the headband 12 on their head. The worker can adjust the length of the headband 12 to fit the size of their head. The headband 12 also includes a power button, a brightness button, a volume button, and the like.

[0037] The visor 11 includes various sensors and a display. The display is a see-through holographic lens. That is, the display can display a three-dimensional image in the worker's field of view using a hologram, and transmits light from real objects in the same way as regular eyeglass lenses. Therefore, a worker wearing the smart glasses 10 can view the displayed three-dimensional image while viewing real objects through the display.

[0038] The sensors of the visor 11 include, for example, multiple visible light cameras that mainly capture images in front of the visor 11, an infrared camera that tracks the worker's line of sight, a depth sensor that measures the distance to an object, and an inertial measurement sensor. The infrared camera tracks the line of sight by measuring the movement of the eyeballs of the wearer of the smart glasses 10. The depth sensor measures the distance to an object using, for example, a Time of Flight (ToF) method. The inertial measurement sensor is composed of an accelerometer, a gyroscope, a magnetometer, etc.

[0039] The smart glasses 10 also have a built-in computer including a CPU, memory, etc. The smart glasses 10 are also provided with a wireless communication mechanism, and the computer in the smart glasses 10 connects to the information communication network 5 using the wireless communication mechanism. The smart glasses 10 are also provided with a microphone, a speaker, a battery, etc.

[0040] 7 is a block diagram showing the functional configuration of the smart glasses 10, the server 70, the work support terminal 80, and the control unit 55 of the substrate processing apparatus 50. The smart glasses 10 include, as hardware elements, an imaging unit 21, a communication unit 22, a display unit 23, a storage unit 24, and a spatial recognition unit 25. The imaging unit 21 includes a visible light camera provided on the visor 11 described above. The imaging unit 21 includes, for example, four visible light cameras that capture images of the front and diagonally forward, and can capture images of the field of view of the worker wearing the smart glasses 10.

[0041] The communication unit 22 includes the wireless communication mechanism of the smart glasses 10 described above. The communication unit 22 transmits and receives data to and from the work support terminal 80 and the server 70 via the information and communication network 5. The communication unit 22 can also transmit and receive data directly to and from the control unit 55 of the substrate processing apparatus 50 if the distance is short. That is, the communication unit 22 can transmit data and commands to the control unit 55 of the substrate processing apparatus 50 directly or via the information and communication network 5.

[0042] The display unit 23 includes the display of the visor 11. The display unit 23 has a holographic processing device and displays a 3D image at a predetermined spatial position using hologram technology. Note that the 3D image displayed by the display unit 23 is not limited to a 3D shape, and may be a 2D image such as a document.

[0043] The storage unit 24 includes a memory and storage installed in the smart glasses 10. The memory and storage provided in the smart glasses 10 are, for example, a dynamic random access memory (DRAM) and a universal flash storage (UFS), respectively. The storage unit 24 stores applications and data used by the computer of the smart glasses 10.

[0044] The spatial recognition unit 25 includes the depth sensor of the smart glasses 10 described above. The depth sensor can measure the distance to an object. The spatial recognition unit 25 can perform spatial mapping by scanning a space using the depth sensor. Specifically, the spatial recognition unit 25 creates a spatial mesh, which is mesh data, by scanning the space. Because the spatial recognition unit 25 uses the depth sensor, it is possible to scan a space even in a dark place.

[0045] The smart glasses 10 also include an image conversion unit 31, a comparison unit 32, a warning issuance unit 33, and a position registration unit 34. The image conversion unit 31, the comparison unit 32, the warning issuance unit 33, and the position registration unit 34 are function processing units realized by the CPU of the smart glasses 10 executing a predetermined processing program. The processing contents of the image conversion unit 31, the comparison unit 32, the warning issuance unit 33, and the position registration unit 34 will be described in further detail below.

[0046] The control unit 55 of the substrate processing apparatus 50 controls the operation of mechanisms provided in the processing unit 52, such as the discharge nozzle 65. The control unit 55 of the substrate processing apparatus 50 can communicate with the communication unit 22 of the smart glasses 10, and can also control the operation of various mechanisms provided in the processing unit 52 in accordance with operation instruction commands transmitted from the smart glasses 10.

[0047] The work support terminal 80 is installed, for example, in a factory of a vendor that manufactures and maintains the substrate processing apparatus 50. The server 70 is installed in a clean room 40 in which a plurality of substrate processing apparatuses 50 are installed. The work support terminal 80 and the server 70 are capable of communicating with the smart glasses 10 via the information and communication network 5. The work support terminal 80 and the server 70 are also capable of communicating with each other via the information and communication network 5.

[0048] The work support terminal 80 and the server 70 are general computer systems. That is, the work support terminal 80 and the server 70 include a CPU which is a circuit that performs various arithmetic processing, a ROM which is a read-only memory that stores basic programs, a RAM which is a readable and writable memory that stores various information, a storage unit (for example, a magnetic disk or SSD) that stores control software and data, and a communication unit that communicates with the information communication network 5.

[0049] The work support terminal 80 is a computer that, for example, allows a work supporter on the vendor side to support the work of a worker in the clean room 40. The work supporter can send various information from the work support terminal 80 to the smart glasses 10 worn by the worker in the clean room 40.

[0050] In the work support system according to the present invention, the server 70 is a computer that executes predetermined processes in response to requests from the smart glasses 10 and the work support terminal 80. The server 70 includes a storage unit 74 with a relatively large capacity. Large-sized data created by the smart glasses 10 and the work support terminal 80 may be stored in the storage unit 74.

[0051] Next, a work assistance method using the work assistance system having the above-described configuration will be described. The work assistance method according to the present invention includes a preparation step of registering normal state data as reference data, and a step of a worker wearing the smart glasses 10 detecting an abnormality in the work area. First, the preparation step will be described.

[0052] 8 is a flowchart showing the steps of advance preparation in the work support method according to the present invention. First, a spatial mesh in a normal state is created for any location in the clean room 40 using the smart glasses 10 (step S11). The locations to be scanned to create the spatial mesh include, for example, the outer wall of the substrate processing apparatus 50 and the floor of the clean room 40. For example, the space recognition unit 25 of the smart glasses 10 scans the space including the outer wall of the substrate processing apparatus 50 and the floor of the clean room 40 to create the spatial mesh, which is mesh data.

[0053] To operate the smart glasses 10 to create a spatial mesh, the worker wearing the smart glasses 10 simply turns on the scan mode. The worker selects the scan mode using a hand gesture from a menu screen displayed as a 3D image on the display unit 23, for example. The hand gesture is captured and detected by the imaging unit 21, and the computer of the smart glasses 10 recognizes from the detection result that the scan mode has been selected and turns on the scan mode. Alternatively, the worker may turn on the scan mode by pressing a predetermined button provided on the smart glasses 10. When the scan mode is turned on, the space recognition unit 25 of the smart glasses 10 starts scanning the space.

[0054] In step S11, the space recognition unit 25 of the smart glasses 10 scans a space including the target location in a normal state and creates a space mesh. The normal state is a safe and constant state in which no foreign objects, openings, etc. are present.

[0055] The space mesh created by the space recognition unit 25 by scanning the space is mesh data represented by a large number of triangular meshes. Various shapes, including curved and flat surfaces, are represented by a set of many connected triangles.

[0056] A fundamental characteristic of spatial meshes is that complex shapes including irregularities are represented by a high density of triangles, while flat shapes are represented by a relatively low density of triangles. Figure 9 shows an example of a spatial mesh created by scanning a space containing irregular foreign objects. In the example of Figure 9, the space surrounding a triangular cone installed in a clean room 40 is scanned. On the other hand, Figure 10 shows an example of a spatial mesh created by scanning a flat surface. In the example of Figure 10, the floor surface of the clean room 40 is scanned.

[0057] As shown in Figure 9, a spatial mesh created by scanning a space containing foreign objects such as triangular cones placed in the clean room 40 is represented by a collection of many relatively small triangles. Therefore, the density of triangles in the spatial mesh is high, and the number of vertices of triangles contained within a unit area is also large (vertices are contained at a high density). In contrast, as shown in Figure 10, a spatial mesh created by scanning the floor, which is a flat surface within the clean room 40, is represented by a collection of few relatively large triangles. Therefore, the density of triangles in the spatial mesh is low, and the number of vertices of triangles contained within a unit area is also small (vertices are contained at a low density).

[0058] Another fundamental characteristic of spatial mesh creation is that the positions of the vertices of triangles are different each time the same object is scanned (i.e., the vertices are not reproducible). As shown in the example of Figure 9, a spatial mesh created by scanning a space containing a foreign object has a high density of triangular vertices, so even if the positions of individual vertices change with each scan, the shape of the foreign object is captured in all spatial meshes from a macroscopic perspective. On the other hand, as shown in the example of Figure 10, a spatial mesh created by scanning a flat surface has a low density of triangular vertices, so if the positions of individual vertices change with each scan, a different spatial mesh will be created with each scan.

[0059] In this embodiment, it is determined whether the spatial mesh created in step S11 satisfies the detection condition (step S12). This process may be performed by the smart glasses 10 or a separate computer (e.g., the work support terminal 80). Specifically, if the density of triangle vertices in the spatial mesh created in step S11 is less than a predetermined threshold, the spatial mesh is determined not to satisfy the detection condition. For example, the spatial mesh ( FIG. 9 ) created by scanning the space containing the substrate processing apparatus 50 and foreign matter in the clean room 40 is determined to satisfy the detection condition because the density of triangle vertices is relatively high. On the other hand, the spatial mesh ( FIG. 10 ) created by scanning the floor of the clean room 40 is determined to not satisfy the detection condition because the density of triangle vertices is relatively low.

[0060] If it is determined that the spatial mesh created in step S11 satisfies the detection conditions, the process proceeds from step S12 to step S13, where the created spatial mesh is registered as a reference spatial mesh. For example, the created spatial mesh may be stored in the storage unit 24 of the smart glasses 10 as the reference spatial mesh. This registers the reference spatial mesh, which is mesh data acquired by scanning the space in the clean room 40, including uneven areas in a normal state. Note that if the amount of data for the reference spatial mesh is large, it may be stored in the storage unit 74 of the server 70, for example.

[0061] On the other hand, if it is determined that the spatial mesh created in step S11 does not satisfy the detection conditions, the process proceeds from step S12 to step S14, where the created spatial mesh is converted into a two-dimensional image.

[0062] The spatial mesh, which is mesh data, can be converted into a two-dimensional image using a 3D object library such as "Open GL" (registered trademark) manufactured by Silicon Graphics, Inc. FIG. 11 is a diagram showing an example of an image obtained by converting a spatial mesh created by scanning the floor of a clean room 40. When the spatial mesh obtained by scanning a flat surface such as a floor is converted into a two-dimensional image, the flat surface is represented by being filled in. In the image of FIG. 11, the floor of the clean room 40 is represented as a filled-in image object AJ1. Note that the hole (not filled in) near the center of the image object AJ1 in the example of FIG. 11 corresponds to the installation location of the substrate processing apparatus 50 on the floor.

[0063] Thereafter, the data of the two-dimensional image obtained by the conversion is registered as reference image data (step S15). For example, the obtained two-dimensional image data may be stored as reference image data in the storage unit 24 of the smart glasses 10. This results in the registration of reference image data, which is two-dimensional image data of a flat surface in a normal state within the clean room 40. Note that if the amount of data of the reference image data is large, it may be stored in the storage unit 74 of the server 70, for example.

[0064] Next, anomaly detection using the smart glasses 10 will be described with reference to Figs. 12 to 14. Fig. 12 is a flowchart showing the procedure for the initial stage of anomaly detection in the work assistance method according to the present invention. Fig. 13 is a flowchart showing the procedure when the target space mesh satisfies the detection conditions. Fig. 14 is a flowchart showing the procedure when the target space mesh does not satisfy the detection conditions.

[0065] First, a worker wearing the smart glasses 10 enters the work area in the clean room 40 at an appropriate time to create a space mesh (step S21). FIG. 15 is a diagram illustrating a situation in which a worker wearing the smart glasses 10 performs work for anomaly detection. A substrate processing apparatus 50 is disposed in a work area 101 in the clean room 40. The worker performing anomaly inspection walks around the work area 101 while wearing the smart glasses 10 and with the scan mode turned on. In the example of FIG. 15 , the worker wearing the smart glasses 10 walks around the periphery of the substrate processing apparatus 50 along the path indicated by the arrow AR15. As the worker walks around the work area 101 with the scan mode of the smart glasses 10 turned on, the space recognition unit 25 sequentially scans the space near the smart glasses 10 to acquire a space mesh (target space mesh). When the worker wearing the smart glasses 10 faces the substrate processing apparatus 50, the space recognition unit 25 scans the space including the substrate processing apparatus 50 to acquire a space mesh. Furthermore, when the worker is looking at the floor surface of the clean room 40, the space recognition unit 25 scans the floor surface, which is a flat surface, to obtain a space mesh.

[0066] Next, it is determined whether the spatial mesh created in step S21 satisfies the detection conditions (step S22). This determination process may be performed, for example, by the smart glasses 10. Specifically, if the density of triangle vertices in the spatial mesh created in step S21 is less than a predetermined threshold, the spatial mesh is determined to not satisfy the detection conditions. The threshold value at this time is preferably the same as the threshold value used in the determination in step S12. That is, it is preferable to align the determination criteria used in the preparatory stage with the determination criteria used when detecting an anomaly. The spatial mesh created in step S21 by scanning the space containing the substrate processing apparatus 50 and foreign matter is determined to satisfy the detection conditions because the density of triangle vertices is relatively high. On the other hand, the spatial mesh created in step S21 by scanning the floor of the clean room 40 is determined to not satisfy the detection conditions because the density of triangle vertices is relatively low.

[0067] If it is determined that the spatial mesh created in step S21 satisfies the detection conditions, the process proceeds to step S23 in Fig. 13 , where the target spatial mesh is compared with the reference spatial mesh. Specifically, the comparison unit 32 of the smart glasses 10 compares the spatial mesh created in step S21 (target spatial mesh) with the reference spatial mesh created and registered in the advance preparation step for the same space. The comparison unit 32 compares the target spatial mesh with the reference spatial mesh and extracts the difference between them.

[0068] Next, the comparison unit 32 determines whether the difference between the target spatial mesh and the reference spatial mesh is equal to or greater than a predetermined threshold (first threshold) (step S24). More specifically, the comparison unit 32 determines whether the difference value obtained by quantifying the difference between the target spatial mesh and the reference spatial mesh is equal to or greater than the threshold.

[0069] If the result of the determination is that the difference between the target space mesh and the reference space mesh is equal to or greater than the threshold value, this means that some change, such as a foreign object, has occurred in the target location when the target space mesh was created compared to the normal state when the reference space mesh was registered. For example, in the example of Figure 15, if a foreign object 105, such as a tool, is placed in the work area 101 after the reference space mesh is registered, the difference between the target space mesh created by scanning the space including the foreign object 105 and the reference space mesh will be equal to or greater than the threshold value.

[0070] If the difference between the target space mesh and the reference space mesh is equal to or greater than the threshold, the process proceeds from step S24 to step S25, where the warning unit 33 of the smart glasses 10 issues a warning. Specifically, for example, the warning unit 33 displays a warning message as a stereoscopic image on the display unit 23. FIG. 17 is a diagram showing an example of a warning message displayed by the warning unit 33. By displaying the warning message, the worker can recognize changes in the surrounding situation and pay attention to those changes, thereby ensuring the worker's safety. In the example of FIG. 15, the worker can pay attention to the presence of the newly placed foreign object 105 and can work safely.

[0071] Next, the position registration unit 34 of the smart glasses 10 registers the position information of the difference between the target space mesh and the reference space mesh in the storage unit 74 of the server 70 (step S26). In the example of FIG. 15 , the position registration unit 34 registers the position information of the foreign object 105, which is the difference, in the server 70. The position information to be registered can be any appropriate information. For example, the coordinate position of the foreign object 105 in the clean room 40 may be registered, or the relative position of the foreign object 105 from the substrate processing apparatus 50 in the work area 101 may be registered.

[0072] On the other hand, if the difference between the target space mesh and the reference space mesh is less than the threshold, it means that no significant changes have occurred in the target location when the target space mesh was created compared to the normal state when the reference space mesh was registered. In this case, the process proceeds from step S24 to step S27, and the worker continues with the desired work. Since no significant changes have occurred in the environment around the worker from the normal state, the worker can work safely.

[0073] Furthermore, if it is determined in step S22 that the spatial mesh created in step S21 does not satisfy the detection conditions, the process proceeds to step S32 in FIG. 14, where the image conversion unit 31 of the smart glasses 10 converts the created target spatial mesh into a two-dimensional image to acquire image data (target image data). The image conversion unit 31 can convert the spatial mesh into a two-dimensional image using a 3D object library, as in step S14 described above. FIG. 16 is a diagram showing an example of an image obtained by converting a spatial mesh created by scanning the floor of the clean room 40 when an abnormality is detected. In the image in FIG. 16, the floor of the clean room 40 is represented as image object AJ2.

[0074] Next, the target image data and the reference image data are compared (step S33). Specifically, the comparison unit 32 of the smart glasses 10 compares the target image data acquired in step S32 with the reference image data created and registered in the advance preparation process for the same space. In the above example, the comparison unit 32 compares the data of the target image shown in FIG. 16 with the data of the reference image shown in FIG. 11. The comparison unit 32 compares the target image data and the reference image data to extract the difference between them.

[0075] Next, the comparison unit 32 determines whether the difference between the target image data and the reference image data is equal to or greater than a predetermined threshold (second threshold) (step S34). More specifically, the comparison unit 32 determines whether the difference value (e.g., the total value of the gradation difference) obtained by quantifying the difference between the target image data and the reference image data is equal to or greater than the threshold.

[0076] If the result of the determination is that the difference between the target image data and the reference image data is greater than or equal to the threshold value, this indicates that some change, such as an opening, occurred in the target location when the target image data was created compared to the normal state when the reference image data was registered. For example, in the example of FIG. 15 , assume that after the reference image data was registered, a grating was removed from the floor in the work area 101, forming an opening 108. A two-dimensional image obtained by converting the spatial mesh created by scanning the floor surface including the opening 108 is shown in FIG. 16 . The comparison unit 32 compares the data of the target image shown in FIG. 16 with the data of the reference image shown in FIG. 11 to extract the difference 109 and determine that the difference 109 is greater than or equal to the threshold value. This difference 109 is caused by the formation of the opening 108 in the floor surface in the normal state. In other words, the formation of the opening 108 compared to the normal state when the reference image data was registered results in the difference 109 between the target image data and the reference image data, which is greater than or equal to the threshold value.

[0077] The processing contents of steps S35 to S37 are generally the same as the processing contents of steps S25 to S27. That is, if the difference between the target image data and the reference image data is equal to or greater than the threshold, the process proceeds from step S34 to step S35, and the warning unit 33 of the smart glasses 10 issues a warning, for example, as shown in FIG. 17 . This allows the worker to recognize changes in the surrounding situation and pay attention to those changes, thereby ensuring the worker's safety. In the example of FIG. 15 , the worker can pay attention to the presence of the newly formed opening 108 in the floor surface and can work safely.

[0078] Next, the position registration unit 34 of the smart glasses 10 registers the position information of the difference part between the target image data and the reference image data in the storage unit 74 of the server 70 (step S36). In the example of Figure 15, the position registration unit 34 registers the position information of the opening 108, which is the difference part, in the server 70.

[0079] On the other hand, if the difference between the target image data and the reference image data is less than the threshold value, this means that no significant changes have occurred in the floor surface of the target location when the target image data was created, compared to the normal state when the reference image data was registered. In this case, the process proceeds from step S34 to step S37, and the worker continues with the desired work. Since no significant changes have occurred in the environment around the worker from the normal state, the worker can work safely.

[0080] For example, if the spatial mesh created for the same target location in the advance preparation process does not satisfy the detection conditions, but the spatial mesh satisfies the detection conditions when an abnormality is detected, it is assumed that some kind of change has occurred in the target location. Therefore, in such a case, the warning issuing unit 33 can issue a warning without performing a comparison. The same applies to the reverse case.

[0081] In this embodiment, the preparation process involves scanning a space including a predetermined location in a normal state to create a spatial mesh. If the density of triangle vertices in the spatial mesh is equal to or greater than a predetermined threshold (i.e., the detection condition is met), the created spatial mesh is registered as a reference spatial mesh. On the other hand, if the density of triangle vertices in the spatial mesh is less than the predetermined threshold (i.e., the detection condition is not met), the created spatial mesh is converted into a two-dimensional image and the acquired two-dimensional image data is registered as reference image data. The density of triangle vertices in the spatial mesh depends on the degree of unevenness in the scanned space. The greater the degree of unevenness, the higher the density of triangle vertices in the spatial mesh; and the closer to a flat surface, the lower the density of triangle vertices. Therefore, for substrate processing apparatuses 50 and the like in a normal state that have a relatively high degree of unevenness within the clean room 40, a reference spatial mesh is registered as mesh data, while for a flat, normal floor surface, the reference image data is registered as two-dimensional image data converted from the mesh data.

[0082] Next, when an abnormality is detected, if the target space mesh satisfies the detection conditions, the target space mesh is compared with the reference space mesh. On the other hand, if the target space mesh does not satisfy the detection conditions, the target space mesh is converted into a two-dimensional image to obtain target image data, and this target image data is compared with the reference image data. If the comparison results in a difference in either data exceeding a threshold, an alarm is issued and the location information of the difference is registered.

[0083] As described above, in this embodiment, for areas with relatively large unevenness, a comparison is made using a spatial mesh, while for flat surfaces such as floors, a comparison is made using image data obtained by converting the spatial mesh into a two-dimensional image. That is, for areas including substrate processing apparatuses 50 with relatively large unevenness, a reference spatial mesh in which a normal state has been pre-registered is compared with a target spatial mesh created when an abnormality is detected. On the other hand, for flat surfaces such as floors, a reference image data in which a normal state has been pre-registered is compared with target image data created when an abnormality is detected.

[0084] As mentioned above, even if the same object is scanned to create a spatial mesh, the positions of the triangle vertices will be different for each scan. In areas with relatively large irregularities, the density of the triangle vertices in the created spatial mesh is high, so even if the positions of the individual vertices change with each scan, the shape of foreign objects, etc. will be captured in all spatial meshes from a macro perspective. Therefore, it is possible to detect foreign objects, etc. by performing a comparison and judgment using the spatial mesh as is.

[0085] In contrast, for flat surfaces such as the floor of a clean room 40, the density of triangle vertices in the generated spatial mesh is low. Therefore, if the positions of individual vertices change with each scan, a completely different spatial mesh will be generated each time the same object is scanned. Therefore, even if the floor remains unchanged from its normal state, the spatial mesh generated during the preparatory process will be different from the spatial mesh generated when an abnormality is detected. As a result, performing a comparison using the spatial mesh may result in false detection of a foreign object, even if there is no change from the normal state. Conversely, even if an opening or other feature is present in the floor, the opening is unlikely to be reflected as a vertex of a triangle in the spatial mesh, and the comparison using the spatial mesh may fail to detect the opening or other feature.

[0086] For this reason, in this embodiment, for flat surfaces such as the floor of the clean room 40, a comparison is made using image data obtained by converting the spatial mesh into a two-dimensional image. Even if the density of the vertices of the triangles in the spatial mesh is low, by converting the spatial mesh into a two-dimensional image, the condition of the floor and other surfaces is accurately reflected in the two-dimensional image, and openings and the like also appear. Therefore, for flat surfaces such as the floor, a comparison is made using image data obtained by converting the spatial mesh into a two-dimensional image, making it possible to accurately detect newly formed openings and the like.

[0087] Furthermore, the spatial recognition unit 25 of the smart glasses 10 that creates the spatial mesh uses a depth sensor to scan the space, making it possible to process even in dark environments. In other words, according to the present embodiment, even in dark environments, foreign objects, openings, etc. can be accurately detected, allowing the worker to recognize changes in the surrounding situation, regardless of whether the target is an area with a large degree of unevenness or a flat surface such as a floor.

[0088] Although the above describes an embodiment of the present invention, various modifications can be made to the present invention without departing from the spirit and scope of the present invention. For example, in the above embodiment, if the created spatial mesh does not satisfy the detection conditions, the spatial mesh is converted into a two-dimensional image and the acquired two-dimensional image data is registered as reference image data in the advance preparation process. However, this is not limited to this. For example, image data acquired by converting a design drawing of the floor surface surrounding the substrate processing apparatus 50 in the clean room 40 into a predetermined data format may be registered as reference image data. In other words, the determination in step S12 is not a required process, and the reference image data may be two-dimensional image data of a flat surface in a normal state, regardless of the format in which it was created.

[0089] In addition to or instead of displaying a warning message on the display unit 23, the warning unit 36 ​​may also emit a warning sound from the speaker of the smart glasses 10.

[0090] Furthermore, in the above embodiment, the worker uses smart glasses 10, but this is not limited thereto. Instead of the smart glasses 10, a portable terminal such as a tablet terminal or a smartphone may be used. That is, any portable terminal equipped with an imaging unit, a display unit, a communication unit, etc. is sufficient. However, since using a tablet terminal or the like would occupy the worker's hands while holding it, it is preferable to use a wearable terminal such as the smart glasses 10.

[0091] Furthermore, the substrate processing apparatus 50 installed in the clean room 40 is not limited to a substrate cleaning apparatus, but may be any apparatus that performs a predetermined process on a substrate, such as a heat treatment apparatus, an exposure apparatus, a coating and developing apparatus, a measuring apparatus, or an inspection apparatus. When the substrate processing apparatus 50 is a substrate cleaning apparatus, it may be a single-wafer type cleaning apparatus that cleans substrates one by one, or a batch type cleaning apparatus that cleans multiple substrates at once.

[0092] Furthermore, the work support technology according to the present invention is not limited to substrate processing equipment, but may be applied to any industrial equipment that performs any type of processing, such as a printing processing equipment, a film forming equipment, a medical equipment, and an appearance inspection equipment.

[0093] 5 Information and communication network 10 Smart glasses 21 Imaging unit 22 Communication unit 23 Display unit 24 Memory unit 25 Spatial recognition unit 31 Image conversion unit 32 Comparison unit 33 Warning issuance unit 34 Position registration unit 40 Clean room 50 Substrate processing apparatus 52 Processing unit 55 Control unit 60 Processing chamber 61 Rotation holding unit 65 Discharge nozzle 70 Server 80 Work support terminal 101 Work area 105 Foreign matter 108 Opening 109 Difference AJ1, AJ2 Image object W Substrate

Claims

1. A work support method for performing specified work on industrial equipment, comprising: a pre-registration step of registering a reference spatial mesh, which is mesh data obtained by scanning a space including a specified location in a normal state using a portable terminal equipped with a display unit, a communication unit, and a spatial recognition unit, and reference image data, which is two-dimensional image data of a flat surface in a normal state; a spatial recognition step of scanning the space including the specified location and / or the flat surface using the portable terminal at a specified timing to obtain a target spatial mesh; an image conversion step of converting the target spatial mesh into a two-dimensional image to obtain target image data when the target spatial mesh does not satisfy a detection condition; a comparison step of comparing the target spatial mesh with the reference spatial mesh if the target spatial mesh satisfies the detection condition, and comparing the target image data with the reference image data if the target spatial mesh does not satisfy the detection condition; and a warning step of issuing a warning if a difference between the target spatial mesh and the reference spatial mesh is equal to or greater than a first threshold, or if a difference between the target image data and the reference image data is equal to or greater than a second threshold.

2. A work support method according to claim 1, further comprising a position registration step of registering position information of the difference portion when a warning is issued in the warning issuing step.

3. A work support method according to claim 1, wherein the target space mesh is determined not to satisfy the detection condition if the density of vertices in the target space mesh is less than a predetermined threshold.

4. A work support method according to claim 1, wherein the reference image data is obtained by converting mesh data acquired by scanning a flat surface in a normal state using the mobile terminal into two-dimensional image data.

5. A work support method according to claim 1, wherein the industrial equipment is a substrate processing apparatus that performs a predetermined process on a substrate.

6. A work support method according to any one of claims 1 to 5, wherein the mobile terminal is a pair of smart glasses.

7. A work support system for performing specified work on industrial equipment using a portable terminal, comprising: a portable terminal equipped with a display unit and a communication unit; a memory unit that stores a reference spatial mesh, which is mesh data obtained by scanning a space including a specified location in a normal state using the portable terminal, and reference image data, which is two-dimensional image data of a flat surface in a normal state; a space recognition unit that scans the space including the specified location and / or the flat surface using the portable terminal at a specified timing to obtain a target spatial mesh; an image conversion unit that converts the target spatial mesh into a two-dimensional image to obtain target image data when the target spatial mesh does not satisfy a detection condition; a comparison unit that compares the target spatial mesh with the reference spatial mesh if the target spatial mesh satisfies the detection condition, and compares the target image data with the reference image data if the target spatial mesh does not satisfy the detection condition; and a warning issuance unit that issues a warning if a difference between the target spatial mesh and the reference spatial mesh is equal to or greater than a first threshold, or if a difference between the target image data and the reference image data is equal to or greater than a second threshold.

8. A work support system according to claim 7, further comprising a position registration unit that registers position information of the difference portion when the warning issuing unit issues a warning.

9. A work support system according to claim 7, wherein the target space mesh is determined not to satisfy the detection condition if the density of vertices in the target space mesh is less than a predetermined threshold.

10. A work support system according to claim 7, wherein the reference image data is obtained by converting mesh data acquired by scanning a flat surface in a normal state using the mobile terminal into two-dimensional image data.

11. A work support system according to claim 7, wherein the industrial equipment is a substrate processing apparatus that performs a predetermined process on a substrate.

12. A work support system according to any one of claims 7 to 11, wherein the mobile terminal is a pair of smart glasses.

Images