Measurement device, measurement data utilization system, and measurement data acquisition method

Abstract

This invention relates to a measurement device that acquires measurement data obtained by measuring maintenance work by a worker by using a 3D sensor installed in a space in which maintenance work for a semiconductor manufacturing device is performed. The measurement device includes: an object identification unit 223 that identifies an object included in a raw data image acquired from a 3D sensor; a coordinate conversion unit 224 that acquires the position in space of the identified identification object; a work determination unit 225 that determines a work process being performed by a worker, on the basis of the movement of one or more identification objects; a region setting unit 226 that sets, in the raw data image, a region that can be output or cannot be output according to the work process; and a data limiting unit 227 that generates limited data in which a region of the raw data image that the region setting unit has set as a region that cannot be output is made invisible.

Description

Measurement device, measurement data utilization system, and measurement data acquisition method

[0001] The present invention relates to a measurement device, a measurement data utilization system, and a measurement data acquisition method.

[0002] To improve the productivity of semiconductor devices, semiconductor manufacturing equipment must not only improve its performance but also its uptime. Because much of the maintenance work for semiconductor manufacturing equipment involves manual labor, it must be performed efficiently and appropriately. To prevent rework during maintenance work, a system that accumulates and analyzes logs of maintenance work performed by workers is considered effective. In this case, to quantitatively characterize maintenance work, logs must also be accumulated of how each worker moves with respect to the equipment and parts. For example, the hand movements when wiping a part while holding it, the relative positions of both hands when installing a part, the way hands are handled when disassembling and installing large parts, and the relationship between workers working inside the equipment and those preparing in the surrounding area.

[0003] Patent Literature 1 relates to a technology for restricting images so that areas other than the area of ​​a monitored object are concealed. When remotely supporting work such as maintenance on a monitored object, an operator takes an image of the monitored object and sends it to a remote terminal device, and an instructor checks the image on the terminal device to support the worker's work. The technology of Patent Literature 1 conceals confidential information around the monitored object that the instructor does not want to see, enabling remote support.

[0004] Japanese Patent Application Laid-Open No. 2022-184481

[0005] Distance measurement technology has advanced in recent years, and by using a 3D sensor, it is now possible to measure the spatial movement of an operator relative to a device at the actual size level.

[0006] While Patent Document 1 cites a substrate processing apparatus as an example of a monitored object, maintenance work on such an apparatus is often performed in a user's factory, where there is a large amount of other companies' equipment and other confidential information unrelated to the maintenance work. Furthermore, maintenance work often involves removing parts from the equipment being maintained and performing the work in a location other than the equipment's installation location. For this reason, simply monitoring the monitored object in a fixed manner, as in Patent Document 1, is insufficient.

[0007] In general, maintenance work involves frequent changes in work location, so it is necessary to acquire measurement data that tracks the movements of parts or workers. However, in the process, there is a risk that information that should not be displayed may be unintentionally acquired, which could lead to information leaks.

[0008] Furthermore, the amount of data from the 3D sensor is large, and even in areas where data acquisition is possible, continuing to acquire data when no change remains for a long period of time is not desirable from the perspective of data volume.

[0009] A measurement device that is one embodiment of the present invention is a measurement device that acquires measurement data obtained by measuring maintenance work performed by workers using a first 3D sensor installed in a space where maintenance work on semiconductor manufacturing equipment is performed, and includes an object identification unit that identifies objects included in an image of raw data acquired from the first 3D sensor, a coordinate conversion unit that acquires the spatial position of the identified object identified by the object identification unit, a work judgment unit that determines the work process being performed by the worker based on the movement of one or more identified objects, an area setting unit that sets areas of the image of raw data that can be output / cannot be output in accordance with the work process determined by the work judgment unit, and a data restriction unit that generates restricted data that makes invisible areas of the image of raw data that the area setting unit has determined cannot be output.

[0010] The present invention provides a measurement device, a measurement data utilization system, and a measurement data acquisition method that can acquire measurement data with a low risk of information leakage and a small data volume. Other objects and novel features will become apparent from the description of this specification and the accompanying drawings.

[0011] 1 is a diagram showing a space where workers perform maintenance work. FIG. 2 is an example of the hardware configuration of an information processing device. FIG. 3 is a functional block diagram of the measurement function of a data processing device. FIG. 4 is a flow of maintenance work on a gate valve. FIG. 5 is a diagram showing data read into a data storage unit. FIG. 6 is an example of an ROI set in a virtual space. FIG. 7 is an example of the data configuration of a work process table. FIG. 8 is an example of a gate valve dismantling process. FIG. 9 is a diagram showing a method of reducing an ROI based on a gate valve model. FIG. 10 is an example of an upper chamber assembly process. FIG. 11 is an example of an upper chamber dismantling process. FIG. 12 is an example of a measurement data utilization system.

[0012] The present embodiment will be described below with reference to the drawings.

[0013] FIG. 1 shows the space in which a worker performs maintenance work on semiconductor manufacturing equipment in Example 1. The real space 100 shown on the left side of FIG. 1 is a top view that schematically illustrates the space in which the worker performs maintenance work (only the worker is shown as a side view for convenience of drawing). The semiconductor manufacturing equipment 40 is the target of the maintenance work, and an aisle 10 and a work area 15 are provided around it. The aisle and work area are each divided into multiple sections, and a reference symbol will be used to indicate an aisle or work area in a specific section. The semiconductor manufacturing equipment 40 is an example of an etcher, and the figure shows a base plate 41, a lid 42, and a valve box 43.

[0014] During maintenance work, for example, a worker 30 disassembles a gate valve of semiconductor manufacturing equipment 40. Specifically, the worker 30 removes the gate valve 44 from the valve box 43 and moves to one of the work areas (work area 15b in this example) depending on the situation to perform maintenance work on the gate valve 44. To measure the worker's work using the 3D sensor 21, one or more 3D sensors 21 are fixedly installed in the real space 100 so as to observe the space where the worker may be working. When multiple 3D sensors are installed, it is desirable to install them so that the observation ranges of adjacent 3D sensors overlap. Note that specific 3D sensors are indicated by symbols. In this example, 3D sensor 21a is positioned so that the work in work area 15b is included in the field of view 23a, and 3D sensor 21b is positioned so that the work on the semiconductor manufacturing equipment 40 is included in the field of view 23b. The 3D sensor 21 is connected to the data processing device 20 via a cable 22. In this example, the worker 30 wears an HMD (Head Mounted Display) 31. The HMD 31 also functions as a 3D sensor.

[0015] The data processing device 20 receives measurement data (raw data) from the 3D sensor 21 or the HMD 31 and has a function of limiting the visible area of ​​the image of the measurement data according to the work process being performed by the worker. Hereinafter, measurement data with a limited visible area will be referred to as limited data. Note that the data processing device 20 does not need to be located within the real space 100 and may be located outside the real space 100.

[0016] The data processing device 20 holds a spatial model corresponding to the real space 100 for generating limited data. The right side of FIG. 1 shows a virtual space 110 represented by the spatial model. In the virtual space 110, the aisle 10 corresponds to the aisle model 50, the work area 15 corresponds to the work area model 55, the base plate 41, the lid 42, the valve box 43, and the gate valve 44 correspond to the base plate model 81, the lid model 82, the valve box model 83, and the gate valve model 84, respectively, and the worker 30 corresponds to the worker model 70. In the example of the virtual space 110 shown in FIG. 1 , the aisle 10 and the work area 15 are modeled according to the layout, but they may be simplified, or the sections such as the aisle model 50 and the work area model 55 may be omitted. A coordinate reference 51 indicates the origin of the world coordinate system of the virtual space. To associate the virtual space 110 with the real space 100, a marker 11 is placed at a position in the real space 100 corresponding to the coordinate reference 51 of the virtual space 110. The marker 11 can be, for example, a two-dimensional barcode or an easily identifiable pattern displayed on the floor surface, or tape attached thereto.

[0017] The size of the real space 100 in which the spatial model is created may be determined depending on the maintenance work to be measured. In the above example, even if the entire building is not modeled, it is sufficient that the area in which workers may work is modeled as the virtual space 110.

[0018] FIG. 2A shows an example of the hardware configuration of the data processing device 20. The data processing device 20 is realized by an information processing device including, as shown in FIG. 2A , a processor (CPU) 201, a memory 202, a storage device 203, an input interface (I / F) 204, an output I / F 205, a communication I / F 206, an input / output port 207, and a bus 208 as its main components. The processor 201 functions as a functional unit (functional block) that provides a predetermined function by executing processing in accordance with a program loaded in the memory 202. The storage device 203 stores data and programs used by the functional unit. The storage device 203 may be a non-volatile storage medium such as a hard disk drive (HDD) or a solid state drive (SSD). The input I / F 204 is an interface for connecting an input device 209 such as a keyboard or pointing device, and the output I / F 205 is an interface for connecting a display device 210. The communication I / F 206 enables communication with other information processing devices via a network. The input / output port 207 is connected to the cable 22 of the 3D sensor 21, and receives measurement data from the 3D sensor 21. These are connected to each other via a bus 208 so that they can communicate with each other.

[0019] The data processing device 20 does not have to be realized by a single information processing device, but may be realized by multiple information processing devices. Furthermore, some or all of the functions of the data processing device 20 may be realized as an application on a cloud.

[0020] FIG. 2B shows a functional block diagram of the measurement function of the data processing device 20. This function includes an initial setting unit 221, a data acquisition unit 222, an object identification unit 223, a coordinate conversion unit 224, a work judgment unit 225, a region setting unit 226, a data limitation unit 227, a data output unit 228, and a data storage unit 229 that stores data necessary for the measurement function. Details of these functions will be described below with reference to the maintenance work on the gate valve shown in FIG. 3 as an example. This maintenance work includes gate valve dismantling work performed near the semiconductor manufacturing equipment 40 and gate valve wiping work in the work area. The measurement function of the data processing device 20 according to this embodiment can flexibly limit the visible area of ​​raw data images. This allows measurement data to be acquired during maintenance work without revealing confidential information within the building where the semiconductor manufacturing equipment 40 is installed, and the data can be shared with third parties without risk of confidential information being leaked.

[0021] [Initial Setting Unit 221] The initial setting unit 221 activates the 3D sensor 21. Here, an example is shown in which a LiDAR (Light Detection and Ranging) camera is used as the 3D sensor 21. The LiDAR camera can output depth image data indicating the distance to an object in addition to RGB (color) image data. However, the 3D sensor 21 may be any sensor capable of obtaining measurement data that can be converted into three-dimensional data (volume data such as point cloud data, mesh data, or voxels). In addition to the LiDAR camera, the 3D sensor 21 may be a stereo camera equipped with two CMOS image sensors, a structured light sensor that combines a projection pattern light-emitting element and an image sensor, or a sensor device that combines a distance sensor and an RGB camera and adjusts the relationship between pixels. Furthermore, a sensor equipped with a function for estimating distance information for each pixel from an RGB image using machine learning or the like may be used.

[0022] The initial setting unit 221 also reads and initializes data for executing the measurement function. FIG. 4A shows the data read into the data storage unit 229. CAD data 301 is CAD data for the equipment to be maintained, in this example, semiconductor manufacturing equipment 40. Space model 302 is a model of the real space where the maintenance work is performed by the worker described with reference to FIG. 1 . A region (hereinafter referred to as ROI (Region of Interest)) with attribute information regarding whether or not raw data images can be output is set in space model 302. FIG. 4B shows an example of an ROI set in the virtual space represented by space model 302. The attribute information of the ROI is divided into three categories: "output allowed," "output prohibited," and "output prohibited." For example, ROI 321 is a region with an "output prohibited" attribute, and measurement data output is always prohibited. ROIs 322, 323, and 324 are regions with "output allowed" / "output prohibited" attributes, and measurement data output can be permitted at the discretion of the data processing device 20. In the initial state, all ROIs are set to "output not possible" or "output prohibited."

[0023] Here, we will explain how to set an ROI for a space model. The ROI does not need to be divided according to the corridor 10 or work area 15 described in FIG. 1 . For example, in FIG. 4B , a portion of the work area model 55a is designated as "output prohibited" by the ROI 321. However, if a metal surface or mirror is installed on the wall of the work area 15a corresponding to the work area model 55a, unintended information may be reflected on the metal surface or mirror and included in the measurement data, resulting in leakage. To prevent this situation, an ROI 321 with an "output prohibited" attribute can be set in a portion of the work area model 55a corresponding to the wall of the work area 15a. Furthermore, while the work area model 55b is designated as a single ROI (ROI 322), it may be divided into, for example, multiple ROIs. The ROI can be set according to the actual usage situation of the real space. It is sufficient for the ROI with the "output enabled" / "output disabled" attribute to be set so as to cover the entire field of view of the 3D sensor installed in the real space, but the ROI with the "output disabled" attribute is set to cover the entire virtual space regardless of the placement of the 3D sensor installed in the real space, in order to be able to restrict measurement data from a moving 3D sensor such as the HMD 31.

[0024] The work process table 303 stores information about the work process of the maintenance work to be measured. FIG. 4C shows an example of the data configuration of the work process table 303. An ID identifying the work process is registered in the ID 331. The name of the work process is registered in the work process name 332. The judgment criteria 333 store judgment criteria for determining whether the work process is being performed based on measurement data from a 3D sensor, etc. Candidate work area 334 stores candidate work areas where the work process is to be performed. There may be one candidate work area, or multiple candidate work areas. Associated ROI 335 stores ROIs corresponding to the candidate work area. Attributes 336 store attributes of the associated ROI. As described above, in the initial setting by the initial setting unit 221, ROIs with the "output allowed" / "output disabled" attribute are registered as "output disabled." The limitation mode 337 stores the mode for limiting raw data images. There are various ways of limiting the visibility, but one possible way is to blur or black out the image area to be made invisible.

[0025] [Data Acquisition Unit 222] The data acquisition unit 222 processes the measurement data (raw data) received from the 3D sensor 21 so that it can be processed by the object identification unit 223. Specifically, the data acquisition unit 222 matches the angles of view of the RGB image data and depth image data obtained from the 3D sensor 21 and acquires camera parameters for identifying the position of an object included in the image data in a world coordinate system with the coordinate reference 51 as the origin. The camera parameters include in-camera parameters and out-of-camera parameters. The in-camera parameters refer to parameters for camera calibration, including lens distortion in the 3D sensor 21 and the positional relationship between the RGB sensor and the distance sensor, while the out-of-camera parameters refer to parameters indicating the position and orientation of the 3D sensor 21 in the world coordinate system. If the in-camera parameters are stored in the built-in memory of the 3D sensor 21, the in-camera parameters are acquired from the 3D sensor 21. If not, the in-camera parameters are generated in advance by performing calibration. On the other hand, the out-of-camera parameters may be set when the 3D sensor 21 is installed, or may be set based on the installation position and orientation estimated using a self-position estimation algorithm such as SLAM (Simultaneous Localization and Mapping) processing.

[0026] The data processing in the data acquisition unit 222 depends on the data processing method in the subsequent object identification unit 223. When the object identification in the object identification unit 223 is 2D processing, a process of matching the angles of view of the RGB image data and the depth image data is performed. On the other hand, when the object identification in the object identification unit 223 is 3D processing, the RGB image data and the depth image data with the matched angles of view are converted into 3D data (3D point cloud data) of a virtual space using camera parameters.

[0027] [Object Identification Unit 223] The object identification unit 223 performs object identification processing using the image data or 3D data processed by the data acquisition unit 222 and camera parameters. The object identification processing may be performed as either 2D processing or 3D processing, and known techniques can be used. The object identification unit 223 extracts objects identified by the object identification processing (identified objects) as 3D data. Identification information is assigned to each extracted identified object.

[0028] [Coordinate Transformation Unit 224] The coordinate transformation unit 224 transforms the 3D data of the extracted identified object into the world coordinate system using the camera parameters. Through the above process, the data processing device 20 can grasp the identified object included in the image of the raw data acquired by the 3D sensor 21, as well as its position in the virtual space.

[0029] [Work determination unit 225] By continuously performing the above processing on the measurement data from the 3D sensor 21, the data processing device 20 can grasp the movement of the identified object in the virtual space. Here, the movement of the identified object includes not only the movement of the identified object in the virtual space but also changes in the shape of the identified object itself. If the movement of one or more identified objects in the virtual space satisfies the determination criteria 333 in the work process table 303, the work determination unit 225 detects the work process as such.

[0030] 5 shows an example of a gate valve dismantling process. The process of gate valve 44 being removed from valve box 43 in real space is represented in virtual space as a gate valve model 84 moving away from valve box model 83. For example, the movement of gate valve model 84 can be determined from the trajectory of the minimum distance between valve box model 83 and gate valve model 84, and the removal of the gate valve can be determined. Note that gate valve model 84 is an identified object extracted from image data from 3D sensor 21. However, because valve box model 83 is a fixed device, its shape can be determined in advance based on CAD data 301 without being extracted as an identified object.

[0031] The data processing device 20 tracks the gate valve removed during dismantling of the gate valve. When multiple 3D sensors 21 are arranged in real space, the gate valve can be tracked by tracing its trajectory in virtual space as it continues to be identified as an identified object by the object identification unit 223. Alternatively, when the worker 30 is wearing a sensor such as an HMD 31 that can identify the position of the worker 30, the gate valve may be tracked based on the position information of the worker 30 performing maintenance.

[0032] The work determination unit 225 traces the movement of the gate valve (identification object) in the virtual space and compares it with the determination criteria 333 in the work process table 303. When the maintenance work includes a series of work processes from dismantling the gate valve to reassembling it into the valve box (see FIG. 4C (ID: P01-1 to 3)), if it is confirmed that the gate valve model 84 has moved to any of the ROIs registered as related ROIs 335, the work determination unit 225 detects that the gate valve has been wiped.

[0033] [Area Setting Unit 226] Here, an example will be described in which an area for outputting / not outputting a raw data image related to a gate valve wiping operation is set. The area setting unit 226 sets an area for outputting / not outputting a raw data image acquired by the 3D sensor 21. In this example, the attribute of the ROI to which the gate valve model 84 has moved is switched from the initial state of not outputting to outputting.

[0034] If there is a range in the image of the raw data in which an ROI with an output prohibition attribute is reflected, that range is always regarded as an output prohibition area. Because the 3D sensor 21 is fixed and the range in which an ROI with an output prohibition attribute is reflected is also fixed within the image data, it is possible to determine in advance the output prohibition area based on the ROI with an output prohibition attribute.

[0035] When an identified object moves into an ROI with an "output enabled" / "output disabled" attribute, as in this example, the simplest method is to switch the attribute from "output disabled" to "output enabled" when the identified object moves, and then switch it back from "output enabled" to "output disabled" when the identified object leaves. Additionally, additional conditions may be added to the attribute switching of an ROI with an "output enabled" / "output disabled" attribute. For example, the attribute may be switched to "output enabled" after a certain time (e.g., about 5 seconds) has elapsed since entering the ROI. This eliminates the need for processing ROIs that are merely passing through temporarily and not for work. Furthermore, a worker's hand may be recognized as an identified object, and when it is determined that the worker's hand has left the gate valve, the attribute of the ROI may be switched from "output enabled" to "output disabled."

[0036] In addition to switching the attributes of the ROI, the range of the ROI may also be limited. Depending on the operation of the real space, for example, if another worker is performing a different task in the same work area, it may be impossible to output images of that task. For this reason, the ROI may be adjustable for reduction. FIG. 6 illustrates a method for reducing the ROI based on the gate valve model 84. In this example, a bounding box 350 of the gate valve model 84 is calculated, and the ROI is enlarged by a predetermined factor (e.g., 1.5 times) based on the center of gravity of the bounding box 350 so that the worker's hands are included within the bounding box. Here, the reduced ROI of ROI 322 is shown as ROI 322'. Alternatively, the ROI may be adjustable by the worker. The ROI can be reduced by displaying the ROI in AR on the HMD 31 worn by the worker 30 and allowing the worker 30 to specify the output range with a hand motion.

[0037] In addition, the area setting unit 226 is provided with an editing screen that presents the results of the output possible / not possible setting and the area that is visibly output to the user (information manager) so that the user can confirm the output possible / not possible setting made by the algorithm of the area setting unit 226 or the judgment of the worker, and adjusts the output area as necessary.

[0038] Furthermore, it is desirable for the area setting unit 226 to output a log indicating the algorithm of the measurement device applied to the setting of whether or not to allow output as described above and the editing process as a limitation report. For example, the limitation report includes information about the process of limiting the measurement data. Specifically, the report includes information about areas that have been set in advance as not allowing data output, information about objects (identified objects) identified as the basis for determining the work process, and the location of the objects or the workers holding the objects. In addition, an image of the raw data and an image of the limited data may be displayed for comparison.

[0039] [Data Limiting Unit 227] The data limiting unit 227 generates limited data by making invisible a portion of the image of the raw data that has been determined by the area setting unit 226 to be unsuitable for output. According to the limiting mode 337 in the work process table 303, the unsuitable and prohibited-from-output areas are processed by, for example, blacking out, blurring, or deforming (a patch may be prepared in advance, or a simple graphic may be substituted). Alternatively, the unsuitable and prohibited-from-output areas may be blurred, since they may be outputtable in some cases, while the prohibited-from-output areas may be blacked out. However, if there are many blacked-out areas, spatial recognition may be impaired. In such cases, it may be advisable to blur or deform the prohibited-from-output areas as well.

[0040] [Data Output Unit 228] The data output unit 228 outputs the limited data generated by the data limiting unit 227.

[0041] While Example 1 shows an example in which measurements are made using a fixed 3D sensor, Example 2 shows an example in which measurements are made using a movable 3D sensor in addition to the fixed 3D sensor. There are no particular limitations on the movable 3D sensor, but here, an example is shown in which an HMD 31 worn by a worker 30 is used as the 3D sensor. Unlike a fixed 3D sensor, a movable 3D sensor does not allow an ROI to be determined in advance in the virtual space.

[0042] An example of the assembly process of the upper chamber taken up in Example 2 will be described using Figure 7. Here, semiconductor manufacturing equipment 40 is shown schematically as comprising a main body 401, an upper chamber 402, and a lower chamber 403. As shown in the top view (schematic diagram) of upper chamber 402 in the lower right of Figure 7, upper chamber 402 has a sample stage 404 provided in its center and a sample stage base 405 that supports sample stage 404. Specifically, the maintenance work involves disassembling semiconductor manufacturing equipment 40, pulling out upper chamber 402, and attaching a cover to sample stage base 405.

[0043] As shown on the left side of FIG. 7 , the semiconductor manufacturing equipment 40 is typically integrated with a main body 401, an upper chamber 402, and a lower chamber 403, and in this state, it is included in the ROI 411, which is the ROI of the fixed 3D sensor. To perform the above-mentioned maintenance work, the main body 401 is raised, while the upper chamber 402 is pulled out to the side. For example, the upper chamber 402 is pulled out to the aisle 10c (see FIG. 1 ), and a fixed 3D sensor that includes the aisle 10c in its field of view is not installed, so no ROI is set, and measurements are taken using the HMD 31 worn by the operator 30 as a 3D sensor. The processing in this case will be described. Below, the following description will focus on the differences from Example 1.

[0044] The HMD 31 acquires RGB image data and depth data as measurement data (raw data) in the same manner as the 3D sensor 21 described in Example 1. The measurement data of the HMD 31 is also processed by the data acquisition unit 222, the object identification unit 223, and the coordinate conversion unit 224 as described in Example 1, and an identified object is extracted and its position in the world coordinate system is specified.

[0045] [Work determination unit 225] The movement of a characteristic identified object is defined in the determination criteria 333 of the work process table 303, and the work determination unit 225 determines whether the work is assembling the upper chamber 402. For example, a detection condition is that the worker is holding the cover and air spray to be installed (see FIG. 4C (ID: P02-1)). The work determination unit 225 detects the cover, air spray, and hand, and if the cover and air spray are each within a predetermined distance from the center of gravity of the hand, it determines that the work is assembling the upper chamber 402.

[0046] [Region setting unit 226] In response to the determination by the work determination unit 225, the region setting unit 226 sets an ROI with an output attribute based on the identified object of the upper chamber 402 for the measurement data of the HMD 31. As in Fig. 6, the region setting unit 226 calculates a bounding box for the upper chamber model, and sets an ROI 412 with an output attribute as an area enlarged by a predetermined factor (for example, 1.5 times) based on the center of gravity of the bounding box so that the worker's hands are included in the bounding box.

[0047] This allows measurement data to be acquired using a 3D sensor worn by a worker even in areas where no fixed 3D sensor is installed, and by limiting the output video data to only the surroundings of the work target, it is possible to prevent unintended information leakage. Note that when measurement data is acquired from the HMD 31 within an ROI with an output attribute, the entire area of ​​the image of the measurement data may be set as the ROI.

[0048] The third embodiment is also an example in which a movable 3D sensor is used, and here, an example in which a 3D sensor mounted on a cart used to transport large parts is shown.

[0049] 8, an example of the dismantling process of the upper chamber taken up in Example 3 will be described. A worker separates the upper chamber 402 from the lower chamber 403 and loads the upper chamber 402 onto the dolly 420.

[0050] The 3D sensor mounted on the cart 420 acquires RGB image data and depth data as measurement data (raw data) in the same manner as the 3D sensor 21 described in Example 1. The measurement data of the 3D sensor mounted on the cart 420 is also subjected to the processing described in Example 1 by the data acquisition unit 222, object identification unit 223, and coordinate conversion unit 224, and an identified object is extracted and its position in the world coordinate system is specified.

[0051] [Work determination unit 225] The movement of the characteristic identified object is defined in the determination criteria 333 of the work process table 303, and the work determination unit 225 determines whether the work is dismantling the upper chamber 402. A first condition is that the 3D sensor 21b detects that the upper chamber 402 and the lower chamber 403 have been separated. It can be determined that the two have been separated when the distance between the center of gravity of the bounding box of the upper chamber 402 and the center of gravity of the bounding box of the lower chamber 403 is equal to or greater than a predetermined distance. A second condition is that the 3D sensor mounted on the dolly 420 detects that the upper chamber 402 has been placed on the dolly 420. For example, it can be determined that the upper chamber 402 has been placed on the dolly 420 when the distance between the top surface of the dolly 420 and the center of gravity of the bounding box of the upper chamber 402 is less than a predetermined distance. When these two detection conditions are met, it is determined that the work is dismantling the upper chamber 402.

[0052] [Region setting unit 226] In this example, the field of view of the 3D sensor mounted on the cart 420 is fixed to the upper surface of the cart 420. In this case, based on the determination of the work determination unit 225, an ROI 413 with an output attribute is set for the entire field of view of the 3D sensor mounted on the cart 420. Alternatively, an ROI with an output attribute may be set based on the bounding box of an object mounted on the upper surface of the cart 420.

[0053] This means that even in areas where fixed 3D sensors are not installed, measurement data can be obtained using 3D sensors installed on a mobile body, and by limiting the output video data to only the surrounding area related to the work target, unintentional information leakage can be prevented.

[0054] FIG. 9 shows an example configuration of a measurement data utilization system that utilizes measurement data about maintenance work on semiconductor manufacturing equipment 40 collected by the measurement device (data processing device 20) described in Examples 1 to 3. This system is operated by a vendor, which is the manufacturer of semiconductor manufacturing equipment 40, and a user, which is the user of semiconductor manufacturing equipment 40. In FIG. 9, the left side shows an information processing device owned by the user site, and the right side shows an information processing device owned by the vendor site. The user site and the vendor site are capable of communicating via a network. Of these information processing devices, devices that can access raw data measured by a 3D sensor about maintenance work on semiconductor manufacturing equipment 40 are shown surrounded by a solid black frame, and devices that can access limited data, which is measurement data with some areas concealed by the measurement device 20, are shown surrounded by a dashed black frame.

[0055] The raw data, limited data, and limited reports collected by the measurement device 20 are stored in a data sharing server 501. A user with access authority to the confidential information can check both the raw data and the limited data while referring to the limited report on a user terminal 502. Meanwhile, a vendor's representative staying at the user site can also check the measurement data from a vendor room server 504 via an authentication server 503, but the measurement data that can be accessed is limited to the limited data.

[0056] The restricted data can be accessed via the network, even at the vendor site, via the authentication server 511. For example, the vendor can use the analysis server 512 to analyze maintenance work logs based on the restricted data and report the analysis results to the user. This makes it possible to train workers remotely.

[0057] The above embodiments and modifications have been described in detail to make the present invention easier to understand, and are not necessarily limited to those including all of the described configurations. Furthermore, it is possible to replace part of the configuration of one embodiment or modification with the configuration of another embodiment or modification, and it is also possible to add the configuration of another embodiment or modification to the configuration of one embodiment or modification. Furthermore, it is possible to add, delete, or replace part of the configuration of each embodiment or modification with other configurations.

[0058] 10: Passageway, 11: Marker, 15: Work area, 21: 3D sensor, 22: Cable, 23: Field of view, 30: Worker, 31: HMD, 40: Semiconductor manufacturing equipment, 41: Base plate, 42: Lid, 43: Valve box, 44: Gate valve, 50: Passageway model, 51: Coordinate reference, 55: Work area model, 70: Worker model, 81: Base plate model, 82: Lid model, 83: Valve box model, 84: Gate valve model, 100: Real space, 110: Virtual space, 201: Processor (CPU), 202: Memory, 203: Storage device, 204: Input interface, 205: Output interface, 206: Communication interface, 207: Input / output port, 208: Bus, 209: Input device, 210: Display device, 221: Initial setting unit, 222: Data acquisition acquisition unit, 223: object identification unit, 224: coordinate conversion unit, 225: work judgment unit, 226: area setting unit, 227: data limitation unit, 228: data output unit, 229: data storage unit, 301: CAD data, 302: space model, 303: work process table, 321, 322, 322', 323, 324: ROI, 331: ID, 332: work process name, 333: judgment criteria, 334: work area candidate, 3 35: Related ROI, 336: Attributes, 337: Limitation aspect, 350: Bounding box, 401...Main body, 402...Upper chamber, 403...Lower chamber, 404...Sample stage, 405: Sample stage base, 411, 412, 413: ROI, 420: Cart, 501: Data sharing server, 502: User terminal, 503, 511: Authentication server, 504: Vendor room server, 512: Analysis server.

Claims

1. A measurement device that acquires measurement data obtained by measuring maintenance work performed by a worker on semiconductor manufacturing equipment using a first 3D sensor installed in a space where the work is performed, the measurement device having: an object identification unit that identifies objects included in an image of raw data acquired from the first 3D sensor; a coordinate conversion unit that acquires the position in the space of an identified object identified by the object identification unit; a work judgment unit that determines the work process being performed by the worker based on the movement of one or more of the identified objects; a region setting unit that sets areas of the image of raw data that can be output / cannot be output in accordance with the work process determined by the work judgment unit; and a data restriction unit that generates restricted data that makes invisible areas of the image of raw data that the region setting unit has determined cannot be output.

2. A measuring device according to claim 1, wherein ROIs having attributes relating to whether or not the image of the raw data can be output are set in the space, the ROIs including a first ROI with an output prohibition attribute in which output of the image of the raw data is always prohibited, and a second ROI in which output of the image of the raw data can be switched between enabled and disabled, and the region setting unit sets at least one of the second ROIs, determined as a related ROI in the work process judged by the work judgment unit, as output permitted, and sets other regions as output prohibited.

3. A measurement device according to claim 2, wherein the region setting section switches between enabling and disabling output of the second ROI in accordance with movement of the identified object.

4. A measurement device according to claim 2, wherein the region setting section is capable of further reducing and setting the region of the second ROI that can be output from the image of the raw data.

5. A measuring device according to claim 4, wherein the area setting unit is capable of reducing and setting the area of ​​the second ROI that is set to be outputtable in the image of the raw data to a range that includes the identified object or a range indicated by the worker, and the range indicated by the worker is set by a second 3D sensor worn by the worker.

6. A measuring device according to claim 2, wherein the worker wears a second 3D sensor, the object identification unit identifies an object contained in an image of raw data acquired from the second 3D sensor, and the area setting unit sets a range of the image of raw data acquired from the second 3D sensor that includes an identified object identified by the object identification unit as an area that can be output.

7. A measuring device according to claim 2, wherein a cart is used in the maintenance work, the cart is equipped with a third 3D sensor whose upper surface is a third ROI, the object identification unit identifies an object included in an image of raw data acquired from the third 3D sensor, and the area setting unit, upon detecting that an object has been placed on the cart, sets the third ROI for the image of raw data acquired from the third 3D sensor as outputtable.

8. A measuring device according to claim 1, wherein the area setting section displays the raw data and the limited data so that they can be compared, and is capable of editing invisible areas of the image of the limited data.

9. A measurement device according to claim 1, wherein the area setting unit creates a limited report including information about the process by which the area to be made invisible in the limited data was set.

10. A measuring device according to claim 1, wherein the data limiting section applies one of blacking out, blurring and deformation to areas of the image of the raw data that the area setting section has determined cannot be output.

11. A measurement data utilization system comprising: a measurement device according to any one of claims 1 to 10; and an information processing device connectable to said measurement device via a network, wherein said information processing device is prohibited from being provided with said raw data and is provided with said limited data.

12. A measurement data acquisition method that uses a measurement device to acquire measurement data obtained by measuring maintenance work performed by a worker using a 3D sensor installed in a space where the maintenance work is performed on semiconductor manufacturing equipment, wherein the measurement device has an object identification unit, a coordinate conversion unit, a work judgment unit, an area setting unit, and a data restriction unit, wherein the object identification unit identifies objects included in an image of raw data acquired from the 3D sensor, the coordinate conversion unit acquires the position in the space of an identified object identified by the object identification unit, the work judgment unit determines the work process being performed by the worker based on the movement of one or more of the identified objects, the area setting unit sets areas of the image of the raw data that can be output / cannot be output according to the work process determined by the work judgment unit, and the data restriction unit generates restricted data that makes invisible areas of the image of the raw data that the area setting unit has determined cannot be output.

13. A measurement data acquisition method according to claim 12, wherein ROIs having attributes regarding whether or not the image of the raw data can be output are set in the space, and the ROIs include a first ROI with an output prohibition attribute in which output of the image of the raw data is always prohibited, and a second ROI in which output of the image of the raw data can be switched between enabled and disabled, and the region setting unit sets at least one of the second ROIs among the images of the raw data that is determined as a related ROI in a work process determined by the work judgment unit as enabled for output, and sets other regions as disabled for output.

Images