Time synchronization method and manufacturing apparatus

Abstract

To provide a technology that can synchronize time between a plurality of pieces of captured image data without communicating with an external network.SOLUTION: A time synchronization method to synchronize times attached to multiple pieces of imaging data includes a step of a) acquiring imaging data for each camera by imaging an imaging target with a plurality of cameras, a step of b) detecting event images EI1 to EI3 in which a predetermined event is captured from the imaging data for each of the pieces of imaging data, and a step of c) synchronizing the times attached to the multiple of pieces of imaging data on the basis of the times attached to each of the event image EI1 to EI3.SELECTED DRAWING: Figure 10

Description

Technical Field

[0001] The present invention relates to a time synchronization method and a manufacturing apparatus for synchronizing the times attached to a plurality of imaging data.

Background Art

[0002] There are cases where an apparatus to be controlled is imaged by a plurality of imaging apparatuses, and the operation of the apparatus to be controlled is controlled based on the imaging data. In this case, it is necessary to synchronize the times between the imaging data captured by each imaging apparatus.

[0003] For example, in Patent Document 1, the image recording unit records the image data acquired by the camera extension unit processing unit in time series in association with information regarding the time when the image data was acquired, and in response to an instruction from the CPU unit, the image data and the information regarding the time are transmitted to the CPU unit. With the above configuration, it is possible to synchronize the time with high precision for the device recording regarding the image data captured by the camera unit.

[0004] Furthermore, as a method for synchronizing the times between each imaging data, for example, a method such as Non-Patent Document 1 is known. In Non-Patent Document 1, the time is synchronized between devices connected to an external network.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Non-Patent Documents

[0006]

Non-Patent Document 1

[0007] However, for devices that need to be operated in isolation from external networks for security reasons, it was difficult to synchronize the time between multiple image data sets through communication with the external network. Furthermore, if the device were to be equipped with a time server function, the available space within the device would be limited, and additional installation costs would be required.

[0008] This invention has been made in view of these circumstances, and aims to provide a technology that can synchronize the time between multiple imaging data without communicating with an external network. [Means for solving the problem]

[0009] To solve the above problems, the first invention of this application is: In a substrate processing apparatus that supplies processing liquid from a nozzle head to a substrate, A time synchronization method for synchronizing the timestamps attached to multiple image data, wherein a) multiple cameras , a predetermined area including the nozzle head Target of imaging as The process includes: a) acquiring imaging data for each camera by taking an image; b) detecting an event image from the imaging data in which a predetermined event has been captured for each of the imaging data; and c) synchronizing the timestamps attached to a plurality of imaging data based on the timestamp attached to each of the event images.

[0010] The second invention of this application is a time synchronization method of the first invention, wherein the event is that a predetermined portion included in the image target passes through a predetermined location in the imaging area of ​​a plurality of cameras.

[0011] The third invention of this application is a time synchronization method of the second invention, wherein in step b), the event image is detected based on a reference image that serves as the basis for detection.

[0012] The fourth invention of this application is, A time synchronization method for synchronizing the timestamps attached to multiple image data sets, comprising: a) acquiring image data for each camera by capturing an object using multiple cameras; b) detecting an event image from the image data in which a predetermined event has been captured, for each image data set; and c) synchronizing the timestamps attached to the multiple image data sets based on the timestamp attached to each event image. The aforementioned event is when the brightness of a predetermined region in the imaging data reaches a predetermined threshold.

[0013] The fifth invention of this application is a time synchronization method of the fourth invention, wherein step a) includes a step of emitting light into the imaging areas of the plurality of cameras at a predetermined timing between imaging the object to be imaged by the plurality of cameras.

[0014] The sixth invention of this application is a time synchronization method of the fourth or fifth invention, wherein step b) is a step of detecting the one captured image as the event image if the event is captured in one captured image in one of the imaging data, and detecting one captured image as the event image from the multiple captured images based on the brightness of the multiple captured images if the event is captured in multiple captured images in one of the imaging data.

[0015] The seventh invention of this application is a time synchronization method according to any one of the first to fifth inventions, wherein step b) is a step of detecting the one image captured as the event image if the event is captured in one image captured in one image of the image data, and detecting one image captured as the event image from the multiple image captured based on the time assigned to each of the multiple image captured in the image data.

[0016] The eighth invention of this application is a time synchronization method according to any one of the first to fifth inventions, wherein the imaging data is a video, and the event image is a frame included in the video.

[0017] The ninth invention of the present application is a time synchronization method for any one of the first to fifth inventions, and the steps a), b), and c) are repeatedly executed at a predetermined time interval.

[0018] The tenth invention of the present application is The nozzle head supplies the processing liquid to the substrate, A manufacturing apparatus that performs processing based on a plurality of imaging data, comprising a plurality of cameras that image an imaging target and a control unit, wherein the control unit a) causes the plurality of cameras to A predetermined area including the nozzle head the imaging target as acquire the imaging data for each camera by imaging; b) detect, for each of the imaging data, an event image in which a predetermined event is imaged from a plurality of imaging images included in the imaging data; and c) synchronize the times attached to the plurality of imaging data based on the times attached to the respective event images.

Advantages of the Invention

[0019] According to the first to tenth inventions of the present application, it is possible to synchronize the times between a plurality of imaging data without communicating with an external network.

[0020] In particular, according to the second invention of the present application, it is possible to synchronize the times between a plurality of imaging data based on a predetermined operation of the imaging target.

[0021] In particular, according to the fourth invention of the present application, it is possible to synchronize the times between a plurality of imaging data based on a change in the luminance of a predetermined region in the imaging data.

[0022] In particular, according to the sixth invention of the present application, it is possible to synchronize the times between a plurality of imaging data with higher accuracy.

[0023] In particular, according to the seventh invention of the present application, it is possible to synchronize the times between a plurality of imaging data with higher accuracy.

[0024] In particular, according to the ninth invention of this application, the time between multiple imaging data can be continuously synchronized. [Brief explanation of the drawing]

[0025] [Figure 1] This is a plan view of the substrate processing device. [Figure 2] This is a longitudinal cross-section of the processing unit. [Figure 3] This is a conceptual diagram illustrating how images are captured by a camera. [Figure 4] This is a conceptual diagram illustrating how images are captured by a camera. [Figure 5] This is a block diagram showing the connections between the computer and the various parts within the processing unit. [Figure 6] This is a flowchart showing the circuit board processing procedure. [Figure 7] This flowchart shows the time synchronization process. [Figure 8] This flowchart shows the time synchronization process. [Figure 9] This is a diagram conceptually illustrating the structure of a video. [Figure 10] This diagram conceptually illustrates the structure of a video after time synchronization has been performed. [Modes for carrying out the invention]

[0026] The embodiments of the present invention will be described in detail below with reference to the drawings.

[0027] <1. Overall configuration of the substrate processing equipment> Figure 1 is a plan view of a substrate processing apparatus 100, which is an example of a manufacturing apparatus according to the present invention. This substrate processing apparatus 100 is an apparatus that supplies a processing liquid to the surface of a disc-shaped substrate W (silicon wafer) in the semiconductor wafer manufacturing process to process the surface of the substrate W. As shown in Figure 1, the substrate processing apparatus 100 comprises an indexer 101, a plurality of processing units 102, and a main transport robot 103.

[0028] The indexer 101 is the part that receives substrates W before processing from the outside and delivers processed substrates W to the outside. Multiple carriers for accommodating multiple substrates W are arranged in the indexer 101. The indexer 101 also has a transfer robot (not shown in the figure). The transfer robot transfers the substrates W between the carriers in the indexer 101 and the processing unit 102 or the main transfer robot 103.

[0029] The processing unit 102 is a so-called single-wafer processing unit that processes substrates W one at a time. Multiple processing units 102 are arranged around the main transport robot 103. In this embodiment, four processing units 102 arranged around the main transport robot 103 are stacked in three layers in the height direction. That is, the substrate processing apparatus 100 of this embodiment has a total of 12 processing units 102. Multiple substrates W are processed in parallel in each processing unit 102. However, the number of processing units 102 in the substrate processing apparatus 100 is not limited to 12, and may be, for example, 1, 4, 8, 24, etc.

[0030] The main transport robot 103 is a mechanism for transporting substrates W between the indexer 101 and a plurality of processing units 102. The main transport robot 103 has, for example, a hand for holding substrates W and an arm for moving the hand. The main transport robot 103 takes the substrates W before processing from the indexer 101 and transports them to the processing units 102. When processing of the substrates W in the processing units 102 is completed, the main transport robot 103 takes the processed substrates W from the processing units 102 and transports them to the indexer 101.

[0031] <2. Processing Unit Configuration> Next, the detailed configuration of the processing unit 102 will be described. Below, one of the multiple processing units 102 of the substrate processing apparatus 100 will be described, but the other processing units 102 have a similar configuration.

[0032] Figure 2 is a longitudinal cross-sectional view of the processing unit 102. As shown in Figure 2, the processing unit 102 includes a chamber 10, a substrate holding section 20, a rotating mechanism 30, a processing liquid supply section 40, a processing liquid collection section 50, a shut-off plate 60, a first camera 71, a second camera 72, a third camera 73, and a computer 80.

[0033] Chamber 10 is a housing that encloses a processing space 11 for processing substrates W. Chamber 10 has side walls 12 surrounding the sides of the processing space 11, a top plate 13 covering the upper part of the processing space 11, and a bottom plate 14 covering the lower part of the processing space 11. A substrate holding unit 20, a rotating mechanism 30, a processing liquid supply unit 40, a processing liquid collection unit 50, a shutoff plate 60, and first cameras 71 to third cameras 73 are housed inside Chamber 10. Part of the side wall 12 is provided with an inlet / outlet for loading substrates W into and out of Chamber 10, and a shutter for opening and closing the inlet / outlet.

[0034] The substrate holding section 20 is a mechanism that holds the substrate W horizontally (with its normal vector facing vertically) inside the chamber 10. As shown in Figure 2, the substrate holding section 20 has a disc-shaped spin base 21 and a plurality of chuck pins 22. The plurality of chuck pins 22 are provided at equal angular intervals along the outer circumference of the upper surface of the spin base 21. The substrate W is held by the plurality of chuck pins 22 with the surface to be processed, where the pattern is formed, facing upwards. Each chuck pin 22 contacts the lower surface and outer end surface of the peripheral edge of the substrate W, supporting the substrate W at a position above the upper surface of the spin base 21 with a small gap between them.

[0035] Inside the spin base 21, there is a chuck pin switching mechanism 23 for switching the positions of multiple chuck pins 22. The chuck pin switching mechanism 23 switches the multiple chuck pins 22 between a holding position for holding the substrate W and a release position for releasing the substrate W.

[0036] The rotation mechanism 30 is a mechanism for rotating the substrate holder 20. The rotation mechanism 30 is housed inside a motor cover 31 located below the spin base 21. As shown by the dashed line in Figure 2, the rotation mechanism 30 has a spin motor 32 and a support shaft 33. The support shaft 33 extends vertically, its lower end is connected to the spin motor 32, and its upper end is fixed to the center of the lower surface of the spin base 21. When the spin motor 32 is driven, the support shaft 33 rotates about its axis 330. Along with the support shaft 33, the substrate holder 20 and the substrate W held by the substrate holder 20 also rotate about the axis 330.

[0037] The processing liquid supply unit 40 is a mechanism that supplies processing liquid to the upper surface of the substrate W held by the substrate holding unit 20. The processing liquid supply unit 40 has an upper nozzle 41 and a lower nozzle 42. As shown in Figures 1 and 2, the upper nozzle 41 has a nozzle arm 411, a nozzle head 412 provided at the tip of the nozzle arm 411, and a nozzle motor 413. The nozzle arm 411 rotates horizontally around its base end by the drive of the nozzle motor 413. This allows the nozzle head 412 to be moved between a processing position above the substrate W held by the substrate holding unit 20 (position of the dashed line in Figure 1) and a retracted position outside the processing liquid collection unit 50 (position of the solid line in Figure 1).

[0038] The nozzle head 412 is connected to a liquid supply unit (not shown) for supplying the processing liquid. Examples of processing liquids used include SPM cleaning solution (a mixture of sulfuric acid and hydrogen peroxide), SC-1 cleaning solution (a mixture of ammonia water, hydrogen peroxide, and pure water), SC-2 cleaning solution (a mixture of hydrochloric acid, hydrogen peroxide, and pure water), DHF cleaning solution (dilute hydrofluoric acid), and pure water (deionized water). When the nozzle head 412 is positioned in the processing location and the valve of the liquid supply unit is opened, the processing liquid supplied from the liquid supply unit is discharged from the nozzle head 412 toward the upper surface of the substrate W held by the substrate holding unit 20.

[0039] The nozzle head 412 may also be a so-called two-fluid nozzle that mixes a processing liquid with a pressurized gas to generate droplets and sprays the mixed fluid of the droplets and gas onto the substrate W. Furthermore, a single processing unit 102 may be provided with multiple top nozzles 41.

[0040] The lower nozzle 42 is positioned inside a through-hole located in the center of the spin base 21. The discharge port of the lower nozzle 42 faces the lower surface of the substrate W held by the substrate holding section 20. The lower nozzle 42 is also connected to a liquid supply section for supplying processing liquid. When processing liquid is supplied from the liquid supply section to the lower nozzle 42, the processing liquid is discharged from the lower nozzle 42 toward the lower surface of the substrate W.

[0041] The processing liquid collection section 50 is the part that collects the processing liquid after use. As shown in Figure 2, the processing liquid collection section 50 has an inner cup 51, an intermediate cup 52, and an outer cup 53. The inner cup 51, intermediate cup 52, and outer cup 53 can move up and down independently of each other by a lifting mechanism (not shown).

[0042] The inner cup 51 has an annular first guide plate 510 that surrounds the substrate holding portion 20. The middle cup 52 has an annular second guide plate 520 located outside and above the first guide plate 510. The outer cup 53 has an annular third guide plate 530 located outside and above the second guide plate 520. The bottom of the inner cup 51 extends below the middle cup 52 and the outer cup 53. On the upper surface of the bottom, a first drainage groove 511, a second drainage groove 512, and a third drainage groove 513 are provided, in order from the inside.

[0043] The processing liquid discharged from the upper nozzle 41 and lower nozzle 42 of the processing liquid supply unit 40 is supplied to the substrate W, and then scattered outwards by the centrifugal force caused by the rotation of the substrate W. The processing liquid scattered from the substrate W is then collected on one of the first guide plate 510, the second guide plate 520, or the third guide plate 530. The processing liquid collected on the first guide plate 510 is discharged to the outside of the processing unit 102 through the first drainage groove 511. The processing liquid collected on the second guide plate 520 is discharged to the outside of the processing unit 102 through the second drainage groove 512. The processing liquid collected on the third guide plate 530 is discharged to the outside of the processing unit 102 through the third drainage groove 513.

[0044] Thus, this processing unit 102 has multiple discharge paths for the processing liquid. Therefore, the processing liquid supplied to the substrate W can be separated and recovered according to its type. Consequently, the disposal or recycling of the recovered processing liquid can also be carried out separately according to the properties of each processing liquid.

[0045] The shielding plate 60 is a component used to suppress the diffusion of gas near the surface of the substrate W during certain processes such as drying. The shielding plate 60 has a disc-shaped outer form and is positioned horizontally above the substrate holding portion 20. As shown in Figure 2, the shielding plate 60 is connected to a lifting mechanism 61. When the lifting mechanism 61 is operated, the shielding plate 60 moves up and down between an upper position, which is above the upper surface of the substrate W held by the substrate holding portion 20, and a lower position, which is closer to the upper surface of the substrate W than the upper position. The lifting mechanism 61 uses, for example, a mechanism that converts the rotational motion of a motor into linear motion using a ball screw.

[0046] Furthermore, an outlet 62 for blowing out a drying gas (hereinafter referred to as "drying gas") is provided in the center of the lower surface of the shut-off plate 60. The outlet 62 is connected to an air supply unit (not shown) that supplies the drying gas. For example, heated nitrogen gas is used as the drying gas.

[0047] When the processing liquid is supplied to the substrate W from the upper nozzle 41, the shut-off plate 60 retracts to the upper position. After the processing liquid is supplied, when drying is performed on the substrate W, the shut-off plate 60 is lowered to the lower position by the lifting mechanism 61. Then, drying gas is blown from the outlet 62 toward the upper surface of the substrate W. At this time, the shut-off plate 60 prevents the diffusion of the gas. As a result, drying gas is efficiently supplied to the upper surface of the substrate W.

[0048] The first camera 71 to the third camera 73 are mechanisms for imaging a predetermined area within the chamber 10. The first camera 71 to the third camera 73 capture a video including a predetermined event occurring in imaging area A. The processing unit 102 may have two cameras or four or more cameras.

[0049] The first camera 71 to the third camera 73 are installed, for example, in a position close to the inner surface of the side wall 12 of the chamber 10. As shown in Figure 2, the first camera 71 to the third camera 73 are each covered by covers 74 to 76. This protects the first camera 71 to the third camera 73 from the processing liquid and the gases generated by the evaporation of the processing liquid.

[0050] Figure 3 is a conceptual diagram showing the imaging by the first camera 71 to the third camera 73 in the first embodiment described later. In the first embodiment, the first camera 71 to the third camera 73 capture images of the movement of the nozzle head 412 in a predetermined imaging area A that includes the nozzle head 412.

[0051] Figure 4 is a conceptual diagram showing the imaging process by the first camera 71 to the third camera 73 in the second embodiment described later. In the second embodiment, the processing unit 102 includes a light-emitting unit 90. The light-emitting unit 90 is a mechanism that instantaneously emits light to the first camera 71 to the third camera 73. In the second embodiment, the first camera 71 to the third camera 73 image the light emitted by the light-emitting unit 90.

[0052] Furthermore, in the video captured by the first camera 71 to the third camera 73, as long as the change in brightness in the imaging area A is recorded, the light-emitting unit 90 may or may not be included in the imaging area A.

[0053] As a result, in both the first and second embodiments, the first camera 71, the second camera 72, and the third camera 73 acquire the first video M1, the second video M2, and the third video M3, respectively. The first cameras 71 to the third cameras 73 then transmit the obtained first video M1 to third video M3 to the computer 80. The first video M1 to third video M3 correspond to the "imaging data" of the present invention.

[0054] The computer 80 controls the operation of each part within the processing unit 102. Figure 5 is a block diagram showing the electrical connections between the computer 80 and each part within the processing unit 102. As conceptually shown in Figure 5, the computer 80 includes a processor 81 such as a CPU, memory 82 such as RAM, a storage unit 83 such as a hard disk drive, and a control unit 84.

[0055] As shown in Figure 5, the computer 80 is connected to the chuck pin switching mechanism 23, spin motor 32, nozzle motor 413, valve of the processing liquid supply unit 40, lifting mechanism of the processing liquid collection unit 50, lifting mechanism 61 of the shut-off plate 60, and the first camera 71 to the third camera 73, respectively, via wired or wireless communication.

[0056] The memory unit 83 stores program P, which is a computer program for controlling the operation of each part of the processing unit 102.

[0057] Furthermore, the memory unit 83 stores the first reference image RI1, the second reference image RI2, and the third reference image RI3. The first reference image RI1 to the third reference image RI3 are used as references for detecting the first event image EI1 to the third event image EI3, which will be described later, in the time synchronization process described later. Details of the first reference image RI1 to the third reference image RI3 will be described later.

[0058] The control unit 84 is realized when the processor 81 reads the program P stored in the storage unit 83, loads it into the memory 82, and executes processing according to the program P. The control unit 84 includes a substrate processing control unit 841 and a time synchronization unit 842.

[0059] The substrate processing control unit 841 performs processing on the substrate W in the processing unit 102. Details of the processing of the substrate W in the processing unit 102 will be described later.

[0060] The time synchronization unit 842 performs a time synchronization process to synchronize the times assigned to the first video M1 to the third video M3, respectively. Details of the time synchronization process will be described later.

[0061] After the time synchronization process is performed by the time synchronization unit 842, the control unit 84 controls the operation of each part in the processing unit 102 based on the first video M1 to the third video M3 whose times have been synchronized by the time synchronization process.

[0062] <3. Operation of the substrate processing device> Next, we will explain the processing of the substrate W in the processing unit 102, which is carried out by the substrate processing control unit 841. Figure 6 is a flowchart showing the processing procedure for the substrate W.

[0063] When processing the substrate W in the processing unit 102, first, the main transport robot 103 loads the substrate W to be processed into the chamber 10 (step S101). The substrate W loaded into the chamber 10 is held horizontally by a plurality of chuck pins 22 of the substrate holding section 20. Then, the rotation of the substrate W is started by driving the spin motor 32 of the rotation mechanism 30 (step S102). Specifically, the support shaft 33, spin base 21, the plurality of chuck pins 22, and the substrate W held by the chuck pins 22 rotate around the axis 330 of the support shaft 33.

[0064] Next, processing liquid is supplied from the processing liquid supply unit 40 (step S103). In step S103, the nozzle motor 413 drives the nozzle head 412 to a processing position facing the upper surface of the substrate W. Then, the processing liquid is discharged from the nozzle head 412 positioned at the processing position. Parameters such as the discharge speed and discharge time of the processing liquid are pre-set in the memory unit 83. The substrate processing control unit 841 executes the discharge operation of the processing liquid from the upper nozzle 41 according to these settings.

[0065] In step S103, the processing liquid may be discharged from the upper nozzle 41 while the upper nozzle 41 is oscillating horizontally at the processing position. Additionally, if necessary, the processing liquid may be discharged from the lower nozzle 42.

[0066] During the processing liquid supply process in step S103, the shut-off plate 60 is positioned above the upper nozzle 41. Once the supply of processing liquid to the substrate W is complete and the upper nozzle 41 is in the retracted position, the computer 80 operates the lifting mechanism 61 to move the shut-off plate 60 from the upper position to the lower position. The computer then increases the rotation speed of the spin motor 32 to speed up the rotation of the substrate W, and blows drying gas from the outlet 62 located on the lower surface of the shut-off plate 60 towards the substrate W. This dries the surface of the substrate W (step S104).

[0067] Once the drying process of the substrate W is complete, the spin motor 32 is stopped to halt the rotation of the substrate W. The substrate W is then released from its grip by the multiple chuck pins 22. Subsequently, the main transport robot 103 removes the processed substrate W from the substrate holding unit 20 and transports it out of the chamber 10 (step S105).

[0068] Each processing unit 102 repeatedly performs the processes described in steps S101 to S105 on multiple substrates W that are transported sequentially.

[0069] <4. Time synchronization process> Next, the time synchronization process executed by the time synchronization unit 842 will be described. The time synchronization process is a process to synchronize the timestamps attached to the first video M1 to the third video M3 captured by the first camera 71 to the third camera 73. The flow of the time synchronization process in the first and second embodiments will be described below.

[0070] <4-1. First Embodiment> Figure 7 is a flowchart showing the flow of the time synchronization process in the first embodiment. In the first embodiment, the passage of the nozzle head 412 through a predetermined location in the imaging area A is defined as an "event". In the first embodiment, the first cameras 71 to the third cameras 73 capture the movement of the nozzle head 412 in step S103 and acquire the first video M1 to the third video M3 (step S201). Note that the movement of the nozzle head 412 captured in step S201 does not have to be performed in step S103. For example, the movement of the nozzle head 412 may be performed in advance before steps S101 to S105 are executed.

[0071] The first camera 71 to the third camera 73 transmit the first video M1, the second video M2, and the third video M3, which were acquired in step S201, to the computer 80. The first video M1 to the third video M3 transmitted to the computer 80 are stored in the storage unit 83.

[0072] Next, the time synchronization unit 842 detects the first event images EI1 to the third event images EI3 from the first video M1 to the third video M3 stored in the storage unit 83 (step S202). Figure 9 is a conceptual diagram showing the configuration of the first video M1 to the third video M3. As shown in Figure 9, the first video M1 is composed of a plurality of frame images F1 (F11, F12, ...) captured at minute time intervals. Similarly, the second video M2 and the third video M3 are composed of a plurality of frame images F2 (F21, F22, ...) and a plurality of frame images F3 (F31, F32, ...). Frame images F1 to F3 correspond to the "captured images" of the present invention.

[0073] In step S202, the first event images EI1 to the third event images EI3 are detected from the first video M1 to the third video M3 based on the first reference images RI1 to the third reference images RI3 stored in the memory unit 83.

[0074] The first reference image RI1 to the third reference image RI3 record the moment when the nozzle head 412 passes a predetermined location in the imaging area A, which is defined as an "event" in the first embodiment. The first reference image RI1 is captured with the same imaging range and angle as the first video M1. The second reference image RI2 is captured with the same imaging range and angle as the second video M2. The third reference image RI3 is captured with the same imaging range and angle as the third video M3.

[0075] The time synchronization unit 842 compares each of the multiple frame images F1 with the first reference image RI1 for a predetermined area of ​​each image. The time synchronization unit 842 then detects the frame image that is most similar to the first reference image RI1 as the first event image EI1. In the example shown in Figure 9, frame image F12 is detected as the first event image EI1.

[0076] Similar to the first video M1, the second event image EI2 and the third event image EI3 are detected from the second video M2 and the third video M3, respectively. In the example shown in Figure 9, frame image F23 and frame image F35 are detected as the second event image EI2 and the third event image EI3, respectively.

[0077] In step S202, the time synchronization unit 842 may detect the first event image EI1 to the third event image EI3 using a predetermined image processing technique, such as an object detection algorithm.

[0078] Figure 10 is a conceptual diagram showing the structure of the video after the time information has been corrected. After step S202, the time synchronization unit 842 synchronizes the times assigned to the first video M1 to the third video M3 based on the times assigned to the first event images EI1 to the third event images EI3 (step S203). The times assigned to the first video M1 to the third video M3 are, for example, time information based on the internal clocks of the first camera 71 to the third camera 73.

[0079] Specifically, the time synchronization unit 842 first selects the image with the latest timestamp among the first event image EI1 to the third event image EI3 as the reference for time synchronization. For example, in the example shown in Figure 9, the time synchronization unit 842 selects the third event image EI3 as the reference.

[0080] The time synchronization unit 842 then corrects the times assigned to the first video M1 and the second video M2 so that the times assigned to the first event image EI1 and the second event image EI2 match the times assigned to the third event image EI3. As a result, the times assigned to the first video M1 to the third video M3 are synchronized, as shown in Figure 10.

[0081] Furthermore, each time that the processing unit 102 repeatedly executes step S102 for multiple substrates W being transported sequentially, steps S201 to S203 may be repeatedly executed. Alternatively, the process of executing steps S201 to S203 after operating the nozzle head 412 at a predetermined timing may be repeatedly executed at predetermined time intervals. This allows the timings assigned to the first video M1 to the third video M3 to be synchronized at predetermined time intervals. Therefore, the timings between the first video M1 to the third video M3 can be continuously synchronized.

[0082] <4-2. Second Embodiment> Figure 8 is a flowchart showing the flow of the time synchronization process in the second embodiment. In the second embodiment, an "event" is defined as when the brightness of a predetermined area in the first video M1 to the third video M3 reaches a predetermined threshold. In the second embodiment, first cameras 71 to 73 capture video in imaging area A (step S301).

[0083] The light-emitting unit 90 emits light instantaneously at any time while step S301 is being executed. As a result, the light emitted from the light-emitting unit 90 is recorded in the first video M1 to the third video M3. The first cameras 71 to the third cameras 73 each transmit the first video M1 to the third video M3 they have acquired to the computer 80. The first video M1 to the third video M3 transmitted to the computer 80 are stored in the storage unit 83.

[0084] Next, the time synchronization unit 842 detects the first event image EI1 to the third event image EI3 from the first video M1 to the third video M3 stored in the memory unit 83 (steps S302 to S304).

[0085] The specific procedures for steps S302 to S304 will be explained using the first video M1 as an example. First, the time synchronization unit 842 extracts an image from among the multiple frame images F1 that make up the first video M1 in which the brightness of pixels in a predetermined area is equal to or greater than a predetermined threshold (step S302). Hereafter, the image extracted in step S302 will be referred to as the "extracted image".

[0086] If only one image is extracted (step S303: No), the time synchronization unit 842 detects the extracted image as the event image EI.

[0087] In this case, depending on the emission time by the light-emitting unit 90 or the threshold value, multiple consecutive frame images F1 may be extracted. That is, in this case, events are recorded in multiple extracted images in the first video M1. If multiple extracted images exist (step S303: Yes), the time synchronization unit 842 selects one extracted image from among the multiple extracted images and detects it as the first event image EI1 (step S304).

[0088] For example, the time synchronization unit 842 may select the first event image EI1 based on the brightness of a predetermined region in a plurality of extracted images. Specifically, the time synchronization unit 842 may select as the first event image EI1 the extracted image in which the average brightness of multiple pixels constituting a predetermined region is the highest among the plurality of extracted images.

[0089] Alternatively, the time synchronization unit 842 may select the first event image EI1 based on the timestamps attached to each of the multiple extracted images. Specifically, the time synchronization unit 842 may select as the first event image EI1 one extracted image that has a timestamp closest to the average value of the times attached to each of the multiple extracted images.

[0090] The time synchronization unit 842 detects the second event image EI2 and the third event image EI3 for the second video M2 and the third video M3, respectively, by executing steps S302 to S304, just as with the first video M1.

[0091] Next, the time synchronization unit 842 synchronizes the times assigned to the first video M1 to the third video M3 based on the times assigned to the first event images EI1 to the third event images EI3 (step S305). The contents of step S305 are the same as those of step S203 in the first embodiment, so the explanation is omitted.

[0092] The light-emitting unit 90 may also emit light periodically at predetermined time intervals. In this case, the time synchronization unit 842 periodically executes steps S302 to S305. This allows the time assigned to the first video M1 to the third video M3 to be synchronized at predetermined time intervals. Therefore, the time between the first video M1 to the third video M3 can be continuously synchronized.

[0093] As described above, the substrate processing device 100 detects a frame image F in which a predetermined event is captured from each of the multiple video Ms, as an event image EI. Then, it synchronizes the time between the multiple video Ms based on the time attached to the event image EI for each video M. This makes it possible to synchronize the time between the multiple video Ms without communication between the substrate processing device 100 and an external network.

[0094] In step S202 of the first embodiment, if multiple images corresponding to the event image EI are detected for a single video M, these multiple images may be used as extracted images, and one event image EI may be detected using the same procedure as in step S304 of the second embodiment. Specifically, the time synchronization unit 842 may select as the event image EI one extracted image that has a time closest to the average value of the times attached to each of the multiple extracted images extracted in step S202.

[0095] In the second embodiment, the first event image EI1 to the third event image EI3 may be detected by the same procedure as in step S202 of the first embodiment. In this case, the time synchronization unit 842 may detect the first event image EI1 to the third event image EI3 based on the first reference image RI1 to the third reference image RI3, each of which records the moment when the light-emitting unit 90 emitted light.

[0096] <5. Variation> Although one embodiment of the present invention has been described above, the present invention is not limited to the above-described embodiment.

[0097] In the above embodiment, the time in multiple videos M capturing the substrate processing apparatus 100 was synchronized. However, the time synchronization method in this disclosure can be applied to various devices. For example, the time synchronization method in this disclosure may be applied to a printing apparatus. In this case, for example, an "event" may be defined as the passage of a predetermined resist mark printed on a recording medium to a predetermined location.

[0098] In the first embodiment described above, an "event" was defined as the nozzle arm 411 passing over a predetermined location in the imaging area A. In the second embodiment described above, an "event" was defined as the brightness of a predetermined area in the first video M1 to the third video M3 reaching a predetermined threshold. However, the content of an event is not limited to those defined in the above embodiments. For example, an event may be a predetermined operation by the substrate processing apparatus 100, such as the nozzle head 412 discharging processing liquid. Alternatively, an event may be a change in the pixel value of a predetermined area in the first video M1 to the third video M3 due to a change in the amount or color of light emitted by the light-emitting unit 90.

[0099] In the above embodiment, the first cameras 71 to the third cameras 73 transmitted the first video M1 to the third video M3 to the computer 80 provided in the processing unit 102. Then, the time synchronization processing was performed by the time synchronization unit 842 of the computer 80. However, each of the first cameras 71 to the third cameras 73 may be equipped with its own computer for processing the first video M1 to the third video M3. Each computer provided in the first cameras 71 to the third cameras 73 may perform steps S201 to S202 in the first embodiment or steps S301 to S304 in the second embodiment for each video M1 to M3, and then transmit the first event image EI1 to the third event image EI3 and each video M1 to M3 to the computer 80, respectively. After that, step S203 or S305 may be performed by the time synchronization unit 842.

[0100] In the above embodiment, the first camera 71 to the third camera 73 each captured the first video M1 to the third video M3. However, the first camera 71 to the third camera 73 may intermittently capture multiple images at predetermined time intervals. In this case, during the time synchronization process, the time synchronization unit 842 may detect the first event image EI1 to the third event image EI3 from the group of multiple images captured by the first camera 71 to the third camera 73. The time synchronization unit 842 may then synchronize the times assigned to the group of multiple images based on the times assigned to the first event image EI1 to the third event image EI3.

[0101] The elements that appear in the above embodiments and modifications may be combined as appropriate, to the extent that no contradictions arise. [Explanation of symbols]

[0102] 40: Processing liquid supply unit 41: Top nozzle 71: Camera 1 72: Second camera 73: Third camera 90: Light-emitting part 100: Substrate processing equipment 411: Nozzle Arm 412: Nozzle head 413: Nozzle motor 841: Substrate Processing Control Unit 842: Time synchronization unit A: Imaging area EI1: First event image EI2: Second event image EI3: Third Event Image F1: Frame image F2: Frame image F3: Frame image M1: First video M2: Second video M3: Third video RI1: First reference image RI2: Second reference image RI3: Third reference image W: Circuit board

Claims

1. A substrate processing apparatus that supplies processing liquid from a nozzle head to a substrate, comprising a time synchronization method for synchronizing the timestamps attached to a plurality of imaging data, a) A step of acquiring imaging data for each camera by imaging a predetermined area including the nozzle head with multiple cameras, b) A step of detecting an event image in which a predetermined event has been captured from the imaging data for each of the imaging data, c) A step of synchronizing the timestamps attached to multiple image data based on the timestamp attached to each of the event images, A time synchronization method having the following characteristics.

2. A time synchronization method according to claim 1, A time synchronization method in which the event is that a predetermined portion included in the image target passes through a predetermined location in the imaging area of ​​a plurality of cameras.

3. A time synchronization method according to claim 2, Step b) above is a time synchronization method for detecting the event image based on a reference image that serves as the basis for detection.

4. A time synchronization method for synchronizing the timestamps attached to multiple imaging data, a) A step of acquiring imaging data for each camera by imaging the target object with multiple cameras, b) A step of detecting an event image in which a predetermined event has been captured from the imaging data for each of the imaging data, c) A step of synchronizing the timestamps attached to multiple image data based on the timestamp attached to each of the event images, It has, The aforementioned event is a time synchronization method in which the brightness of a predetermined region in the imaging data reaches a predetermined threshold.

5. A time synchronization method according to claim 4, Step a) is a time synchronization method that includes emitting light into the imaging areas of the plurality of cameras at a predetermined timing between imaging the target object with the plurality of cameras.

6. A time synchronization method according to claim 4 or claim 5, The aforementioned step b) is, If the event is captured in one of the captured images within one of the aforementioned imaging data, the captured image is detected as the event image. In a single image data, if the event is captured in multiple images, the process involves detecting one of the multiple images as the event image based on the brightness of the multiple images. Time synchronization method.

7. A time synchronization method according to any one of claims 1 to 5, The aforementioned step b) is, If the event is captured in one of the captured images within one of the aforementioned imaging data, the captured image is detected as the event image. If the event is captured in multiple images within a single image data set, the process involves detecting one of the multiple images as the event image based on the timestamps attached to each of the multiple images. Time synchronization method.

8. A time synchronization method according to any one of claims 1 to 5, The aforementioned imaging data is a video, The aforementioned event image is a frame included in the aforementioned video. Time synchronization method.

9. A time synchronization method according to any one of claims 1 to 5, A time synchronization method comprising repeatedly performing steps a), b), and c) at predetermined time intervals.

10. A manufacturing apparatus that supplies a processing liquid from a nozzle head to a substrate and performs processing based on a plurality of imaging data, Multiple cameras to capture the target object, Control unit and Equipped with, The control unit, a) A step of acquiring imaging data for each camera by having multiple cameras image a predetermined area including the nozzle head as the imaging target, b) A step of detecting an event image in which a predetermined event has been captured from a plurality of captured images included in the imaging data, for each set of imaging data, c) A step of synchronizing the timestamps attached to multiple image data based on the timestamp attached to each of the event images, A manufacturing device that performs this task.

Images