Time synchronization method and manufacturing apparatus

Abstract

A time synchronization method for synchronizing times given to a plurality of pieces of imaging data comprises: a step a for acquiring imaging data at each of a plurality of cameras by imaging an object to be imaged by the plurality of cameras; a step b in which event images (EI1-EI3) in which predetermined events have been captured are detected from the imaging data for each of the imaging data; and a step c for synchronizing the times given to the plurality of pieces of imaging data on the basis of the times given to the event images (EI1-EI3). This makes it possible to synchronize the time between the plurality of pieces of imaging data without communicating with an external network.

Description

Technical Field

[0001] The present invention relates to a time synchronization method and a manufacturing device for synchronizing times assigned to a plurality of photographic data. Background Art

[0002] There may be a case where a plurality of imaging devices are used to capture images of a device to be controlled, and the operation of the device to be controlled is controlled based on the captured data. In this case, it is necessary to synchronize the time of the captured data captured by each imaging device.

[0003] For example, in Patent Document 1, the image recording unit can be configured as follows: the image data acquired by the camera expansion unit processing unit is associated with the information related to the time when the image data has been acquired and recorded in time series, and the image data and the information related to the time are sent to the CPU unit in response to an instruction from the CPU unit. With the above configuration, the time and the device recording related to the image data taken by the camera unit can be synchronized with high precision.

[0004] As a method of synchronizing the time between each piece of imaging data, for example, there is known a method disclosed in Non-Patent Document 1. In Non-Patent Document 1, the time is synchronized between devices connected to an external network.

[0005] Prior art literature

[0006] Patent Literature

[0007] Patent Document 1: Japanese Patent Application Publication No. 2020-134985

[0008] Non-patent literature

[0009] Non-patent document 1: “IEEE Standard for a Precision Clock Synchronization Protocol for Networked Measurement and Control Systems IEEE Std 1588-2008”. Summary of the invention

[0010] Problem that the invention aims to solve

[0011] However, for a device that needs to be disconnected from an external network for security reasons, it is difficult to synchronize the time between multiple captured data through communication with an external network. In addition, if the device has a time server function, there are problems such as limited locations where it can be installed due to vacant space in the device and the need for additional installation costs.

[0012] The present invention has been made in view of the above-mentioned problems, and an object of the present invention is to provide a technology capable of synchronizing the time between a plurality of photographic data without performing communication with an external network.

[0013] Means used to solve problems

[0014] In order to solve the above-mentioned problem, the first invention of the present application is a time synchronization method, which is assigned to a plurality of shooting data for time synchronization, wherein the method comprises: step a, shooting a shooting object by a plurality of cameras, thereby obtaining the shooting data from each of the cameras; step b, for each of the shooting data, detecting an event image of a predetermined event shot from the shooting data; and step c, synchronizing the time assigned to the plurality of shooting data based on the time assigned to each of the event images.

[0015] A second invention of the present application is the time synchronization method as described in the first invention, wherein the event is that a predetermined portion included in the photographic object passes through a predetermined position in the photographing areas of the plurality of cameras.

[0016] A third invention of the present application is the time synchronization method according to the second invention, wherein in the step b, the event image is detected based on a reference image serving as a reference for detection.

[0017] A fourth invention of the present application is the time synchronization method as described in the first invention, wherein the event is that the brightness of a predetermined area in the captured data reaches a predetermined threshold.

[0018] The fifth invention of the present application is the time synchronization method as described in the fourth invention, wherein the step a includes the step of emitting light toward the photographing areas of the plurality of cameras at a predetermined timing during the period in which the plurality of cameras photograph the photographing object.

[0019] The sixth invention of the present application is a moment synchronization method as described in the fourth invention or the fifth invention, wherein the process b is the following process: in one of the shooting data, when a shooting image captures the event, the one shooting image is detected as the event image; in one of the shooting data, when multiple shooting images capture the event, one shooting image is detected from the multiple shooting images as the event image based on the brightness of the multiple shooting images.

[0020] The seventh invention of the present application is a moment synchronization method as described in any one of the first to fifth inventions, wherein the process b is the following process: in one of the shooting data, when one shooting image captures the event, the one shooting image is detected as the event image; in one of the shooting data, when multiple shooting images capture the event, one shooting image is detected from the multiple shooting images as the event image based on the moments respectively assigned to the multiple shooting images.

[0021] An eighth invention of the present application is the time synchronization method as described in any one of the first to seventh inventions, wherein the photographed data is a dynamic image (animation), and the event image is a frame (frame) included in the dynamic image.

[0022] A ninth invention of the present application is the time synchronization method according to any one of the first to eighth inventions, wherein the step a, the step b, and the step c are repeatedly executed at predetermined time intervals.

[0023] The tenth invention of the present application is a manufacturing device that performs processing based on multiple shooting data, wherein the device comprises: multiple cameras that shoot a shooting object, and a control unit; the control unit executes: step a, causing the multiple cameras to shoot the shooting object, thereby obtaining the shooting data from each camera; step b, for each shooting data, detecting an event image that shoots a predetermined event from the multiple shooting images included in the shooting data; and step c, synchronizing the time assigned to the multiple shooting data based on the time assigned to each of the event images.

[0024] Effects of the Invention

[0025] According to the first to tenth inventions of the present application, the time of a plurality of imaging data can be synchronized without communicating with an external network.

[0026] In particular, according to the second invention of the present application, it is possible to synchronize the times of a plurality of imaging data based on a predetermined behavior of the imaging subject.

[0027] In particular, according to the fourth invention of the present application, the times of a plurality of imaging data can be synchronized based on changes in brightness of a predetermined area in the imaging data.

[0028] In particular, according to the sixth invention of the present application, the times of a plurality of imaging data can be synchronized with higher accuracy.

[0029] In particular, according to the seventh invention of the present application, the times of a plurality of imaging data can be synchronized with higher accuracy.

[0030] In particular, according to the ninth invention of the present application, the times of a plurality of imaging data can be continuously synchronized. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a top view of the substrate processing apparatus.

[0032] Figure 2 It is a longitudinal sectional view of the processing unit.

[0033] Figure 3 This is a diagram conceptually showing a situation in which a camera performs photographing.

[0034] Figure 4 This is a diagram conceptually showing a situation in which a camera performs photographing.

[0035] Figure 5 It is a block diagram showing the connection between various parts within a computer and a processing unit.

[0036] Figure 6 Detailed description of the process of the substrate is given in FIG.

[0037] Figure 7 : is a flowchart showing the flow of the time synchronization process.

[0038] Figure 8 : is a flowchart showing the flow of the time synchronization process.

[0039] Fig. 9 This is a diagram conceptually showing the structure of a moving image.

[0040] Fig.10 This is a diagram conceptually showing the structure of a moving image after time synchronization. DETAILED DESCRIPTION

[0041] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0042] [1. Overall Structure of Substrate Processing Apparatus 100]

[0043] Figure 1 1 is a top view of a substrate processing apparatus 100 as an example of a manufacturing apparatus of the present invention. The substrate processing apparatus 100 is an apparatus that supplies a processing liquid to the surface of a disk-shaped substrate W (silicon wafer) to process the surface of the substrate W in a manufacturing process of a semiconductor wafer. Figure 1 As shown, the substrate processing apparatus 100 includes an indexer 101 , a plurality of processing units 102 , and a main transfer robot 103 .

[0044] The indexer 101 is a part for carrying in a substrate W before processing from the outside and carrying out the processed substrate W to the outside. The indexer 101 is provided with a plurality of carriers for accommodating a plurality of substrates W. In addition, the indexer 101 has a transport robot (not shown). The transport robot transports the substrate W between the carrier in the indexer 101 and the processing unit 102 or the main transport robot 103.

[0045] The processing unit 102 is a so-called single-sheet processing unit that processes substrates W piece by piece. A plurality of processing units 102 are arranged around the main transport robot 103. In the present embodiment, the four processing units 102 arranged around the main transport robot 103 are stacked in three layers in the height direction. That is, the substrate processing apparatus 100 of the present embodiment has twelve processing units 102 in total. A plurality of substrates W are processed in parallel in each processing unit 102. However, the number of processing units 102 provided in the substrate processing apparatus 100 is not limited to twelve, and may be, for example, one, four, eight, twenty-four, etc.

[0046] The main transport robot 103 is a mechanism for transporting substrates W between the indexer 101 and the plurality of processing units 102. The main transport robot 103 has, for example, a hand for holding the substrate W and an arm for moving the hand. The main transport robot 103 takes out the substrate W before processing from the indexer 101 and transports it to the processing unit 102. In addition, when the processing of the substrate W in the processing unit 102 is completed, the main transport robot 103 takes out the processed substrate W from the processing unit 102 and transports it to the indexer 101.

[0047] [2. Configuration of Processing Unit 102]

[0048] Next, a detailed configuration of the processing unit 102 will be described. Although the following description focuses on one processing unit 102 among the plurality of processing units 102 included in the substrate processing apparatus 100, the other processing units 102 also have the same configuration.

[0049] Figure 2 is a longitudinal sectional view of the processing unit 102. Figure 2 As shown, the processing unit 102 includes a chamber 10 , a substrate holding unit 20 , a rotating mechanism 30 , a processing liquid supply unit 40 , a processing liquid collecting unit 50 , a baffle plate 60 , a first camera 71 , a second camera 72 , a third camera 73 and a computer 80 .

[0050] The chamber 10 is a frame having a processing space 11 for processing a substrate W. The chamber 10 includes a side wall 12 surrounding the side of the processing space 11, a top plate 13 covering the upper part of the processing space 11, and a bottom plate 14 covering the lower part of the processing space 11. The substrate holding part 20, the rotating mechanism 30, the processing liquid supply part 40, the processing liquid collecting part 50, the baffle plate 60, and the first to third cameras 71 to 73 are accommodated inside the chamber 10. A portion of the side wall 12 is provided with a loading and unloading port for loading and unloading the substrate W into the chamber 10 and unloading the substrate W from the chamber 10, and a shutter for opening and closing the loading and unloading port.

[0051] The substrate holding unit 20 holds the substrate W horizontally (with its normal line facing the vertical direction) inside the chamber 10. Figure 2 As shown, the substrate holding part 20 has a disc-shaped rotating base 21 and a plurality of chuck pins 22. The plurality of chuck pins 22 are arranged at equal angles along the outer periphery of the upper surface of the rotating base 21. The substrate W is held by the plurality of chuck pins 22 in a state where the processed surface with the pattern formed thereon faces upward. Each chuck pin 22 contacts the lower surface and the outer peripheral end surface of the peripheral portion of the substrate W, and supports the substrate W at an upper position with a small gap from the upper surface of the rotating base 21.

[0052] A chuck pin switching mechanism 23 for switching the positions of the chuck pins 22 is provided inside the spin base 21. The chuck pin switching mechanism 23 switches the chuck pins 22 between a holding position for holding the substrate W and a release position for releasing the substrate W.

[0053] The rotating mechanism 30 is a mechanism for rotating the substrate holding portion 20. The rotating mechanism 30 is housed in a motor housing 31 provided below the rotating base 21. Figure 2 As shown by the dotted line in FIG. 3 , the rotating mechanism 30 includes a rotating motor 32 and a supporting shaft 33. The supporting shaft 33 extends in the vertical direction, the lower end of the supporting shaft 33 is connected to the rotating motor 32, and the upper end of the supporting shaft 33 is fixed to the center of the lower surface of the rotating base 21. When the rotating motor 32 is driven, the supporting shaft 33 rotates around the axis 330 of the supporting shaft 33. Moreover, the substrate holding part 20 and the substrate W held by the substrate holding part 20 also rotate together with the supporting shaft 33 around the axis 330.

[0054] The processing liquid supply unit 40 supplies the processing liquid to the upper surface of the substrate W held by the substrate holding unit 20. The processing liquid supply unit 40 includes an upper surface nozzle 41 and a lower surface nozzle 42. Figure 1 as well as Figure 2As shown, the upper surface nozzle 41 has a nozzle arm 411, a nozzle head 412 and a nozzle motor 413. The nozzle head 412 is arranged at the front end of the nozzle arm 411. The nozzle arm 411 is driven by the nozzle motor 413 to rotate in the horizontal direction with the base end of the nozzle arm 411 as the center. Figure 1 The position of the double-dotted line in the figure) and the retreat position ( Figure 1 The processing position ( Figure 1 The position of the double-dashed line in FIG. 1 is the position above the substrate W held by the substrate holding portion 20. The retracted position ( Figure 1 The position of the solid line in FIG. 5 is located outside the processing liquid collecting portion 50.

[0055] The nozzle head 412 is connected to a liquid supply unit (not shown) for supplying a processing liquid. The processing liquid used may include, for example, SPM cleaning liquid (a mixture of sulfuric acid and hydrogen peroxide solution), SC-1 cleaning liquid (a mixture of ammonia water, hydrogen peroxide solution, and pure water), SC-2 cleaning liquid (a mixture of hydrochloric acid, hydrogen peroxide solution, and pure water), DHF cleaning liquid (diluted hydrofluoric acid), pure water (deionized water), etc. When the valve of the liquid supply unit is opened with the nozzle head 412 disposed at the processing position, the processing liquid supplied from the supply unit is ejected from the nozzle head 412 toward the upper surface of the substrate W held by the substrate holding unit 20.

[0056] The nozzle head 412 may be a so-called two-fluid nozzle that mixes the processing liquid with the pressurized gas to generate droplets and sprays the mixed fluid of the droplets and the gas onto the substrate W. In addition, a plurality of upper surface nozzles 41 may be provided in one processing unit 102 .

[0057] The lower surface nozzle 42 is arranged inside the through hole provided at the center of the rotating base 21. The spray port of the lower surface nozzle 42 faces the lower surface of the substrate W held by the substrate holding portion 20. The lower surface nozzle 42 is also connected to a liquid supply portion for supplying a processing liquid. When the processing liquid is supplied from the liquid supply portion to the lower surface nozzle 42, the processing liquid is sprayed toward the lower surface of the substrate W from the lower surface nozzle 42.

[0058] The treatment liquid collection part 50 is a part for collecting the treatment liquid after use. Figure 2 As shown, the processing liquid collecting section 50 includes an inner cover 51, a middle cover 52, and an outer cover 53. The inner cover 51, the middle cover 52, and the outer cover 53 can be independently moved up and down between an ascending position and a descending position by a lifting mechanism (not shown). The ascending position is a position where the upper end is arranged above the substrate W, and the descending position is a position where the upper end is arranged below the substrate W. Figure 2 In the embodiment, the inner cover 51, the middle cover 52 and the outer cover 53 are all arranged in the lowered position.

[0059] The inner cover 51 has a first annular guide plate 510 surrounding the substrate holding portion 20. The middle cover 52 has a second annular guide plate 520 located outside and above the first guide plate 510. The outer cover 53 has a third annular guide plate 530 located outside and above the second guide plate 520. In addition, the bottom of the inner cover 51 extends below the middle cover 52 and the outer cover 53. In addition, the first drain groove 511, the second drain groove 512, and the third drain groove 513 are sequentially provided on the upper surface of the bottom of the inner cover 51 from the inside.

[0060] After being supplied to the substrate W, the processing liquid sprayed from the upper surface nozzle 41 and the lower surface nozzle 42 of the processing liquid supply unit 40 is scattered toward the outside by the centrifugal force generated by the rotation of the substrate W. Moreover, the processing liquid scattered from the substrate W is captured by any one of the first guide plate 510, the second guide plate 520, and the third guide plate 530. When the inner cover 51, the middle cover 52, and the outer cover 53 are all in the ascending position, the processing liquid scattered from the substrate W is captured by the first guide plate 510. The processing liquid captured by the first guide plate 510 passes through the first liquid drain groove 511 and is discharged to the outside of the processing unit 102. In addition, when the inner cover 51 is in the descending position and the middle cover 52 and the outer cover 53 are in the ascending position, the processing liquid scattered from the substrate W is captured by the second guide plate 520. The processing liquid captured by the second guide plate 520 passes through the second liquid drain groove 512 and is discharged to the outside of the processing unit 102. When the inner cover 51 and the middle cover 52 are in the lowered position and the outer cover 53 is in the raised position, the processing liquid scattered from the substrate W is collected by the third guide plate 530. The processing liquid collected by the third guide plate 530 passes through the third drain groove 513 and is discharged to the outside of the processing unit 102.

[0061] Thus, the processing unit 102 has a plurality of processing liquid discharge paths. Therefore, the processing liquid supplied to the substrate W can be classified into various types and recovered. Therefore, the recovered processing liquid can be disposed of and recycled separately according to the properties of each processing liquid.

[0062] The baffle plate 60 is a member for suppressing diffusion of gas near the surface of the substrate W when a part of the process such as drying is performed. The baffle plate 60 has a circular plate-like shape and is horizontally arranged above the substrate holding portion 20. Figure 2As shown in FIG. 1 , the blocking plate 60 is connected to the lifting mechanism 61. When the lifting mechanism 61 is operated, the blocking plate 60 is lifted and lowered between an upper position where the blocking plate 60 has moved upward from the upper surface of the substrate W held by the substrate holding portion 20 and a lower position where the blocking plate 60 is closer to the upper surface of the substrate W than the upper position. The lifting mechanism 61 uses a mechanism that converts the rotational motion of a motor into a linear motion, for example, using a ball screw.

[0063] Furthermore, a blowout port 62 for blowing out a drying gas (hereinafter referred to as "drying gas") is provided at the center of the lower surface of the baffle plate 60. The blowout port 62 is connected to a gas supply unit (not shown) for supplying the drying gas. For example, heated nitrogen gas is used as the drying gas.

[0064] When the processing liquid is supplied to the substrate W from the upper surface nozzle 41, the baffle plate 60 is retracted to the upper position. After the processing liquid is supplied, when the substrate W is dried, the baffle plate 60 is lowered to the lower position by the lifting mechanism 61. Then, the drying gas is sprayed from the blowing port 62 toward the upper surface of the substrate W. At this time, the diffusion of the gas is prevented by the baffle plate 60. As a result, the drying gas is efficiently supplied to the upper surface of the substrate W.

[0065] The first camera 71 to the third camera 73 are mechanisms for photographing a predetermined area in the chamber 10. The area photographed by the first camera 71 to the third camera 73 is referred to as a "photographing area A". The first camera 71 to the third camera 73 photographs a dynamic image including a predetermined event occurring in the photographing area A. In addition, the number of cameras provided in the processing unit 102 may be two, or may be four or more.

[0066] The first to third cameras 71 to 73 are disposed, for example, at positions close to the inner surface of the side wall 12 of the chamber 10. Figure 2 As shown, the first to third cameras 71 to 73 are respectively covered by housings 74 to 76. Thus, the first to third cameras 71 to 73 are protected from the treatment liquid and the gas generated by the volatilization of the treatment liquid.

[0067] Figure 3 This is a diagram conceptually showing how the first to third cameras 71 to 73 in the first embodiment described below perform imaging. In the first embodiment, the first to third cameras 71 to 73 capture images of movement of the nozzle head 412 in a predetermined imaging region A including the nozzle head 412.

[0068] Figure 4This is a diagram conceptually showing a situation where the first camera 71 to the third camera 73 in the second embodiment described later perform photographing. In the second embodiment, the processing unit 102 includes a light emitting unit 90. The light emitting unit 90 is a mechanism that instantly emits light to the first camera 71 to the third camera 73. In the second embodiment, the first camera 71 to the third camera 73 photograph the light emitted by the light emitting unit 90.

[0069] In addition, as long as the change in brightness in the imaging area A is recorded in the dynamic image captured by the first camera 71 to the third camera 73 , the imaging area A may or may not include the light emitting unit 90 .

[0070] Thus, in any one of the first embodiment and the second embodiment, the first camera 71 obtains the first dynamic image M1, the second camera 72 obtains the second dynamic image M2, and the third camera 73 obtains the third dynamic image M3. Furthermore, the first camera 71 sends the obtained first dynamic image M1 to the computer 80. The second camera 72 sends the obtained second dynamic image M2 to the computer 80. Furthermore, the third camera 73 sends the obtained third dynamic image M3 to the computer 80. Furthermore, the first dynamic image M1 to the third dynamic image M3 are the "shooting data" of the present invention.

[0071] The computer 80 controls the operation of each component in the processing unit 102 . Figure 5 FIG. 8 is a block diagram showing the electrical connections between the computer 80 and the various components within the processing unit 102. Figure 2 As conceptually shown in FIG. 8 , the computer 80 includes a processor 81 such as a CPU, a memory 82 such as a RAM, and a storage unit 83 such as a hard disk drive. Figure 5 As conceptually shown in FIG. 8 , the computer 80 includes a control unit 84 .

[0072] like Figure 5 As shown, the computer 80 is respectively connected to the chuck pin switching mechanism 23, the rotating motor 32, the nozzle motor 413, the valve of the processing liquid supply part 40, the lifting mechanism of the processing liquid collecting part 50, the lifting mechanism 61 of the baffle plate 60 and the first camera 71 to the third camera 73 described above through wired or wireless communication.

[0073] The storage unit 83 stores a program P as a computer program for controlling the operation of each part of the processing unit 102 .

[0074] In addition, the storage unit 83 stores a first reference image RI1, a second reference image RI2, and a third reference image RI3. The first reference image RI1 to the third reference image RI3 are used as references for detecting first event images EI1 to third event images EI3 described later in the time synchronization process described later. The first reference image RI1 to the third reference image RI3 will be described in detail later.

[0075] The control unit 84 is implemented by the processor 81 reading the program P stored in the storage unit 83 and developing it in the memory 82 , and executing processing according to the program P. The control unit 84 includes a substrate processing control unit 841 and a time synchronization unit 842 .

[0076] The substrate processing control unit 841 executes processing of the substrate W in the processing unit 102. Details of the processing of the substrate W in the processing unit 102 will be described later.

[0077] The time synchronization unit 842 executes a time synchronization process for synchronizing the times assigned to the first to third moving images M1 to M3 . Details of the time synchronization process will be described later.

[0078] After the time synchronization unit 842 performs the time synchronization process, the control unit 84 controls the operation of each unit in the processing unit 102 based on the first to third moving images M1 to M3 that are time-synchronized by the time synchronization process.

[0079] [3. Operation of Substrate Processing Apparatus 100]

[0080] Next, the processing of the substrate W in the processing unit 102 executed by the substrate processing control section 841 will be described. Figure 6 1 is a flowchart showing the processing procedure of the substrate W.

[0081] When processing the substrate W in the processing unit 102, first, the main transport robot 103 carries the substrate W to be processed into the chamber 10 (step S101). The substrate W carried into the chamber 10 is held horizontally by the plurality of chuck pins 22 of the substrate holding portion 20. Thereafter, the rotation motor 32 of the rotation mechanism 30 is driven to start rotating the substrate W (step S102). Specifically, the support shaft 33, the rotation base 21, the plurality of chuck pins 22, and the substrate W held by the chuck pins 22 rotate around the axis 330 of the support shaft 33 as the center.

[0082] Next, the processing liquid is supplied from the processing liquid supply unit 40 (step S103). In step S103, the nozzle head 412 is moved toward the processing position opposite to the upper surface of the substrate W by driving the nozzle motor 413. Next, the processing liquid is ejected from the nozzle head 412 arranged at the processing position. Parameters such as the ejection speed and ejection time of the processing liquid are pre-set in the storage unit 83. The substrate processing control unit 841 executes the action of ejecting the processing liquid from the upper surface nozzle 41 according to the setting.

[0083] In step S103, the upper surface nozzle 41 may be swung in the horizontal direction at the processing position while the processing liquid is ejected from the upper surface nozzle 41. In addition, the processing liquid may be ejected from the lower surface nozzle 42 as needed.

[0084] During the process liquid supply process of step S103, the baffle plate 60 is arranged at an upper position above the upper surface nozzle 41. When the supply of the process liquid to the substrate W is finished and the upper surface nozzle 41 is arranged at the retreat position, the computer 80 operates the lifting mechanism 61 to move the baffle plate 60 from the upper position to the lower position. Next, the rotation speed of the rotary motor 32 is increased to increase the rotation speed of the substrate W, and the drying gas is sprayed toward the substrate W from the blowing port 62 provided on the lower surface of the baffle plate 60. Thus, the surface of the substrate W is dried (step S104).

[0085] When the drying process of the substrate W is completed, the rotation motor 32 is stopped to stop the rotation of the substrate W. Next, the plurality of chuck pins 22 release the substrate W from being held. Thereafter, the main transport robot 103 takes out the processed substrate W from the substrate holding portion 20 and carries it out of the chamber 10 (step S105).

[0086] Each processing unit 102 repeatedly executes the processing of step S101 to step S105 described above on a plurality of substrates W that are sequentially conveyed.

[0087] [4. Time synchronization processing]

[0088] Next, the time synchronization process performed by the time synchronization unit 842 is described. The time synchronization process is a process for synchronizing the time given to the first to third moving images M1 to M3 respectively captured by the first to third cameras 71 to 73. The flow of the time synchronization process in the first and second embodiments is described below.

[0089] [4-1. First embodiment]

[0090] Figure 7Flowchart showing the flow of the moment synchronization process in the first embodiment. In the first embodiment, the passage of the nozzle head 412 through a predetermined portion in the shooting area A is defined as an "event". In the first embodiment, the first camera 71 to the third camera 73 shoot the movement of the nozzle head 412 in step S103, and obtain the first dynamic image M1 to the third dynamic image M3 (step S201). In addition, the movement of the nozzle head 412 captured in step S201 may not be performed in step S103. For example, the movement of the nozzle head 412 may be performed in advance before executing steps S101 to S105.

[0091] The first to third cameras 71 to 73 transmit the first moving image M1, the second moving image M2, and the third moving image M3 respectively acquired in step S201 to the computer 80. The first to third moving images M1 to M3 transmitted to the computer 80 are stored in the storage unit 83.

[0092] Next, the time synchronization unit 842 detects the first event image EI1 to the third event image EI3 from the first moving image M1 to the third moving image M3 stored in the storage unit 83 , respectively (step S202 ). Fig. 9 FIG. 2 is a diagram conceptually showing the structure of the first dynamic image M1 to the third dynamic image M3. Fig. 9 As shown, the first dynamic image M1 is composed of a plurality of frame images F1 (F11, F12, ..., F18, ...) shot at small time intervals. The second dynamic image M2 and the third dynamic image M3 are also composed of a plurality of frame images F2 (F21, F22, ..., F28, ...) and a plurality of frame images F3 (F31, F32, ..., F38, ...). In addition, the frame images F1 to F3 are equivalent to the "shot images" of the present invention.

[0093] In step S202 , first to third event images EI1 to EI3 are detected from the first to third moving images M1 to M3 based on the first to third reference images RI1 to RI3 stored in the storage unit 83 .

[0094] The first reference image RI1 to the third reference image RI3 record the moment when the nozzle head 412, which is defined as an "event" in the first embodiment, passes through a predetermined portion in the imaging area A. In addition, the first reference image RI1 is captured in the same imaging range and the same imaging angle as the first dynamic image M1. In addition, the second reference image RI2 is captured in the same imaging range and the same imaging angle as the second dynamic image M2. In addition, the third reference image RI3 is captured in the same imaging range and the same imaging angle as the third dynamic image M3.

[0095] The time synchronization unit 842 compares each of the plurality of frame images F1 with the first reference image RI1 for a predetermined area of ​​each image. Next, the time synchronization unit 842 detects the frame image F1 most similar to the first reference image RI1 as the first event image EI1. Fig. 9 In the example shown, the frame image F12 is detected as the first event image EI1 .

[0096] Similar to the first moving image M1, the second event image EI2 and the third event image EI3 are detected from the second moving image M2 and the third moving image M3, respectively. Fig. 9 In the illustrated example, the frame image F23 and the frame image F35 are detected as the second event image EI2 and the third event image EI3 , respectively.

[0097] Furthermore, in step S202 , the time synchronization unit 842 may detect the first event image EI1 to the third event image EI3 using a predetermined image processing technique such as an object detection algorithm.

[0098] Fig.10 2 is a diagram conceptually showing the structure of a moving image after the time information is corrected. After step S202, the time synchronization unit 842 synchronizes the time assigned to the first moving image M1 to the third moving image M3 based on the time assigned to the first event image EI1 to the third event image EI3 (step S203). In addition, the time assigned to the first moving image M1 to the third moving image M3 is, for example, time information based on the internal clock of each of the first camera 71 to the third camera 73.

[0099] Specifically, first, the time synchronization unit 842 selects the image with the latest time from among the first event image EI1 to the third event image EI3 as a reference for time synchronization. Fig. 9 In the illustrated example, the time synchronization unit 842 selects the third event image EI3 as a reference.

[0100] Next, the time synchronization unit 842 corrects the time assigned to the first dynamic image M1 and the time assigned to the second dynamic image M2 so that the time assigned to the first event image EI1 and the second event image EI2 is consistent with the time assigned to the third event image EI3. Fig.10 As shown, the time points assigned to the first dynamic image M1 to the third dynamic image M3 are synchronized.

[0101] In addition, each time each processing unit 102 repeatedly performs step S102 on a plurality of substrates W that are sequentially transported, steps S201 to S203 may be repeatedly performed. Alternatively, after the nozzle head 412 is operated at a predetermined time, the processing of steps S201 to S203 may be repeatedly performed at a predetermined time interval. Thus, the time given to the first dynamic image M1 to the third dynamic image M3 can be synchronized at a predetermined time interval. Therefore, the time between the first dynamic image M1 to the third dynamic image M3 can be continuously synchronized.

[0102] [4-2. Second embodiment]

[0103] Figure 8 1 is a flowchart showing the process of the moment synchronization process in the second embodiment. In the second embodiment, the brightness of a predetermined area in the first dynamic image M1 to the third dynamic image M3 reaches a predetermined threshold value, which is defined as an "event". In the second embodiment, first, the first camera 71 to the third camera 73 shoot the dynamic image in the shooting area A (step S301).

[0104] The light emitting unit 90 emits light instantaneously at any time during the execution of step S301. As a result, the light emitted from the light emitting unit 90 is recorded in the first dynamic image M1 to the third dynamic image M3. The first camera 71 to the third camera 73 transmit the first dynamic image M1 to the third dynamic image M3 respectively acquired to the computer 80. The first dynamic image M1 to the third dynamic image M3 transmitted to the computer 80 are stored in the storage unit 83.

[0105] Next, the time synchronization unit 842 detects the first event image EI1 to the third event image EI3 from the first moving image M1 to the third moving image M3 stored in the storage unit 83 , respectively (steps S302 to S304 ).

[0106] Steps S302 to S304 are described using the first dynamic image M1 as an example. First, the timing synchronization unit 842 extracts an image whose brightness of pixels in a predetermined region is greater than a predetermined threshold value from among the plurality of frame images F1 constituting the first dynamic image M1 (step S302). Hereinafter, the image extracted in step S302 is referred to as an "extracted image".

[0107] When there is only one extracted image (No in step S303 ), the time synchronization unit 842 detects the extracted image as the event image EI.

[0108] At this time, there may be a case where a plurality of continuous frame images F1 are extracted due to the light emission time or the threshold value of the light emitting unit 90. That is, in this case, in the first dynamic image M1, events are recorded in a plurality of extracted images. When there are a plurality of extracted images (step S303 is yes), the time synchronization unit 842 selects one extracted image from the plurality of extracted images as the first event image EI1 for detection (step S304).

[0109] For example, the time synchronization unit 842 may select the first event image EI1 based on the brightness of the predetermined area in the plurality of extracted images. Specifically, the time synchronization unit 842 may select an extracted image having the highest average brightness of the plurality of pixels constituting the predetermined area in the plurality of extracted images as the first event image EI1.

[0110] Alternatively, the time synchronization unit 842 may select the first event image EI1 based on the times assigned to the plurality of extracted images. Specifically, the time synchronization unit 842 may select an extracted image assigned a time closest to the average of the times assigned to the plurality of extracted images as the first event image EI1.

[0111] Similar to the first moving image M1 , the time synchronization unit 842 executes steps S302 to S304 for the second moving image M2 and the third moving image M3 , thereby detecting the second event image EI2 and the third event image EI3 , respectively.

[0112] Next, the time synchronization unit 842 synchronizes the time given to the first to third moving images M1 to M3 based on the time given to the first to third event images EI1 to EI3 (step S305 ). Step S305 is the same as step S203 of the first embodiment, and thus its description is omitted.

[0113] In addition, the light emitting unit 90 may emit light periodically at a predetermined time interval. In this case, the time synchronization unit 842 periodically performs steps S302 to S305. Thus, the time given to the first dynamic image M1 to the third dynamic image M3 can be synchronized at a predetermined time interval. Therefore, the time between the first dynamic image M1 to the third dynamic image M3 can be continuously synchronized.

[0114] As described above, in the substrate processing apparatus 100, frame images F1 to F3 in which a predetermined event is captured are detected from each of the plurality of first to third dynamic images M1 to M3 as first event images EI1 to third event images EI3. Then, based on the time of the first event image EI1 to third event image EI3 assigned to each of the first to third dynamic images M1 to M3, the time between the plurality of first to third dynamic images M1 to M3 is synchronized. Thus, the time between the plurality of first to third dynamic images M1 to M3 can be synchronized without performing communication between the substrate processing apparatus 100 and an external network.

[0115] Furthermore, in step S202 of the first embodiment, when a plurality of images corresponding to the first event image EI1 to the third event image EI3 are detected for one dynamic image M, the plurality of images may be used as extracted images and one first event image EI1 to the third event image EI3 may be detected in the same order as step S304 of the second embodiment. Specifically, the time synchronization unit 842 may select, as the first event image EI1 to the third event image EI3, one extracted image to which a time closest to an average value is assigned, which is an average value of the times assigned to the plurality of extracted images extracted in step S202.

[0116] In the second embodiment, the first to third event images EI1 to EI3 may be detected in the same order as step S202 in the first embodiment. In this case, the time synchronization unit 842 may detect the first to third event images EI1 to EI3 based on the first to third reference images RI1 to RI3 that respectively record the moments when the light emitting unit 90 emits light.

[0117] [5. Modifications]

[0118] Although one embodiment of the present invention has been described above, the present invention is not limited to the above embodiment.

[0119] In the above-mentioned embodiment, the moments in the plurality of dynamic images M of the substrate processing device 100 are synchronized. However, the moment synchronization method in this specification can be applied to various devices. For example, the moment synchronization method in this specification can be applied to a printing device. In this case, for example, a predetermined resist mark printed on a recording medium passing through a predetermined position can be defined as an "event".

[0120] In the above-mentioned first embodiment, the passage of the nozzle arm 411 through a predetermined portion in the shooting area A is defined as an "event". In addition, in the above-mentioned second embodiment, the brightness of a predetermined area in the first dynamic image M1 to the third dynamic image M3 reaching a predetermined threshold is defined as an "event". However, the content of the event is not limited to the matters defined in the above-mentioned embodiments. For example, the event may be a predetermined action performed by the substrate processing device 100, such as the action of the nozzle head 412 spraying a processing liquid. Alternatively, the event may be that the pixel value of a predetermined area in the first dynamic image M1 to the third dynamic image M3 changes due to a change in the amount of light emitted by the light emitting unit 90 or a change in color.

[0121] In the above-mentioned embodiment, the first camera 71 to the third camera 73 send the first dynamic images M1 to the third dynamic images M3 to the computer 80 provided in the processing unit 102. Then, the time synchronization process is performed by the time synchronization unit 842 of the computer 80. However, the first camera 71 to the third camera 73 may respectively include a computer for processing the first dynamic images M1 to the third dynamic images M3. After executing steps S201 to S202 in the first embodiment or steps S301 to S304 in the second embodiment on each of the first dynamic images M1 to the third dynamic images M3, each computer provided in the first camera 71 to the third camera 73 may respectively send the first event image EI1 to the third event image EI3 and each of the first dynamic images M1 to the third dynamic images M3 to the computer 80. Thereafter, step S203 or step S305 may be performed by the time synchronization unit 842.

[0122] In the above-mentioned embodiment, the first camera 71 to the third camera 73 respectively capture the first dynamic image M1 to the third dynamic image M3. However, the first camera 71 to the third camera 73 may intermittently capture a plurality of images at predetermined time intervals. In this case, in the time synchronization process, the time synchronization unit 842 may respectively detect the first event image EI1 to the third event image EI3 from the group of the plurality of images respectively captured by the first camera 71 to the third camera 73. Moreover, the time synchronization unit 842 may synchronize the time assigned to the group of the plurality of images based on the time assigned to the first event image EI1 to the third event image EI3.

[0123] The respective elements described in the above-mentioned embodiment and the modified examples may be appropriately combined within a range where no contradiction occurs.

[0124] Description of Reference Numerals

[0125] 40: Processing liquid supply unit

[0126] 41: Upper surface nozzle

[0127] 71: First Camera

[0128] 72: Second Camera

[0129] 73: Third Camera

[0130] 90: Luminous part

[0131] 100: Substrate processing device

[0132] 411: Nozzle arm

[0133] 412: Nozzle head

[0134] 841: Substrate processing control unit

[0135] 842: Time Synchronization Department

[0136] A: Shooting area

[0137] EI1: First Event Image

[0138] EI2: Second Event Image

[0139] EI3: Third Event Image

[0140] F1: Frame image

[0141] F2: Frame Image

[0142] F3: Frame Image

[0143] M1: First Moving Image

[0144] M2: Second dynamic image

[0145] M3: Third moving image

[0146] W: substrate.

Claims

1. A time synchronization method for synchronizing the time assigned to a plurality of shooting data, wherein: have: Step a, photographing the subject by using a plurality of cameras, thereby obtaining the photographing data from each of the cameras; Step b, for each of the photographic data, detecting an event image in which a predetermined event is photographed from the photographic data; as well as Step c is to synchronize the time points assigned to the plurality of pieces of imaging data based on the time points assigned to the respective event images.

2. The time synchronization method according to claim 1, wherein: The event is that a predetermined portion included in the photographic object passes through a predetermined location in the photographing areas of the plurality of cameras.

3. The time synchronization method according to claim 2, wherein: In the step b, the event image is detected based on a reference image serving as a reference for detection.

4. The time synchronization method according to claim 1, wherein: The event is that the brightness of a predetermined area in the photographic data reaches a predetermined threshold.

5. The time synchronization method according to claim 4, wherein: The step a includes the step of emitting light toward imaging areas of the plurality of cameras at a predetermined timing during the period in which the imaging object is imaged by the plurality of cameras.

6. The time synchronization method according to claim 4 or 5, wherein: The step b is the following step, In one of the photographic data, when one photographic image captures the event, detecting the one photographic image as the event image, In one of the photographic data, when the event is photographed in a plurality of photographic images, one of the photographic images is detected as the event image from among the plurality of photographic images based on the brightness of the plurality of photographic images.

7. The time synchronization method according to any one of claims 1 to 5, wherein: The step b is the following step, In one of the photographic data, when one photographic image captures the event, detecting the one photographic image as the event image, In the one of the photographic data, when the event is photographed in a plurality of photographic images, one of the photographic images is detected as the event image from among the plurality of photographic images based on the times respectively assigned to the plurality of photographic images.

8. The time synchronization method according to any one of claims 1 to 7, wherein: The shooting data is a dynamic image. The event image is a frame included in the dynamic image.

9. The time synchronization method according to any one of claims 1 to 8, wherein: The step a, the step b, and the step c are repeatedly performed at predetermined time intervals.

10. A manufacturing device, which performs processing based on a plurality of photographic data, wherein: have: Multiple cameras to photograph the subject, and Control Department; The control unit performs: Step a, causing the plurality of cameras to photograph the subject, thereby obtaining the photographing data from each of the cameras; Step b, for each of the photographic data, detecting an event image in which a predetermined event is photographed from a plurality of photographic images included in the photographic data; as well as Step c is to synchronize the time points assigned to the plurality of pieces of imaging data based on the time points assigned to the respective event images.

Images