Substrate processing apparatus and information processing method of substrate processing apparatus

Abstract

In order to efficiently recognize the posture of a component in a substrate processing apparatus, reference shape information relating to the two-dimensional shape of a 3D model is acquired for each of a plurality of first virtual camera positions in a plurality of virtual surfaces along a virtual spherical surface of the 3D model surrounding the target component on the basis of design information of the target component. For each first virtual camera position, the degree of coincidence between the reference shape information and the actual shape information relating to the two-dimensional shape of the object in the actual image is calculated. A high-degree-of-coincidence virtual camera position with the largest degree of coincidence among the plurality of first virtual camera positions is detected. A virtual plane in which a high-consistency virtual camera position is set is divided to generate a plurality of virtual divided planes. Reference shape information relating to the two-dimensional shape of the 3D model is generated for each of the plurality of second virtual camera positions in the plurality of virtual split surfaces on the basis of the design information of the target component. For each second virtual camera position, a numerical value indicating the degree of coincidence between the actual shape information and the reference shape information is calculated.

Description

TECHNICAL FIELD

[0001] The present application relates to a technology of recognizing a posture of a component in a substrate processing apparatus that performs processing of a substrate. The substrate that becomes a processing target in the substrate processing apparatus includes, for example, a semiconductor substrate, a flat panel display (FPD) substrate for a liquid crystal display device or an organic EL (Electroluminescence) display device, a glass substrate for a photomask, a substrate for an optical disc, a substrate for a magnetic disc, or a substrate for a solar cell. BACKGROUND

[0002] The substrate processing apparatus has a processing chamber, a substrate holding portion, a nozzle, a camera, an image processing portion, and a monitoring portion (for example, refer to Patent Literature 1).

[0003] In the substrate processing apparatus, the substrate holding portion, the nozzle, and the camera are arranged in the processing chamber. The substrate holding portion holds the substrate in a horizontal posture. The substrate holding portion rotates the substrate in a horizontal plane. The nozzle moves between a standby position that is offset to a side of the substrate and a spray position that is above the substrate, by rotation of a drive arm on which the nozzle is fixed. The nozzle is arranged at the standby position when the substrate is attached to or detached from the substrate holding portion, and is arranged at the spray position when processing liquid is sprayed from the nozzle toward the substrate. The camera is installed at a predetermined position in the processing chamber, and captures a predetermined area including the nozzle that moves to the spray position. The image processing portion acquires second nozzle position information that indicates a position of the nozzle, based on an image from the camera, and outputs the second nozzle position information to the monitoring portion. The monitoring portion judges whether or not there is an abnormality in the position of the nozzle, based on correspondence between first nozzle position information that is information from the control portion indicating a position at which the nozzle is arranged directly or indirectly, and the second nozzle position information from the image processing portion.

[0004] PRIOR ART DOCUMENTS

[0005] PATENT LITERATURE

[0006] Patent Literature 1: International Publication No. 2019 / 146456 SUMMARY

[0007] [PROBLEMS TO BE SOLVED BY THE INVENTION]

[0008] However, with respect to the substrate processing apparatus, there is room for improvement in terms of efficiently performing recognition of the posture of the component.

[0009] [MEANS OF SOLVING THE PROBLEMS]

[0010] The substrate processing apparatus of the first aspect is a substrate processing apparatus that performs processing of a substrate, and includes: a storage unit that stores three-dimensional design information related to an object part; a photographing unit that obtains an actual image that captures the object part by photographing; and a search processing unit that searches for a virtual camera position at which a degree of coincidence between reference shape information and actual shape information is greatest, among a plurality of virtual camera positions, based on the reference shape information and the actual shape information, the reference shape information being information related to a two-dimensional shape of a three-dimensional model in each of a plurality of virtual images that can be obtained by photographing the three-dimensional model from the plurality of virtual camera positions, the virtual camera positions being respectively generated based on the three-dimensional design information, the actual shape information being information related to a two-dimensional shape of an object in the actual image, the search processing unit including: a first shape information acquisition unit that acquires the reference shape information generated for each of a plurality of first virtual camera positions, based on the three-dimensional design information, assuming a case in which the three-dimensional model is photographed from each of the plurality of first virtual camera positions, the plurality of first virtual camera positions being a plurality of virtual camera positions that are virtually set by virtually setting one virtual camera position for each of a plurality of virtual surfaces that virtually surround the three-dimensional model along a virtual sphere that surrounds the three-dimensional model with a reference point of the three-dimensional model as a center; a first calculation unit that calculates, for each of the plurality of first virtual camera positions, a value that indicates a degree of coincidence between the actual shape information and the reference shape information; a first detection unit that detects, based on a result of the calculation by the first calculation unit, a first virtual camera position at which the degree of coincidence between the actual shape information and the reference shape information is greatest, among the plurality of first virtual camera positions, that is, a high coincidence virtual camera position; a divided surface generation unit that generates a plurality of virtual divided surfaces by dividing a virtual surface, among the plurality of virtual surfaces, in which the high coincidence virtual camera position is virtually set, that is, a high coincidence virtual surface; a second shape information generation unit that generates the reference shape information for each of a plurality of second virtual camera positions, based on the three-dimensional design information, assuming a case in which the three-dimensional model is photographed from each of the plurality of second virtual camera positions, the plurality of second virtual camera positions being a plurality of virtual camera positions that are virtually set by virtually setting one virtual camera position for each of the plurality of virtual divided surfaces; and a second calculation unit that calculates, for each of the plurality of second virtual camera positions, a value that indicates a degree of coincidence between the actual shape information and the reference shape information.

[0011] The substrate processing apparatus of the second aspect as described in the first aspect, wherein the first shape information acquisition section acquires the reference shape information for each of the M1 x T1 first virtual camera positions by assuming that the three-dimensional model is imaged from each of the M1 x T1 first virtual camera positions that are virtually set by virtually setting one virtual camera position for each of the T1 face assemblies whose distances from the reference point are different from each other, and calculating a value indicating the degree of coincidence between the actual shape information and the reference shape information for each of the M1 x T1 first virtual camera positions; the first calculation section calculates the value indicating the degree of coincidence between the actual shape information and the reference shape information for each of the M1 x T1 first virtual camera positions; the first detection section detects the high-coincidence virtual camera position, which is the virtual camera position in which the degree of coincidence between the actual shape information and the reference shape information is the highest, among the M1 x T1 first virtual camera positions, based on the calculation result of the first calculation section; the split face generation section splits each of the T2 virtual faces, which are included in the M1 virtual faces of each of the T1 face assemblies, are on the high-coincidence virtual camera position side from the reference point, and intersect a straight line passing through the reference point and the high-coincidence virtual camera position, and are different from each other in distance from the reference point, in the same rule, and generates M2 x T2 virtual split faces by generating M2 virtual split faces for each of the T2 virtual faces; and the second shape information generation section generates the reference shape information for each of the M2 x T2 second virtual camera positions by assuming that the three-dimensional model is imaged from each of the M2 x T2 second virtual camera positions that are virtually set by virtually setting one virtual camera position for each of the M2 x T2 virtual split faces, and generates the reference shape information for each of the M2 x T2 second virtual camera positions; and the second calculation section calculates the value indicating the degree of coincidence between the actual shape information and the reference shape information for each of the M2 x T2 second virtual camera positions.

[0012] The substrate processing apparatus of the third aspect as described in the second aspect, wherein the search processing section includes a second detection section that detects the high-coincidence virtual camera position and a virtual camera position in which the degree of coincidence between the actual shape information and the reference shape information is the highest among the M2 x T2 second virtual camera positions.

[0013] The substrate processing apparatus of the fourth aspect is the substrate processing apparatus of the third aspect, wherein the search processing section performs the first unit processing on the target part and then performs the nth unit processing (n is a natural number of 2 or more) one or more times, the search processing section sequentially performs a first A processing, a first B processing, a first C processing, and a first D processing in the first unit processing, the first A processing is a processing in which the division surface generating section divides each of a plurality of virtual division surfaces generated by dividing the T2 virtual surfaces, i.e., T3 virtual division surfaces (T3 is a natural number of 2 or more) including a virtual division surface including a first reference virtual camera position that is a virtual camera position detected by the first detection section and that is on a side of the first reference virtual camera position farther from the reference point than the reference point and that intersects a straight line passing through the reference point and the first reference virtual camera position and that is different from the reference point in distance, into M3 virtual division surfaces (M3 is a natural number of 2 or more) in the same rule to generate M3 virtual division surfaces (M3 is a natural number of 2 or more) for each of the T3 virtual division surfaces to generate M3 x T3 virtual division surfaces, i.e., M3 x T3 virtual division surfaces for the first, the first B processing is a processing in which the second shape information generating section assumes that each of M3 x T3 virtual camera positions, i.e., M3 x T3 third virtual camera positions for the first, is set by virtually setting one virtual camera position for each of the M3 x T3 virtual division surfaces for the first based on the three-dimensional design information to generate the reference shape information for each of the M3 x T3 third virtual camera positions for the first based on a situation in which the three-dimensional model is imaged from each of the M3 x T3 third virtual camera positions for the first, the first C processing is a processing in which the second calculation section calculates a value indicating a degree of coincidence between the actual shape information and the reference shape information for each of the M3 x T3 third virtual camera positions for the first, and the first D processing is a processing in which the second detection section detects a second reference virtual camera position, i.e., a virtual camera position for which the degree of coincidence between the actual shape information and the reference shape information is the greatest, among the first reference virtual camera position and the M3 x T3 third virtual camera positions for the first.The search processing section sequentially performs an nA process, an nB process, an nC process, and an nD process in each of the n unit processes of the one or more times. The nA process is a process of dividing, by the division surface generation section, each of a plurality of virtual division surfaces generated by dividing the n-1th T3 virtual division surfaces, i.e., T3 virtual division surfaces including a virtual division surface including the nth virtual camera position detected by the second detection section, i.e., the n th reference virtual camera position, and intersecting a straight line passing through the reference point and the n th reference virtual camera position on the side of the n th reference virtual camera position farther from the reference point, and having distances from the reference point different from each other, by the same rule to generate M3 virtual division surfaces, i.e., the n th M3 virtual division surfaces, from each of the n th T3 virtual division surfaces, thereby generating M3 x T3 virtual division surfaces, i.e., the n th M3 x T3 virtual division surfaces. The nB process is a process of assuming, by the second shape information generation section, that the three-dimensional model is photographed from each of M3 x T3 virtual camera positions, i.e., the n th M3 x T3 third virtual camera positions, set by virtually setting one virtual camera position for each of the n th M3 x T3 virtual division surfaces, based on the three-dimensional design information, and generating the reference shape information for each of the n th M3 x T3 third virtual camera positions. The nC process is a process of calculating, by the second calculation section, a value indicating the degree of coincidence between the actual shape information and the reference shape information for each of the n th M3 x T3 third virtual camera positions. The nD process is a process of detecting, by the second detection section, the n th reference virtual camera position and the virtual camera position having the greatest degree of coincidence between the actual shape information and the reference shape information among the n th M3 x T3 third virtual camera positions.

[0014] The substrate processing apparatus of the fifth aspect is the substrate processing apparatus of the fourth aspect, wherein the search processing section ends the execution of the n unit process of the one or more times in response to the n th reference virtual camera position being continuously detected by the second detection section for a first predetermined number of times set in advance from the first reference virtual camera position to the n th reference virtual camera position as the virtual camera position having the greatest degree of coincidence between the actual shape information and the reference shape information.

[0015] The substrate processing apparatus of the sixth aspect is the substrate processing apparatus of the fourth aspect, wherein the search processing section ends the execution of the n unit process of the one or more times in response to the n th unit process being executed for a second predetermined number of times set in advance among the n unit processes of the one or more times.

[0016] The substrate processing apparatus of the seventh aspect is the substrate processing apparatus of any one of the fourth to sixth aspects, further comprising an abnormality detection unit that compares real information related to a posture of the target part identified based on the virtual camera position at which the degree of agreement between the actual shape information and the reference shape information is the greatest in the last nth D processing of the one or more nth unit processes detected by the second detection unit, and normal information related to the posture of the target part based on the three-dimensional design information when the target part is in a normal state, and detects an abnormality of the target part.

[0017] The substrate processing apparatus of the eighth aspect is the substrate processing apparatus of any one of the second to seventh aspects, wherein the same rule includes a rule of dividing the division target face into a plurality of faces by connecting a center point of the division target face and all vertices of the division target face with a plurality of line segments, respectively.

[0018] The substrate processing apparatus of the ninth aspect is the substrate processing apparatus of any one of the first to eighth aspects, wherein each of the plurality of virtual faces is a triangular face, and the face aggregate is a polyhedron composed of a plurality of triangular faces.

[0019] The substrate processing apparatus of the tenth aspect is the substrate processing apparatus of the ninth aspect, wherein the division face generation unit divides the high-agreement virtual face into three virtual division faces as the plurality of virtual division faces by connecting three vertices of the high-agreement virtual face and three line segments of the high-agreement virtual camera position, respectively.

[0020] The information processing method of the substrate processing apparatus of the 11th aspect is an information processing method of a substrate processing apparatus that performs processing of a substrate, and includes: an actual image acquisition step of acquiring, by an arithmetic unit, an actual image of an object captured by a photographing unit; and a search step of searching, by the arithmetic unit, for a virtual camera position at which a degree of coincidence between reference shape information and actual shape information is the highest, among a plurality of virtual camera positions, on the basis of the reference shape information and the actual shape information, the reference shape information being information related to a two-dimensional shape of a three-dimensional model of the object, each of a plurality of virtual images being able to be acquired by photographing the three-dimensional model of the object from the plurality of virtual camera positions, the actual shape information being information related to a two-dimensional shape of an object in the actual image, the search step including: a first shape information acquisition step of acquiring the reference shape information generated for each of a plurality of first virtual camera positions, on the basis of the three-dimensional design information, by assuming that the three-dimensional model is photographed from each of the plurality of first virtual camera positions, the plurality of first virtual camera positions being a plurality of virtual camera positions set by virtually setting one virtual camera position for each of a plurality of virtual surfaces including a surface aggregate of a plurality of virtual surfaces that virtually surround the three-dimensional model along a virtual sphere centered on a reference point of the three-dimensional model; a first calculation step of calculating, for each of the plurality of first virtual camera positions, a value indicating the degree of coincidence between the actual shape information and the reference shape information; a first detection step of detecting, on the basis of a result of the calculation of the first calculation step, a first virtual camera position at which the degree of coincidence between the actual shape information and the reference shape information is the highest, among the plurality of first virtual camera positions, as a high-coincidence virtual camera position; a divided surface generation step of generating a plurality of virtual divided surfaces by dividing a virtual surface of the plurality of virtual surfaces in which the high-coincidence virtual camera position is virtually set, as a high-coincidence virtual surface; a second shape information generation step of generating the reference shape information for each of a plurality of second virtual camera positions, on the basis of the three-dimensional design information, by assuming that the three-dimensional model is photographed from each of the plurality of second virtual camera positions, the plurality of second virtual camera positions being a plurality of virtual camera positions set by virtually setting one virtual camera position for each of the plurality of virtual divided surfaces; and a second calculation step of calculating, for each of the plurality of second virtual camera positions, a value indicating the degree of coincidence between the actual shape information and the reference shape information.

[0021] [Effects of the Invention]

[0022] According to the substrate processing apparatus of the first embodiment, for each of the plurality of virtual segmented surfaces generated by dividing a plurality of virtual surfaces into which a virtual surface with a high-consistency virtual camera position detected by a first detection unit is set, a second virtual camera position is set, and for each second virtual camera position, a value representing the consistency between actual shape information and reference shape information is calculated. Therefore, the high-consistency virtual surface and the plurality of virtual segmented surfaces are not unrelated surfaces, and for the plurality of virtual segmented surfaces, an increase in at least one of the number and area can be reduced. As a result, the computational load for identifying the pose of an object part captured in an actual image can be reduced. Consequently, the identification of part pose can be performed efficiently in the substrate processing apparatus.

[0023] According to the substrate processing apparatus of the second method, when the distance between the imaging unit and the target part changes, the posture of the part can be identified efficiently.

[0024] According to the substrate processing apparatus of the third method, the position of a virtual camera with greater consistency between the actual shape information and the reference shape information can be detected efficiently.

[0025] According to the substrate processing apparatus of the fourth method, during repeated unit processing, a plurality of virtual segmentation surfaces, each containing a virtual camera position with the highest consistency between actual shape information and reference shape information and at different distances from the target part, are segmented to generate a plurality of virtual segmentation surfaces for setting the virtual camera position for the next step. Thus, the virtual segmentation surfaces before and after segmentation are not unrelated surfaces, and the increase in at least one of the number and area of ​​the segmented virtual segmentation surfaces can be reduced. Therefore, the computational load for identifying the pose of the target part captured in the actual image can be reduced. As a result, the pose identification of the part can be performed efficiently in the substrate processing apparatus.

[0026] According to the substrate processing apparatus of the fifth method, the identification of component orientation can be performed efficiently by reducing the amount of computation.

[0027] According to the substrate processing apparatus of the sixth method, the identification of component orientation can be performed efficiently by reducing the amount of computation.

[0028] According to the substrate processing apparatus of the seventh method, since the identification of real-world information related to the posture of components in the substrate processing apparatus can be performed efficiently and effectively, the detection of component anomalies can be performed efficiently and effectively.

[0029] According to the substrate processing apparatus of the eighth method, surface segmentation can be easily performed.

[0030] According to the substrate processing apparatus of the ninth method, it is easy to set up a surface assembly containing a plurality of virtual surfaces.

[0031] According to the substrate processing apparatus of the 10th aspect, the division of the virtual face can be easily performed.

[0032] According to the information processing method of the substrate processing apparatus of the 11th aspect, the number of virtual division faces generated by dividing the virtual face in which the high-consistency virtual camera position detected in the first detection step is set is reduced. Therefore, the number of virtual division faces can be reduced. As a result, the amount of calculation for recognizing the posture of the object part captured in the actual image can be reduced. As a result, in the substrate processing apparatus, the recognition of the posture of the part can be efficiently performed. BRIEF DESCRIPTION OF DRAWINGS

[0033] Figure 1 FIG. 1 is a side view schematically showing an example of an outline configuration of a substrate processing apparatus according to a first embodiment.

[0034] Figure 2 FIG. 2 is a plan view schematically showing an example of an outline configuration of the substrate processing apparatus according to the first embodiment.

[0035] Figure 3 FIG. 3 is a block diagram showing an example of a functional configuration of the substrate processing apparatus according to the first embodiment.

[0036] Figure 4 FIG. 4 is a block diagram showing a specific example of a functional configuration related to a search process of the control section.

[0037] Figure 5 FIG. 5 is a diagram showing a specific example of an actual image obtained by photographing by a camera.

[0038] Figure 6 FIG. 6 is a diagram showing a specific example of a processing target region of the actual image.

[0039] Figure 7 FIG. 7 is a diagram showing a specific example of a processing target actual image.

[0040] Figure 8 FIG. 8 is a diagram showing a specific example of an edge image.

[0041] Figure 9 FIG. 9 is a diagram for explaining a basic idea of a search process.

[0042] Figure 10 FIG. 10 is a diagram for explaining a basic idea of a search process.

[0043] Figure 11 is a diagram to explain the basic idea of the search processing.

[0044] Figure 12 is a diagram to explain the basic idea of the search processing.

[0045] Figure 13 is a diagram to explain the basic idea of the search processing.

[0046] Figure 14 is a diagram to explain the basic idea of the search processing.

[0047] Figure 15 is a diagram to explain the basic idea of the search processing.

[0048] Figure 16 is a diagram to explain the basic idea of the search processing.

[0049] Figure 17 is a diagram to explain the basic idea of the search processing.

[0050] Figure 18 is a diagram to explain the basic idea of the search processing.

[0051] Figure 19 is a diagram to explain the basic idea of the search processing.

[0052] Figure 20 is a diagram to explain the basic idea of the search processing.

[0053] Figure 21 is a diagram to explain the basic idea of the search processing.

[0054] Figure 22 is a diagram to explain the basic idea of the search processing.

[0055] Figure 23 is a diagram to explain the basic idea of the search processing.

[0056] Figure 24 is a diagram to explain the basic idea of the search processing.

[0057] Figure 25 is a diagram showing one specific example of setting a reference region with respect to an edge image as actual shape information.

[0058] Figure 26 is a diagram schematically showing one specific example of dividing a high-consistency virtual face into a plurality of virtual division faces.

[0059] Figure 27 is a diagram schematically showing one example of T2 virtual faces.

[0060] Figure 28 is a diagram schematically showing one example of T3 virtual division faces of the first one.

[0061] Figure 29 is a diagram for explaining a first specific example of generating M3 x T3 virtual division faces of the first one by the division face generation section.

[0062] Figure 30 is a diagram for explaining a second specific example of generating M3 x T3 virtual division faces of the first one by the division face generation section.

[0063] Figure 31 is a diagram for explaining a second specific example of generating M3 x T3 virtual division faces of the first one by the division face generation section.

[0064] Figure 32 is a diagram schematically showing one specific example of T3 virtual division faces of the n-th one, i.e., one example of T3 virtual division faces of the second one.

[0065] Figure 33 is a diagram for explaining a first specific example of generating M3 x T3 virtual division faces of the second one by the division face generation section.

[0066] Figure 34 is a diagram for explaining a second specific example of generating M3 x T3 virtual division faces of the second one by the division face generation section.

[0067] Figure 35 is a diagram for explaining a second specific example of generating M3 x T3 virtual division faces of the second one by the division face generation section.

[0068] Figure 36 is a diagram showing one specific example of a schematic flow of processing by the substrate processing apparatus.

[0069] Figure 37 is a diagram showing one specific example of an image processing flow of steps S3 and S10 of Figure 36

[0070] Figure 38 is a diagram showing one specific example of an image processing flow of steps S3 and S10 of​Figure 36 a flowchart of a specific example of a processing flow of the search processing of step S4 and step Sll of

[0071] Figure 39 is a flowchart of a specific example of a processing flow of the 1st search processing of step Sb l of Figure 38

[0072] Figure 40 is a flowchart of a specific example of a processing flow of the 2nd search processing of step Sb2 of Figure 38

[0073] Figure 41 is a flowchart of a specific example of a processing flow of the 2nd search processing of step Sb2 of Figure 38

[0074] Figure 42 is a flowchart of a specific example of a processing flow of the 2nd search processing of step Sb2 of Figure 38

[0075] Figure 43 is a diagram schematically showing an example of a state in which the object part moves to the origin position.

[0076] Figure 44 is a diagram for explaining the abnormality detection of the jig.

[0077] Figure 45 is a diagram for explaining the abnormality detection of the jig.

[0078] Figure 46 is a diagram for explaining the abnormality detection of the nozzle and the shield.

[0079] Figure 47 is a diagram schematically showing an example of an outline configuration of the substrate processing system. DETAILED DESCRIPTION

[0080] There is a substrate processing apparatus provided with a processing chamber, a substrate holding portion, a nozzle, a camera, an image processing portion, and a monitoring portion.

[0081] ​​​​In the substrate processing apparatus, a substrate holding portion, a nozzle, and a camera are arranged in a processing chamber. The substrate holding portion rotates a substrate in a horizontal plane in a state where the substrate is held in a horizontal posture. The nozzle moves between a standby position deviated to a side of the substrate and a spraying position above the substrate by rotation of a driving arm on which the nozzle is fixed. The nozzle is arranged at the standby position when the substrate is attached to or detached from the substrate holding portion, and is arranged at the spraying position when processing liquid is sprayed from the nozzle toward the substrate. The camera is arranged at a predetermined position in the processing chamber, and captures a predetermined area including the nozzle moved to the spraying position. An image processing portion acquires second nozzle position information indicating a position of the nozzle based on an image from the camera, and outputs the second nozzle position information to a monitoring portion. The monitoring portion judges whether or not the position of the nozzle is abnormal based on correspondence between first nozzle position information from a control portion as information directly or indirectly indicating the position of the nozzle and the second nozzle position information from the image processing portion.

[0082] However, in the substrate processing apparatus, for example, one or more of various parts such as the nozzle, a jig holding a peripheral edge of the substrate in the substrate holding portion, and a shield raised and lowered in a region surrounding the side of the substrate holding portion, it is desired to recognize a posture of the part in order to detect abnormality and an operation state of the part and the like.

[0083] In this case, in the substrate processing apparatus, for example, it is considered that the posture of the part is recognized by sequentially performing the following [Process A] to [Process E].

[0084] [Process A] A virtual polyhedron composed of a plurality of virtual faces is set, the plurality of virtual faces being along a spherical surface centered on a three-dimensional design shape (also referred to as a three-dimensional part model) of the part indicated by three-dimensional CAD (Computer Aided Design) data or the like of the part. Each virtual face is composed of, for example, a triangular face.

[0085] [Process B] A camera (also referred to as a virtual camera) is virtually set at a predetermined position (also referred to as a virtual camera position) on each virtual face constituting the virtual polyhedron set in the above-described Process A. Thus, the plurality of virtual cameras are virtually set in a manner of surrounding the three-dimensional part model with the three-dimensional part model as the center.

[0086] [Process C] A plurality of images (also referred to as virtual images) acquired by virtual capturing of the three-dimensional part model by the plurality of virtual cameras set at the plurality of virtual camera positions in the above-described Process B is generated. The virtual images can be generated, for example, by projecting the three-dimensional part model on a virtual plane by processing such as rendering.

[0087] [Process D] In each of the plurality of virtual images generated in the above-mentioned process C, a matching (comparison) process using template matching or the like is performed with an image (also referred to as an actual image) obtained by actually capturing the part with a camera (also referred to as an actual camera). Thus, among the plurality of virtual camera positions, a virtual camera position in which the degree of agreement (also referred to as agreement degree) between the actual image and the virtual image with respect to the shape, orientation, and size of the captured part and the like is the greatest is detected. The degree of agreement (agreement degree) is also referred to as the degree of similarity (similarity degree).

[0088] [Process E] Based on the detection result of the above-mentioned process D, the posture of the part is recognized. Here, for example, based on the detection result of the above-mentioned process D, the posture of the part with reference to the position of the actual camera is recognized, and thus the posture of the part captured in the actual image is recognized.

[0089] In the above-mentioned method, for example, when the distance between the actual camera and the part changes, in the above-mentioned process A, a plurality of virtual polyhedrons whose distances from the center of the three-dimensional part model are different from each other are set.

[0090] Here, for example, a method in which the plurality of virtual faces constituting the virtual polyhedron in the above-mentioned process A is set to a plurality of virtual faces that are very small, the number of virtual camera positions virtually set in the above-mentioned process B is increased, and thus the accuracy of recognizing the posture of the part is improved is considered. However, according to this method, the time required for calculation becomes longer due to an increase in the amount of calculation.

[0091] Therefore, for example, a method in which the above-mentioned process A to the above-mentioned process D are sequentially repeated a predetermined number of times and then the above-mentioned process E is performed is considered. In this method, in the first above-mentioned process A, a virtual polyhedron is constituted by a plurality of virtual faces that are relatively large, and in the above-mentioned process A after the second time, a virtual polyhedron is constituted by a plurality of virtual faces that are relatively smaller than in the above-mentioned process A of the previous time, and in the above-mentioned process B after the second time, a virtual camera is virtually set at a predetermined virtual camera position on each of a plurality of virtual faces that are close to the virtual camera position detected in the most recent above-mentioned process D among the polyhedrons set in the above-mentioned process A of the previous time. According to this method, the accuracy of recognizing the posture of the part can be improved, and the time required for calculation can be shortened by reducing the amount of calculation.

[0092] However, in this method, the fine virtual polyhedrons are set in the above-mentioned process A at stages a plurality of times, and the virtual polyhedrons set in the above-mentioned process A after the second time are independent of the virtual polyhedrons set in the above-mentioned process A at the last time. Therefore, in order to maintain the accuracy of recognizing the posture of the part, the above-mentioned plurality of virtual faces of the above-mentioned part set in the above-mentioned process B after the second time cover a certain wide area in the vicinity of the position of the virtual camera detected in the above-mentioned process D at the last time. As a result, the above-mentioned plurality of virtual faces of the virtual camera set in the above-mentioned process B after the second time increase at least one of the number and the area to some extent, and the number of the virtual camera positions set in the above-mentioned process B after the second time can increase, so that it is not easy to sufficiently seek the reduction of the amount of calculation.

[0093] Thus, as for the substrate processing apparatus, there is room for improvement in terms of efficiently performing recognition of the posture of the part.

[0094] Hereinafter, embodiments will be described with reference to the accompanying drawings. Note that the drawings are schematic representations and constituent elements or configurations thereof are appropriately omitted or simplified for the convenience of explanation. In addition, the mutual relationship of the sizes and positions of the configurations shown in the drawings is not necessarily accurately recorded, and can be appropriately changed.

[0095] In addition, in the explanation shown below, the same symbols are attached to the same constituent elements and illustrated, and the names and functions thereof are also the same. Therefore, there are cases where detailed explanations thereof are omitted in order to avoid repetition.

[0096] In addition, in the explanation shown below, even if there are cases where ordinal numbers such as "first" or "second" are used, these terms are used for the convenience of understanding the contents of the embodiments, and are not limited to the order that can be generated by these ordinal numbers.

[0097] In the case where a representation indicating a relative or absolute positional relation (for example, "in a direction," "along a direction," "parallel," "orthogonal," "center," "concentric," "coaxial," and the like) is used, the representation indicates not only a strict positional relation but also a state of relative displacement in terms of angle or distance within a range of tolerance or a range in which a similar function is obtained, unless otherwise specified. In the case where a representation indicating equality (for example, "the same," "equal," "uniform," and the like) is used, the representation indicates not only a strict quantitative equality but also a state in which there is a difference within a range of tolerance or a range in which a similar function is obtained, unless otherwise specified. In the case where a representation indicating a shape (for example, "quadrilateral shape" or "cylindrical shape," and the like) is used, the representation indicates not only a strict geometric shape but also a shape in which, for example, any of a concave-convex shape and a chamfered shape, and the like is included within a range in which a similar effect is obtained, unless otherwise specified. In the case where a representation using "provided with," "containing," "equipped with," "including," or "having" a component is used, the representation is not an exclusive representation that excludes the presence of other components. In the case where a representation using "at least one of A, B, and C" is used, the representation includes only A, only B, only C, any two of A, B, and C, and all of A, B, and C.

[0098] <1. Overall configuration of substrate processing apparatus>

[0099] Figure 1 is a side view schematically showing an example of a general configuration of a substrate processing apparatus 1 of Embodiment 1. Figure 2 is a plan view schematically showing an example of a general configuration of the substrate processing apparatus 1 of Embodiment 1.

[0100] The substrate processing apparatus 1 is an apparatus that performs processing of a substrate (for example, a semiconductor wafer) W. In the present Embodiment 1, the substrate processing apparatus 1 is a single-wafer type apparatus that performs processing of the substrate W piece by piece. A thin flat plate of a substantially disc shape is applied to the substrate W. In other words, the substrate W has a circular shape in a plan view. In the case where a condition is not particularly described with respect to a plan view, it means a plan view in which each part is observed from the upper side to the lower side. The substrate processing apparatus 1 performs a predetermined processing of the substrate W while rotating the substrate W and supplying a processing liquid. The substrate processing apparatus 1 is applied, for example, to a substrate cleaning apparatus that uses a processing liquid for a substrate W.

[0101] The substrate processing apparatus 1 is provided with a housing CA. The housing CA cuts off the inside of the housing CA from ambient gas around the housing CA.

[0102] The substrate processing apparatus 1 includes a spin chuck 3. The spin chuck 3 has a circular shape with a diameter larger than the substrate W in plan view. An upper end of a rotation shaft 5 is connected to a lower surface of the spin chuck 3. A lower end of the rotation shaft 5 is connected to a motor 7. The spin chuck 3 is rotated about a rotation center PL1 by driving of the motor 7. The rotation center PL1 extends in the vertical direction.

[0103] The spin chuck 3 includes a plurality of chucks 9. More specifically, the spin chuck 3 includes the plurality of chucks 9 at a peripheral portion of an upper surface of the spin chuck 3. In the first embodiment, the spin chuck 3 includes four chucks 9. The number of the chucks 9 included in the spin chuck 3 is not limited to four, if the spin chuck 3 can be stably rotated about the rotation center PL1 in a state in which the substrate W is supported in a horizontal posture.

[0104] The chuck 9 includes, for example, a lower surface support portion 11, a peripheral support portion 13, and a rotation magnet 15. The lower surface support portion 11 is a portion that supports the substrate W by abutting against a lower surface of the substrate W from below. For example, if the lower surface support portion 11 is configured such that an area of contact with the lower surface of the substrate W is small, the degree of contamination generated between the substrate W and the chuck 9 can be reduced. The lower surface support portion 11 is rotatably attached to the upper surface side of a body portion of the spin chuck 3 about a rotation center PL2. The rotation center PL2 extends in the vertical direction. The peripheral support portion 13 is provided upright on an upper surface of the lower surface support portion 11. For example, if a height of the peripheral support portion 13 from the upper surface of the lower surface support portion 11 is higher than a thickness of the substrate W, the peripheral support portion 13 can stably hold the peripheral edge of the substrate W. The peripheral support portion 13 is provided at a position apart from the outer edge of the lower surface support portion 11 from the rotation center PL2 in plan view. In other words, the peripheral support portion 13 is eccentric from the rotation center PL2. The rotation magnet 15 is attached to a position corresponding to the rotation center PL2 on the lower surface side of the spin chuck 3. The rotation magnet 15 is connected to the lower surface support portion 11. The rotation magnet 15 is rotatably provided about the rotation center PL2.

[0105] The substrate processing apparatus 1 includes a jig driving mechanism 17 disposed below the rotating magnet 15. The jig driving mechanism 17 is disposed on the side of the rotating shaft 5 relative to the jig 9. The jig driving mechanism 17 includes, for example, a cylinder 19 and a driving magnet 21. The driving magnet 21 has a ring shape when viewed from above. The cylinder 19 has a piston rod (also referred to as a rod or an operation shaft) positioned in the vertical direction. The driving magnet 21 is attached to the upper end of the rod of the cylinder 19. The jig driving mechanism 17 operates in accordance with a jig operation command from the control section 45. The jig driving mechanism 17 moves the driving magnet 21 close to the jig 9 by raising the driving magnet 21 using the cylinder 19, and moves the driving magnet 21 away from the jig 9 by lowering the driving magnet 21 using the cylinder 19. The jig driving mechanism 17 moves the driving magnet 21 between a raised position (also referred to as a raised position) and a lowered position (also referred to as a lowered position).

[0106] The jig 9 includes an omitted urging mechanism. If the driving magnet 21 is lowered and moved from the raised position to the lowered position, the jig 9 is moved from the open position to the closed position. If the driving magnet 21 is raised and moved from the lowered position to the raised position, the jig 9 is moved from the closed position to the open position. If the jig 9 is moved from the open position to the closed position, the peripheral support portion 13 rotates about the rotation center PL2, and the peripheral support portion 13 is moved in the direction close to the rotation center PL1 and comes into contact with the peripheral edge of the substrate W. Thus, the plurality of jigs 9 can hold the substrate W in the closed position. If the jig 9 is moved from the closed position to the open position, the peripheral support portion 13 rotates about the rotation center PL2, and the peripheral support portion 13 is moved in the direction away from the rotation center PL1. Thus, the plurality of jigs 9 become in a state where the substrate W is not held in the open position. In this state, the substrate W can be carried onto the rotary chuck 3 from outside the housing CA, and carried out of the rotary chuck 3 to outside the housing CA. In addition, in a state where the substrate W is not placed on the lower surface support portion 11, if the driving magnet 21 is lowered and moved from the raised position to the lowered position, the position of the jig 9 becomes a position where the peripheral support portion 13 is moved to a position slightly inside the substrate W relative to a position where the substrate W is placed on the lower surface support portion 11 when viewed from above (also referred to as a home position or a jig home position). In other words, if the position of the jig 9 is in the home position, the position of the peripheral support portion 13 is on the side of the rotation center PL1 relative to the position when the jig 9 is in the closed position.

[0107] The substrate processing apparatus 1 includes a home sensor Z1 disposed in the vicinity of the rotating magnet 15 of the jig 9. If the jig 9 is moved to the closed position or the home position, the home sensor Z1 changes a signal (also referred to as an output signal) output from the home sensor Z1. For example, if the jig 9 is moved to the closed position or the home position, the home sensor Z1 turns on the output signal.

[0108] The substrate processing apparatus 1 is provided with a shield 23 disposed around the rotary chuck 3. The shield 23 surrounds the side of the rotary chuck 3. The shield 23 is a portion to prevent the processing liquid from flying around on the substrate W supported by the rotary chuck 3 and rotating. The shield 23 has a cylindrical shape. The shield 23 has an opening portion 23a formed in the upper portion. The inner diameter of the opening portion 23a is larger than the outer diameter of the rotary chuck 3.

[0109] The substrate processing apparatus 1 is provided with a shield moving mechanism 25 that can raise and lower the shield 23. The shield moving mechanism 25 is provided with, for example, a cylinder 27 and a stopper piece 29. The shield moving mechanism 25 is disposed, for example, on the outer peripheral side of the shield 23. If the shield moving mechanism 25 can raise and lower the shield 23, it can also be disposed on the inner peripheral side of the shield 23. The cylinder 27 has a piston rod (also called a rod or an operation shaft) positioned in the vertical direction. On the upper end of the rod of the cylinder 27, the stopper piece 29 is installed. The stopper piece 29 is fixed to the outer peripheral surface of the shield 23. If the shield moving mechanism 25 can raise and lower the shield 23, it is not limited to the configuration described above.

[0110] The shield moving mechanism 25 moves the shield 23 between a home position and a processing position according to a shield operation instruction from the control section 45. The home position of the shield 23 is the position of the shield 23 when the shield 23 is lowered by the shield moving mechanism 25. In other words, the home position is the position where the upper end of the shield 23 is low. The processing position of the shield 23 is the position of the shield 23 when the shield 23 is raised by the shield moving mechanism 25. In other words, the home position is the position lower than the processing position, and the processing position is the position higher than the home position. In the state where the shield 23 is positioned at the home position, the position of the upper edge of the shield 23 is lower than the position of the substrate W supported by the rotary chuck 3. In the state where the shield 23 is positioned at the processing position, the position of the upper edge of the shield 23 is higher than the position of the substrate W supported by the rotary chuck 3. Therefore, in the state where the shield 23 is positioned at the processing position, the shield 23 can receive the processing liquid flying around on the substrate W supported by the rotary chuck 3 and rotating, by the inner peripheral surface of the shield 23. The substrate processing apparatus 1 is provided with, for example, a home sensor Z2 disposed on the inner peripheral side of the shield 23. If the shield 23 moves to the home position, the home sensor Z2 changes the signal (output signal) output from the home sensor Z2. For example, if the shield 23 moves to the home position, the home sensor Z2 turns on the output signal.

[0111] The substrate processing apparatus 1 has a not-illustrated drain port on the inner peripheral side of the shield 23. The drain port is a portion to recover the processing liquid scattered from the substrate W supported by the free rotation chuck 3 and rotated, and received by the inner peripheral surface of the shield 23. The number of the shields 23 is not limited to one, and can be plural. The number of the drain ports is not limited to one, and can be plural. In the case where the number of the shields 23 is plural, the drain port can be provided for each shield 23. In this case, each shield 23 can be raised and lowered by the shield moving mechanism 25 to switch the combination of the shield 23 that receives the processing liquid scattered from the substrate W and the drain port that recovers the processing liquid received by the inner peripheral surface of the shield 23, and the plural shields 23 and the plural drain ports are configured. For example, it is considered that each shield 23 is raised and lowered by the shield moving mechanism 25 to switch the combination of the shield 23 that receives the processing liquid scattered from the substrate W and the drain port that recovers the processing liquid received by the inner peripheral surface of the shield 23, according to the kind of the processing liquid.

[0112] The substrate processing apparatus 1 has a processing liquid supply mechanism 31. The processing liquid supply mechanism 31 can be provided on the outer peripheral side of the shield 23. The processing liquid supply mechanism 31 has, for example, a nozzle 33, a base portion 37, and a nozzle moving mechanism 35. In the present first embodiment, the processing liquid supply mechanism 31 has, for example, two nozzles 33. In the following description, when it is necessary to distinguish the two nozzles 33, the nozzle 33 on the left side of the formula (1) is referred to as the nozzle 33A, and the nozzle 33 on the right side is referred to as the nozzle 33B. The number of the nozzles 33 of the processing liquid supply mechanism 31 can be one, or three or more. In the present first embodiment, the two nozzles 33 have the same configuration. Figure 2

[0113] The nozzle 33 has, for example, an extension portion 33a, a downward portion 33b, and a tip portion 33c. One end of the extension portion 33a of the nozzle 33 is attached to the base portion 37. The extension portion 33a extends from the base portion 37 in the horizontal direction. The other end of the extension portion 33a is connected to the downward portion 33b. The downward portion 33b extends from the extension portion 33a toward the lower side in the vertical direction. The tip portion 33c is located on the lower end side of the downward portion 33b. The tip portion 33c ejects the processing liquid from the lower surface side. As the processing liquid, for example, resist liquid, SOG (Spin-on-Glass) liquid, developing liquid, cleaning liquid, pure water, and rinsing liquid are cited.

[0114] ​The nozzle moving mechanism 35 includes, for example, a motor 39, a rotating shaft 41, and a position detection section 43. The motor 39 is disposed in a vertical posture. The rotating shaft 41 is rotated by the motor 39 with the center of rotation PL3 as the center. The rotating shaft 41 is coupled to the base section 37. The base section 37 is rotated by driving of the motor 39. The nozzle 33 is swung by the nozzle moving mechanism 35 with the center of rotation PL3 as the center together with the base section 37. The position detection section 43 detects the position (also referred to as the rotational position) of the rotating shaft 41 in the direction of rotation with the center of rotation PL3 of the rotating shaft 41 as the center. The position detection section 43 detects the angle of the rotating shaft 41 in plan view with the center of rotation PL3 as the center. The position detection section 43 outputs a pulse signal, for example, in accordance with the rotational position of the rotating shaft 41.

[0115] The substrate processing apparatus 1 includes a standby cup 44 disposed at a position apart from the shield 23 to the side in plan view. The standby cup 44 is disposed so that the tip end section 33c of the nozzle 33 can be positioned above the standby cup 44 in the case of plan view. From another viewpoint, the standby cup 44 is positioned below the origin position of the nozzle 33. The standby cup 44 is a section for preventing the tip end section 33c of the nozzle 33 from drying. The standby cup 44 is used for the empty ejection of the processing liquid from the nozzle 33. The nozzle moving mechanism 35 swings the nozzle 33 by driving the motor 39. The nozzle moving mechanism 35 moves the tip end section 33c of the nozzle 33 between the origin position and an ejection position on the center of rotation PL1 of the spin chuck 3 in accordance with a nozzle operation command from the control section 45. The ejection position of the nozzle 33 is a position in which the nozzle 33 is disposed when the processing liquid is ejected from the tip end section 33c of the nozzle 33 toward the substrate W.

[0116] The substrate processing apparatus 1 includes, for example, an origin sensor Z3 disposed on the outer peripheral section side of the rotating shaft 41. If the nozzle 33 is positioned at the origin position, the origin sensor Z3 changes the signal (output signal) output from the origin sensor Z3. For example, if the nozzle 33 is moved to the origin position, the origin sensor Z3 turns on the output signal. In addition, the origin sensor Z3 can be omitted to seek simplification of the configuration. In this case, a protrusion (also referred to as a rotating side protrusion) can be provided to a part of the rotating shaft 41, and a protrusion (also referred to as a fixed side protrusion) disposed on the outer peripheral section side of the rotating shaft 41 and fixed to the housing CA can be provided. For example, the position detection section 43 can also detect that the nozzle 33 is positioned at the origin position by detecting that the rotating side protrusion of the rotating shaft 41 and the fixed side protrusion abut each other and the rotation of the rotating shaft 41 with the center of rotation PL3 as the center cannot be performed. Here, the position of the nozzle 33 at the point in time at which the pulse signal of the position detection section 43 becomes constant can be treated as the origin position.

[0117] The substrate processing apparatus 1 has a camera CM as a photographing section. The camera CM obtains an image (also referred to as an actual image) that captures one or more object parts described later by photographing. The camera CM is mounted at a portion of the housing CA, for example. The position of the camera CM can be any portion if the object parts described later fall within a field of view. The camera CM has a field of view (angle of view) in which all of the one or more object parts described later fall within the field of view, for example. The camera CM has a field of view (angle of view) in which the origin positions of the one or more object parts described later all fall within the field of view, for example.

[0118] The substrate processing apparatus 1 has a control section 45, an instruction section 47, and a reporting section 49. Details of the control section 45 are described later. The instruction section 47 is operated by an operator of the substrate processing apparatus 1. The instruction section 47 is a keyboard or a touch panel, for example. The instruction section 47 outputs a signal corresponding to an operation of the operator to the control section 45, for example. The instruction section 47 instructs the object parts described later, a confirmation timing, an allowable range, a procedure, and a start of processing, and the like. The reporting section 49 reports an abnormality to the operator when the control section 45 detects the abnormality. As the reporting section 49, a display, a lamp, a speaker, and the like are exemplified. The reporting section 49 acts in response to a signal from the control section 45, for example.

[0119] <2. Configuration of control system of substrate processing apparatus>

[0120] Figure 3 is a block diagram that shows an example of a functional configuration of the substrate processing apparatus 1 of the first embodiment.

[0121] The control section 45 has an arithmetic section 45a and a storage section 45b, and the like, for example. The arithmetic section 45a has an electronic circuit that functions as a processor such as a central processing unit (CPU), and an electronic circuit that functions as a memory such as a random access memory (RAM) that temporarily stores data for processing performed by the processor, for example. The storage section 45b has a hard disk or a flash memory that functions as a nonvolatile storage medium, and the like. The storage section 45b can have one portion that functions as a nonvolatile storage medium, or two or more portions that function as nonvolatile storage media, for example.

[0122] The arithmetic unit 45a includes, for example, the operation control unit 51, the image processing unit 59, the search processing unit 61, and the abnormality detection unit 63 as a plurality of functional configurations. The storage unit 45b stores, for example, the procedure information 53, the parameter information 55, the design information 57, and the program Pg1. In the arithmetic unit 45a, for example, the operation control unit 51, the image processing unit 59, the search processing unit 61, and the abnormality detection unit 63 can be functional processing units (also referred to as functional processing units) realized by a CPU reading out and executing the program Pg1 and the like stored in the storage unit 45b.

[0123] The operation control unit 51 controls the operation of the above-described motors 7, 39, the cylinders 19, 27, and the camera CM. The operation control unit 51 is given signals from the origin sensors Z1, Z2, Z3, and the position detection unit 43. The operation control unit 51 controls the operation of each of the motors 7, 39, the cylinders 19, 27, the camera CM, and the like, for example, in accordance with a procedure stored in the procedure information 53 stored in the storage unit 45b. For example, after the operator instructs the start of the operation of each of the units based on the procedure, the operation control unit 51 outputs various operation instructions to each of the units based on the procedure, and causes each of the units such as the motors 7 to operate at a predetermined timing.

[0124] The procedure information 53 is information indicating various procedures stored in advance in the storage unit 45b. The procedure specifies various sequences of processing the substrate W. The operator can instruct the execution of a desired procedure by operating the instruction unit 47.

[0125] The parameter information 55 is information of a timing to be confirmed, an allowable range, and the like for each of the target parts. The target part is a part of the parts constituting the substrate processing apparatus 1 that becomes a specific processing target. In the first embodiment, the specific processing includes the detection of abnormalities. The timing to be confirmed is a timing to confirm the operation state of the target part. The timing to be confirmed can coincide with the timing at which the operation control unit 51 outputs an operation instruction, or can coincide with the timing at which the target part completes the movement according to the operation instruction. The target part, the timing to be confirmed, the allowable range, and the like can be arbitrarily set by the operator operating the instruction unit 47. The operator can instruct, from the instruction unit 47, which part to set as the target part, which timing to set as the timing to be confirmed, which degree of error in the position of the target part to set as the allowable range, and the like.

[0126] The object parts can include, for example, the gripper 9, the shield 23, and the nozzle 33. The confirmation timing can include, for example, timing at which the gripper 9 is moved by the gripper drive mechanism 17 in accordance with the gripper operation command and the movement of the gripper 9 is completed, timing at which the nozzle 33 is moved by the nozzle movement mechanism 35 in accordance with the nozzle operation command and the movement of the nozzle 33 is completed, timing at which the shield 23 is raised and lowered by the shield movement mechanism 25 in accordance with the shield operation command and the movement of the shield 23 is completed, timing at which the gripper 9 is set to be located at the closed position by the gripper operation command, timing at which the nozzle 33 is set to be located at the discharge position by the nozzle operation command, and timing at which the shield 23 is set to be located at the processing position by the shield operation command.

[0127] The allowable range indicates, at the confirmation timing, the degree of allowance for the position at which the object part should originally be located in the case of normal operation of the object part. The allowable range indicates, for example, the degree of allowance for the degree to which the object part can deviate from the position and angle intended by design. The allowable range indicates, based on the processing of the substrate W, the range of deviation of the object part that is allowable as the processing of the substrate W even if the object part deviates from the position and angle intended by design at the confirmation timing.

[0128] The above-described operation control section 51, based on the pulse signal from the position detection section 43 and the parameter information 55 in the storage section 45b, notifies the search processing section 61 of information indicating the nozzle 33 as the object part that is at the confirmation timing and information indicating the position (e.g., the discharge position, etc.) at which the nozzle 33 as the object part that is at the confirmation timing should originally be located at the confirmation timing if the nozzle 33 as the object part is at the confirmation timing. The operation control section 51, based on the signals from the origin sensors Z1, Z2, and Z3, notifies the search processing section 61 of information indicating the object part that is located at the origin position if the object part is located at the origin position. Here, if there is only one object part, the operation control section 51 can not notify the search processing section 61 of information indicating the object part that is located at the origin position. The operation control section 51, based on the operation control of the cylinders 19 and 27 and the motor 39, notifies the search processing section 61 of information indicating the object part that is at the confirmation timing and information indicating the position (e.g., the closed position of the gripper 9, the discharge position of the nozzle 33, or the processing position of the shield 23, etc.) at which the object part should originally be located at the confirmation timing with respect to each object part if the object part is at the confirmation timing. Here, if there is only one object part, the operation control section 51 can not notify the search processing section 61 of information indicating the object part that is at the confirmation timing. The operation control section 51 can also notify the search processing section 61 of the operation command output to each section in accordance with the procedure.

[0129] The design information 57 includes design information related to the components that constitute the substrate processing apparatus 1, which is stored in the storage section 45b. The design information 57 can also include design information related to the substrate W that is the processing target of the substrate processing apparatus 1. The design information 57 includes, for example, data of 3D-CAD (three-dimensional Computer Aided Design). The design information 57 can also include physical property information and the like of the processing liquid and various materials used for processing.

[0130] The 3D-CAD data is expressed, for example, in three axes that are orthogonal to the coordinate axes, and in the case where the components are arranged in three-dimensional space, is expressed in information related to the position and angle of the components (also collectively referred to as position information). The storage section 45b stores in advance design information related to at least the target components transmitted from a host computer not shown. In other words, the storage section 45b stores three-dimensional design information related to the target components. The host computer not shown can also store 3D-CAD data related to all of the components and materials of the substrate processing apparatus 1 as three-dimensional design information. Here, if the design information 57 stored in the storage section 45b is limited to design information of the target components, rather than design information related to all of the components of the substrate processing apparatus 1, the storage capacity of the storage section 45b can be saved.

[0131] The image processing section 59 processes an actual image obtained by imaging with the camera CM. The image processing section 59 performs image processing on the actual image and extracts information related to the two-dimensional shape of the object captured in the actual image (also referred to as actual shape information). The image processing section 59 extracts the actual shape information, for example, for all of the components captured in all regions in the actual image or a portion of the regions set in advance, for example, by performing a processing of extracting an outline. The outline referred to here can include not only the outline of the outer shape of the component, but also a rim portion located inside the outer shape of the component. The actual shape information extracted by the image processing section 59 is given to the search processing section 61.

[0132] The search processing section 61 performs a process of searching for a virtual camera position in which the degree of agreement between the reference shape information and the actual shape information is the greatest, among a plurality of virtual camera positions (also referred to as search processing). Here, the plurality of reference shape information is information generated based on the three-dimensional design information related to the object part stored in the storage section 45b. More specifically, the plurality of reference shape information is information related to the two-dimensional shape of the three-dimensional model (also referred to as 3D model) of the object part in each of a plurality of virtual images (also referred to as virtual image) obtainable by capturing the 3D model of the object part from a plurality of virtual camera positions. For example, one reference shape information is information generated based on the three-dimensional design information related to one object part, and is information related to the two-dimensional shape of the 3D model of the object part in a virtual image obtainable by capturing the 3D model of the object part from one virtual camera position. The 3D model of the object part is a three-dimensional model of the object part generated virtually based on the three-dimensional design information related to the object part. The virtual image can be generated, for example, by projecting the 3D model on a virtual plane using a process such as rendering. The degree of agreement between the reference shape information and the actual shape information refers to, for example, how close the reference shape information is to the actual shape information. The greater the degree of agreement, the more the reference shape information approximates the actual shape information, and the smaller the degree of agreement, the more the reference shape information differs from the actual shape information. The degree of agreement between the reference shape information and the actual shape information is represented, for example, by a numerical value indicating the degree of agreement between the reference shape information and the actual shape information. The degree of agreement and the numerical value indicating the degree of agreement are described further below. The information related to the two-dimensional shape of the 3D model can be, for example, a figure.

[0133] According to the search processing by the search processing section 61, it is possible to search for an object part in which the degree of agreement with the two-dimensional shape of the 3D model of the object part in the virtual image obtainable by capturing from which of the plurality of virtual camera positions is the greatest, among the actual images obtained by capturing using the camera CM. Based on this search result, in the search processing section 61, it is possible to recognize the posture of the object part.

[0134] The search processing section 61 performs the search processing on at least the object part in the timing to be confirmed. Thus, in the substrate processing device 1, it is possible to recognize the posture of the object part in the timing to be confirmed. The search processing section 61 outputs information related to the posture of the object part recognized using the actual image (also referred to as reality information) to the abnormality detection section 63. In other words, the reality information is, for example, information related to the posture of the object part based on the virtual camera position in which the degree of agreement is the greatest detected by the search processing section 61. This reality information is information based on the actual image, and is information indicating the state in which the object part is actually present in the substrate processing device 1 in the timing to be confirmed.

[0135] The search processing section 61 performs the object part-related search processing even at a timing other than the timing at which confirmation is to be made, in accordance with an instruction from the operation control section 51. For example, the search processing section 61 performs the object part-related search processing (also referred to as the origin search processing) at the origin position, in response to an instruction from the operation control section 51 based on the output signals from the origin sensors Z1, Z2, Z3. The origin search processing can be performed only once for each object part, for example, at the time of starting up the substrate processing apparatus 1 and beginning the processing of the substrate W. In addition, the search processing section 61 can perform the object part-related search processing at any timing including the timing at which confirmation is to be made.

[0136] The abnormality detection section 63 detects an abnormality of the object part, based on the information output from the search processing section 61. Specifically, the abnormality detection section 63 detects an abnormality of the object part by comparing the information (reality information) related to the posture of the object part at the virtual camera position at which the degree of coincidence between the actual shape information detected by the search processing section 61 and the reference shape information is the greatest, with the information (also referred to as normal information) related to the posture of the object part based on the three-dimensional design information at the time when the state of the object part is normal. More specifically, the abnormality detection section 63 detects an abnormality related to the object part when the comparison result of the reality information and the normal information is that the reality information and the normal information do not coincide. Here, the normal information related to the object part can include information related to the posture of the object part within an allowable range. In this case, the reality information coincides with the normal information includes the information related to the posture of any one of the object parts within the allowable range of the normal information, and the information related to the posture of the object part of the reality information.

[0137] The abnormality detection section 63 causes the reporting section 49 to perform a reporting operation in accordance with the detection result. Specifically, the abnormality detection section 63 causes the reporting section 49 to perform a reporting operation in response to the detection of an abnormality related to the object part. The reporting operation can be an operation of notifying of the generation of an abnormality, for example. The reporting section 49 can report, together with the generation of an abnormality, information for specifying the object part in which the abnormality is detected, for example, and can also report position information of the object part in which the abnormality is detected. Here, as the information for specifying the object part in which the abnormality is detected, there are, for example, a character string or a sound indicating the name or the symbol of each of a plurality of specified clamps 9, the name or the symbol of each of one or more specified shields 23, and the name or the symbol of each of one or more specified nozzles 33, and the like. As the information indicating the position information of the object part in which the abnormality is detected, there are various information such as a character string or a sound indicating the closed position and the open position related to the clamp 9, the origin position and the processing position related to the shield 23, and the origin position and the ejection position related to the nozzle 33, and the like.

[0138] <3. Specific example of functional configuration related to search processing>

[0139] Figure 4 is a block diagram showing a specific example of the functional configuration of the search processing related to the control section 45. Figure 4 In the functional configuration of the control section 45, the functional configuration related to the storage section 45b, the image processing section 59, and the search processing section 61 is shown.

[0140] <3-1. Image processing section>

[0141] The image processing section 59 has, for example, a processing target region extraction section 591 and a contour extraction section 592 as the plurality of functional processing sections. These processing target region extraction section 591 and contour extraction section 592 are realized, for example, in the control section 45 (more specifically, the arithmetic section 45a) by the CPU reading out and executing the program Pg1 stored in the storage section 45b.

[0142] The processing target region extraction section 591 acquires an actual image in which an object part is captured, obtained by photographing with the camera CM, extracts a portion related to the processing target region (also referred to as a processing target region) from the actual image, and acquires an image (also referred to as a processing target actual image). Thereby, for example, in the processing of the contour extraction section 592 and the processing of the search processing section 61, the amount of processing of the operation can be reduced. As a result, the posture recognition of the object part can be performed efficiently.

[0143] Figures 5 to 7 are diagrams for explaining the processing of extracting a portion related to the processing target region from the actual image. Figure 5 In the functional configuration of the control section 45, the functional configuration related to the storage section 45b, the image processing section 59, and the search processing section 61 is shown. Figure 6 In the functional configuration of the control section 45, the functional configuration related to the storage section 45b, the image processing section 59, and the search processing section 61 is shown. Figure 7 In the functional configuration of the control section 45, the functional configuration related to the storage section 45b, the image processing section 59, and the search processing section 61 is shown.

[0144] The processing target region extraction section 591, for example, acquires the actual image Ir1 shown in Figure 5 by extracting a portion related to the processing target region R1 surrounded by a thick double dashed line in the actual image Ir1 as shown in Figure 6 Figure 7

[0145] ​​The processing target region R1 is, for example, for each object part, a region that can be set in advance on the actual image as a region in which the object part can exist at the time of confirmation. For example, the processing target region R1 can be set in advance by referring to the actual image obtained by the imaging with the camera CM, or can be set based on design information of the parts constituting the substrate processing apparatus 1 and the imaging direction and angle of view of the camera CM.

[0146] The contour extraction section 592 extracts, for example, the two-dimensional shape-related information (actual shape information) of the object captured in the actual image by performing a processing of extracting a contour for all the parts captured in the processing target actual image Ir2 obtained by the processing target region extraction section 591. Thus, the image processing section 59 can acquire the actual shape information of the two-dimensional shape of the object captured in the actual image. The search processing section 61 is given the actual shape information acquired by the image processing section 59.

[0147] Here, the processing target actual image Ir2 obtained by the processing target region extraction section 591 can be regarded as an actual image in which the object part is captured. The contour, as described above, can include not only the contour of the part shape but also a rim portion located inside the part shape. The processing of extracting the contour can be implemented, for example, using a method of detecting an edge such as the Canny edge detection method. Here, the contour can also be extracted for all the parts captured in the processing target actual image Ir2 by further performing a processing of inflating a line on the detected edge. The actual shape information can be, for example, an edge image representing the contour of the object captured in the actual image. As the edge image, for example, a binary image in which the contour of the object can be distinguished from other portions thereof is used. Here, the edge image as the actual shape information can be regarded as information representing the two-dimensional shape of the object in the actual image. In other words, the actual shape information can be, for example, information representing the two-dimensional shape of the object in the actual image. Figure 8 FIG. 6 is a diagram illustrating an example of the edge image Ir3 representing the contour extracted by the contour extraction section 592 as the actual shape information. Figure 8 The illustrated edge image Ir3 is an edge image obtained by performing a processing of extracting a contour for all the parts captured in the processing target actual image Ir2. Figure 7 The illustrated edge image Ir3 is an edge image obtained by performing a processing of extracting a contour for all the parts captured in the processing target actual image Ir2. Figure 8 The illustrated edge image Ir3 is a binary image in which the contour of the part extracted by the contour extraction section 592 is represented in white and other portions are represented in black.

[0148] <3-2. Search Processing Section>

[0149] <3-2-1. Example of Basic Idea of Search Processing>

[0150] Figures 9 to 16are examples of a virtual image and a reference image, respectively, which are used to explain the basic idea of the search processing.

[0151] For example, as shown in FIG. 3A, a three-dimensional model of the object part (e.g., the jig 9) that is virtually generated on the basis of three-dimensional design information related to the object part (e.g., the jig 9), i.e., a 3D model 3dm of the object part, is taken as the center, the positions (virtual camera positions) P1 of a plurality of virtual cameras are appropriately set, and information related to the two-dimensional shape of the 3D model 3dm of the object part in a virtual image that can be obtained by photographing the 3D model 3dm of the object part from each of the virtual camera positions P1 (reference shape information) is generated. That is, the reference shape information is generated for each of the virtual camera positions P1. As the reference shape information, for example, as shown in FIG. 3B, an image (also referred to as a reference image) Iv1 representing the outline of the 3D model 3dm of the object part in the virtual image that can be obtained by photographing the 3D model 3dm of the object part from each of the virtual camera positions P1 can be used. Here, the reference image as the reference shape information can be said to be information representing the two-dimensional shape of the 3D model 3dm in the virtual image. In other words, the reference shape information can be, for example, information representing the two-dimensional shape of the 3D model 3dm in the virtual image. Figure 9 Figure 10 For example, as shown in FIG. 3A, a three-dimensional model of the object part (e.g., the jig 9) that is virtually generated on the basis of three-dimensional design information related to the object part (e.g., the jig 9), i.e., a 3D model 3dm of the object part, is taken as the center, the positions (virtual camera positions) P1 of a plurality of virtual cameras are appropriately set, and information related to the two-dimensional shape of the 3D model 3dm of the object part in a virtual image that can be obtained by photographing the 3D model 3dm of the object part from each of the virtual camera positions P1 (reference shape information) is generated. That is, the reference shape information is generated for each of the virtual camera positions P1. As the reference shape information, for example, as shown in FIG. 3B, an image (also referred to as a reference image) Iv1 representing the outline of the 3D model 3dm of the object part in the virtual image that can be obtained by photographing the 3D model 3dm of the object part from each of the virtual camera positions P1 can be used. Here, the reference image as the reference shape information can be said to be information representing the two-dimensional shape of the 3D model 3dm in the virtual image. In other words, the reference shape information can be, for example, information representing the two-dimensional shape of the 3D model 3dm in the virtual image. Figure 10 Figure 10 The reference image Iv1 illustrated is a binary image in which the outline of the 3D model 3dm of the object part as the two-dimensional shape is represented in black and the other portions are represented in white.

[0152] Furthermore, for example, a value indicating the degree of coincidence between each of the plurality of reference images Iv1 representing the plurality of reference shape information obtained for the plurality of virtual camera positions P1 and the edge image Ir3 as the actual shape information related to the two-dimensional shape of the object captured in the actual image as shown in FIG. 3C is calculated. Figure 11 Figure 11 ​​​The illustrated edge image Ir3 is a binary image where the outline of an object is represented in black, and the rest in white. Here, for example, a relative rotational and parallel translation is performed on the edge image Ir3, which serves as actual shape information, relative to the reference image Iv1, which serves as reference shape information. At this time, a value representing the maximum consistency between the reference image Iv1 (as reference shape information) and the edge image Ir3 (as actual shape information) can be calculated, and this value represents the consistency between the edge image Ir3 (as actual shape information) and the reference image Iv1 (as reference shape information). In other words, for each virtual camera position P1, a value representing the maximum consistency between the reference image Iv1 (as reference shape information) and the edge image Ir3 (as actual shape information) can be calculated, and this value represents the consistency between the edge image Ir3 (as actual shape information) and the reference image Iv1 (as reference shape information). In other words, for each virtual camera position P1, the consistency between the reference image Iv1 (which serves as reference shape information) and the edge image Ir3 (which serves as actual shape information) when the consistency is maximized is set as the consistency between the edge image Ir3 (which serves as actual shape information) and the reference image Iv1 (which serves as reference shape information).

[0153] Furthermore, for example, Figure 12 As shown, among a plurality of virtual camera positions P1, the virtual camera position P1 with the highest consistency between the edge image Ir3 (actual shape information) and the reference image Iv1 (reference shape information) is detected. Therefore, for an object part, it is possible to search for the object part whose 2D shape (3dm) of its 3D model is captured with the highest consistency among the actual images obtained using camera CM, relative to the virtual images obtained from which of the plurality of virtual camera positions.

[0154] However, as Figure 13 As shown, assume the center of the 3D model 3dm of the object part is set as the origin of the reference point Po in a right-handed xyz coordinate system. For example, the center of gravity of the 3D model 3dm can be applied to the center. In this case, the positions P1 of multiple virtual cameras centered on the 3D model 3dm can be defined by the angle (also called latitude) α in the rotation direction centered on the x-axis, the angle (also called longitude) β in the rotation direction centered on the z-axis, and the distance D from the origin. Furthermore, the pose of the virtual camera that takes pictures of the 3D model 3dm of the object part from the virtual camera position P1 can be defined by the angle (also called roll angle) γ in the rotation direction centered on the straight line Ln1 drawn by the reference point Po and the virtual camera position P1.

[0155] Here, if the latitude a and the longitude β of the virtual camera position P1 are changed, the direction in which the 3D model 3dm of the object part is shot from the virtual camera position P1 changes. Therefore, if the latitude a and the longitude β of the virtual camera position P1 are changed, the shape of the 3D model 3dm of the object part, which is depicted in the reference image Iv1 as the reference shape information, for example, as shown in FIG. 9, can change in the shape of the two-dimensional shape of the outline. Figure 14

[0156] In addition, here, even if the distance D of the virtual camera position P1 is changed, the direction in which the 3D model 3dm of the object part is shot from the virtual camera position P1 does not change. Therefore, if the distance D of the virtual camera position P1 is changed, the shape of the 3D model 3dm of the object part, which is depicted in the reference image Iv1 as the reference shape information, for example, as shown in FIG. 10, can change in the size, but not in the shape of the two-dimensional shape of the outline. Figure 15

[0157] In addition, here, if the roll angle γ of the posture of the virtual camera of the virtual camera position P1 is changed, the shape of the 3D model 3dm of the object part, which is depicted in the reference image Iv1 as the reference shape information, for example, as shown in FIG. 11, can change in the orientation, but not in the shape or the size of the two-dimensional shape of the outline. Figure 16

[0158] ​​​In the first embodiment, the search processing section 61 performs the second search processing using the result of the first search processing after performing the first search processing in order to efficiently perform the search processing. In other words, the search processing includes the first search processing and the second search processing performed in sequence. The first search processing is processing of calculating a value indicating the degree of coincidence between the reference shape information and the actual shape information for a limited number of virtual camera positions Pl by making the latitude a and the longitude β somewhat roughly different, and detecting a virtual camera position Pl at which the degree of coincidence between the reference shape information and the actual shape information is the greatest. The second search processing is processing of further finely searching for a virtual camera position Pl at which the degree of coincidence between the reference shape information and the actual shape information is greater, based on the virtual camera position Pl detected in the first search processing. In the first search processing and the second search processing, when the value indicating the degree of coincidence between the reference shape information and the actual shape information is calculated, for example, a roll angle γ at which the degree of coincidence is greater is also obtained, and the value indicating the degree of coincidence is calculated also taking the roll angle γ into account. Thus, the latitude a, the longitude β, the distance D, and the roll angle γ related to the virtual camera position Pl at which the degree of coincidence between the reference shape information and the actual shape information is the greatest can be obtained as the result of the search processing. With these latitude a, longitude β, distance D, and roll angle γ, the posture of the 3D model 3dm with reference to the virtual camera position Pl at which the degree of coincidence between the reference shape information and the actual shape information is the greatest can be recognized. Therefore, according to the result of the search processing, the posture of the object part with reference to the position of the camera CM that has captured the actual image of the object part can be recognized. In other words, according to the result of the search processing, the posture of the object part captured in the actual image can be recognized. Here, for example, the posture of the object part can be recognized in the form of the latitude a, the longitude β, the distance D, and the roll angle γ.

[0159] <3-2-2. Functional Configuration of Search Processing Section>

[0160] The search processing section 61 includes, for example, a first search processing section 611 and a second search processing section 612 as a plurality of functional processing sections. These first search processing section 611 and second search processing section 612 are realized, for example, in the control section 45 (more specifically, the arithmetic section 45a) by the CPU reading and executing the program Pg1 stored in the storage section 45b. The first search processing section 611 is a portion that performs processing related to the first search processing, and the second search processing section 612 performs processing related to the second search processing.

[0161] <3-2-2-1. First Search Processing Section>

[0162] The first search processing section 611 includes, for example, a density determination section 6111, a first shape information acquisition section 6112, a first calculation section 6113, and a first detection section 6114 as a plurality of functional processing sections. These density determination section 6111, first shape information acquisition section 6112, first calculation section 6113, and first detection section 6114 are realized, for example, in the control section 45 (more specifically, the arithmetic section 45a) by the CPU reading and executing the program Pg1 stored in the storage section 45b. In the first search processing section 611, one-time search processing can be realized, for example, by the first shape information acquisition section 6112, first calculation section 6113, and first detection section 6114.

[0163] <<Density determination section 6111>>

[0164] The density determination section 6111 determines, for example, a region in which the density of the outline of the part is low (also referred to as a low-density region) with respect to the edge image serving as the actual shape information acquired by the outline extraction section 592. In the density determination section 6111, for example, the proportion of the outline of the part in each region of a predetermined size (also referred to as a unit determination region) in the edge image serving as the actual shape information is calculated as the density of the outline of the part, and a unit determination region in which the density is a predetermined value or less is determined as a low-density region. The predetermined size of the unit determination region can be set, for example, in accordance with the size of the reference image described above as the reference shape information. The predetermined value can be set, for example, to zero or the like.

[0165] For example, in the case where zero is applied to the predetermined value, a unit determination region in which the outline of the part is not included at all in the edge image serving as the actual shape information is determined as a low-density region. The result of the determination by the density determination section 6111 is given to the first calculation section 6113 and the second search processing section 612. Here, it is assumed that the outline of the target part is not included in the low-density region. Therefore, for example, in the first calculation section 6113 and the second calculation section 6124 described below, the low-density region in the edge image serving as the actual shape information is excluded as a target of calculation for calculating a value indicating the degree of coincidence between the actual shape information and the reference shape information, and thus the amount of calculation required for search processing can be reduced.

[0166] <<First shape information acquisition section 6112>>

[0167] The first shape information acquisition section 6112 acquires reference shape information generated for each of a plurality of first virtual camera positions, for example, based on the three-dimensional design information on the object part stored in the storage section 45b, assuming that a 3D model of the object part is imaged from each of a plurality of virtual camera positions (also referred to as first virtual camera positions), by virtually setting one virtual camera position for each of the plurality of virtual faces included in a face aggregate (also referred to as a face aggregate) that surrounds the 3D model of the object part with a virtual sphere. Here, the reference shape information is information generated based on the three-dimensional design information on the object part, and is information on a two-dimensional shape of the 3D model of the object part in a virtual image that can be acquired from imaging of the virtual camera position. The reference point of the 3D model of the object part can be set to a center point such as a center of gravity of the 3D model of the object part, for example. Here, by setting the number of the plurality of virtual faces constituting the face aggregate to Ml (Ml is a natural number of 2 or more), the plurality of first virtual camera positions is limited to Ml first virtual camera positions. In each virtual face, a first virtual camera position is virtually set at a predetermined position of the virtual face, for example. As the predetermined position, the position of the center of the virtual face is applied, for example. As the center of the virtual face, the center of gravity of the virtual face is applied, for example.

[0168] In the present first embodiment, as each of the plurality of virtual faces constituting the face aggregate, a triangular face is applied, and as the face aggregate, a polyhedron composed of a plurality of triangular faces is applied. In other words, as the face aggregate, a spherical polyhedron composed of a plurality of triangles is applied. Thus, the setting of the face aggregate including the plurality of virtual faces can be easily performed. Here, as the triangle, an equilateral triangle is applied. If the virtual face is a triangular face, the center of the virtual face can be the center of gravity of the triangle, or the incenter, for example.

[0169] The first shape information acquisition section 6112 can acquire the reference shape information generated for each of the plurality of first virtual camera positions, for example, by setting the plurality of first virtual camera positions based on the three-dimensional design information on the object part stored in the storage section 45b, and generating the reference shape information for each of the plurality of first virtual camera positions.

[0170] Here, for example, as Figure 13As shown, the positions and poses of multiple virtual faces can be defined by setting the xyz coordinates of a right-handed system with the reference point Po of the 3dm model of the object part as defined by the 3D design information related to the object part as the origin. Furthermore, the positions of the multiple first virtual camera positions can be defined by the angle (latitude) α in the rotation direction centered on the x-axis, the angle (longitude) β in the rotation direction centered on the z-axis, and the distance D from the origin.

[0171] Figure 17 This is a diagram illustrating an example of how to set multiple first virtual camera positions P11. Figure 18 This diagram illustrates an example of how the position P11 of the first virtual camera in virtual plane St1 is set. Figure 17 To avoid complicating the accompanying diagrams, only the origin of the reference point Po in the right-handed xyz coordinate system is shown, and the three axes of x, y, and z, as well as the angles (latitude) α and (longitude) β, are omitted. Furthermore, Figure 17 The description of the angle (roll angle) γ in the rotational direction centered on the straight line passing through the origin, which serves as the reference point Po, and the 3dm virtual camera position P11, which is used to photograph the pose of the 3D model of the object part, is also omitted. Figure 17 In the diagram, the outline of the near-front portion of the polyhedron is depicted with solid lines, while the outline of the deep portion is depicted with thin dashed lines. Additionally, Figure 17 For convenience, as an example of a 3D model of an object part, a part with hexagonal upper and lower surfaces is shown, and the outline of the part is depicted with thick dashed lines.

[0172] In the first shape information acquisition unit 6112, for example, Figure 17 As shown, a face assembly As1 is virtually defined, comprising a plurality of triangles St1 forming a virtual sphere that surrounds the 3D model 3dm of the object part, centered at a reference point Po. Furthermore, for each of the plurality of triangles of virtual face St1, a plurality of first virtual camera positions P11 are virtually defined as a plurality of virtual camera positions P1.

[0173] Figure 17 In the example of the face assembly As1, it represents a spherical polyhedron composed of approximately 200 virtual triangular faces St1. Figure 17 For convenience, the virtual faces St1 of the three triangles among the plurality of virtual faces St1 are labeled with the symbol "St1". Furthermore, for convenience, each of the three virtual faces St1 of the triangles, namely virtual faces St1a, St1b and St1c, is labeled with a pear-skin-patterned shading. Figure 17In the middle, the first virtual camera position P11 respectively set to 3 virtual surfaces St1 of the plurality of triangular virtual surfaces St1 is illustrated with a black circular mark. More specifically, the first virtual camera position P11 set to the virtual surface St1a is labeled with "P11a", the first virtual camera position P11 set to the virtual surface St1b is labeled with "P11b", and the first virtual camera position P11 set to the virtual surface St1c is labeled with "P11c". Here, for example as shown in Figure 18 the center of the virtual surface St1 of the triangle as the predetermined position of the virtual surface St1 of the triangle. In other words, for example, the first virtual camera position P11 is set to the center of each virtual surface St1 of the triangle as the predetermined position of each virtual surface St1 of the triangle.

[0174] In addition, in the first shape information acquisition section 6112, for example, with respect to each of the plurality of first virtual camera positions P11 as shown in Figure 17 the 3D model 3dm of the object part in the virtual image obtainable from the photographing of the first virtual camera position P11. At this time, the photographing direction of the virtual camera photographing the 3D model 3dm of the object part from the first virtual camera position P11 is set to the direction from the first virtual camera position P11 toward the reference point Po of the 3D model 3dm of the object part. In addition, the roll angle γ of the posture of the virtual camera photographing the 3D model 3dm of the object part from the first virtual camera position P11 is set to zero (0) degrees, for example.

[0175] In the lower part of Figure 17 , an image (reference image) Iv1 showing the outline of the 3D model 3dm of the object part as a two-dimensional shape in the virtual image obtainable from the photographing of the 3D model 3dm of the object part from each first virtual camera position P11 is illustrated as the reference shape information generated with respect to each of the three first virtual camera positions P11.

[0176] However, it is assumed that the distance of the camera CM from the object part changes due to movement of the object part, or error in the position where the object part is mounted in the substrate processing device 1, or the like. In this case, the size of the object part in the actual image obtained by photographing with the camera CM can change. In response to this, as described above, if the distance D of the virtual camera position P1 is changed, the size of the two-dimensional shape of the outline of the 3D model 3dm of the object part in the reference image Iv1 as the reference shape information changes. Therefore, if the case where the distance of the camera CM from the object part changes is assumed, the first shape information acquisition section 6112 needs to virtually set a plurality of first virtual camera positions P11 different in the distance D, and obtain the reference shape information generated with respect to each first virtual camera position P11.

[0177] Therefore, the first shape information acquisition section 6112 acquires reference shape information generated for each of M1XT1 first virtual camera positions P11, for example, based on the three-dimensional design information on the object part stored in the storage section 45b, assuming a case where a 3D model of the object part is imaged from each of M1XT1 first virtual camera positions P11. The M1XT1 first virtual camera positions P11 are a plurality of virtual camera positions set by the first shape information acquisition section 6112 by virtually setting T1 surface assemblies having mutually different distances from a reference point of the 3D model of the object part, and virtually setting one virtual camera position for each of M1 virtual surfaces of each of the T1 surface assemblies. Here, the reference shape information is information generated based on the three-dimensional design information on the object part, and is information on a two-dimensional shape of the 3D model of the object part in a virtual image that can be acquired from the imaging of the first virtual camera position P11. M1 is appropriately set in a range of, for example, 100 to 300 or so. T1 is appropriately set in a range of, for example, 3 to 30 or so.

[0178] Figure 19 is a drawing to illustrate an example of a manner in which the T1 surface assemblies having mutually different distances from the reference point Po of the 3D model 3dm of the object part are virtually set. Figure 19 In, the outer edges of the T1 virtual spherical surfaces (also referred to as virtual spherical surfaces) are depicted with thin double-dot chain lines. The T1 virtual spherical surfaces are virtually generated based on the three-dimensional design information on the object part. The T1 virtual spherical surfaces have mutually different distances from the reference point Po of the 3D model 3dm of the object part. Further, each of the T1 virtual spherical surfaces surrounds the 3D model 3dm of the object part with the reference point Po of the 3D model 3dm of the object part as a center. Figure 19 In, the drawings other than the outer edges of the 3 virtual spherical surfaces Sv1, Sv2, SvT1 among the outer edges of the T1 virtual spherical surfaces Sv1, Sv2,..., SvT1 are omitted. The virtual spherical surface Sv1 is the first virtual spherical surface among the T1 virtual spherical surfaces having a distance D1 from the reference point Po. The virtual spherical surface Sv2 is the second virtual spherical surface among the T1 virtual spherical surfaces having a distance D2 from the reference point Po. The virtual spherical surface SvT1 is the T1th virtual spherical surface among the T1 virtual spherical surfaces having a distance DT1 from the reference point Po.

[0179] In the first shape information acquisition section 6112, for example, along each of the T1 virtual spherical surfaces Sv1, Sv2,..., SvT1 shown in Figure 19 , one virtual camera position is virtually set for each of the M1 virtual surfaces of each of the T1 surface assemblies. Figure 17The face set Asl is shown. Thus, Tl face sets Asl are virtually set. The Tl face sets Asl have shapes having a similar relationship of expansion and contraction with respect to each other with the reference point Po of the 3D model 3dm of the object part as the center. In other words, the Tl face sets Asl differ in size from each other, but have the same shape and the same posture with respect to the 3D model 3dm. In addition, in the first shape information acquisition section 6112, a first virtual camera position Pl 1 is virtually set for each of the Ml virtual faces Stl in each of the Tl face sets Asl. Thus, in each of the Tl face sets Asl, Ml first virtual camera positions Pl 1 are virtually set. In other words, Ml x Tl first virtual camera positions Pl 1 are virtually set. Furthermore, in the first shape information acquisition section 6112, assuming a case in which the 3D model 3dm of the object part is imaged from each of the Ml x Tl first virtual camera positions Pl 1, reference shape information is generated for each of the Ml x Tl first virtual camera positions Pl 1.

[0180] <<First calculation section 6113>>

[0181] The first calculation section 6113 calculates a value indicating the degree of coincidence between the information on the two-dimensional shape of the object captured in the actual image (actual shape information) and the reference shape information for each of the plurality of first virtual camera positions Pl 1.

[0182] Here, as described above, for example, assume a case in which the reference shape information generated for each of the Ml x Tl first virtual camera positions Pl 1 is acquired by the first shape information acquisition section 6112. In this case, the first calculation section 6113 calculates a value indicating the degree of coincidence between the information on the two-dimensional shape of the object captured in the actual image (actual shape information) and the reference shape information for each of the above-described Ml x Tl first virtual camera positions Pl 1.

[0183] Here, regarding the calculation process of the value indicating the degree of coincidence between the actual shape information and the reference shape information with respect to one first virtual camera position Pl 1 by the first calculation section 6113, one specific example is described.

[0184] Figures 20 to 24 is a diagram for explaining one specific example of the calculation process of the value indicating the degree of coincidence between the actual shape information and the reference shape information with respect to one first virtual camera position Pl 1.

[0185] Figure 20The image Iv1 indicates an example of a reference image Iv1 related to the first virtual camera position P11 as reference shape information. In the first calculation section 6113, for example, a value indicating the degree of coincidence of the two-dimensional shape of the outline between the reference image Iv1 as reference shape information acquired by the first shape information acquisition section 6112 and the edge image Ir3 as actual shape information acquired by the image processing section 59 is calculated. Figure 8

[0186] When the first calculation section 6113 calculates a value indicating the degree of coincidence between the edge image as actual shape information and the reference image as reference shape information for each of the first virtual camera positions P11, the calculation is performed in the following order, for example: [process la] a process of detecting the amount of shift of the orientation of the two-dimensional shape of the outline corresponding to the shift of the roll angle γ between the edge image and the reference image, [process lb] a process of rotating the edge image corresponding to the shift of the roll angle γ, and [process lc] a process of detecting the position of the region having the greatest degree of coincidence with the reference image using the rotated edge image as the target.

[0187] In the above process la, the first calculation section 6113 detects the amount of shift of the orientation of the two-dimensional shape of the outline between the edge image as actual shape information and the reference image as reference shape information using a Rotate Invariability Phase Only Correlation (RIPOC) method, for example. The amount of shift of the orientation of the two-dimensional shape of the outline is the amount of shift of the direction of rotation of the outline on the image. Here, the first calculation section 6113 divides the edge image Ir3 as actual shape information input from the image processing section 59 into a plurality of comparison target regions Re1, and then calculates the amount of shift γ1 in the direction of rotation in which the degree of coincidence of the outline between the reference image as reference shape information and the edge image as actual shape information is greatest using the RIPOC method for each of the comparison target regions Re1, as shown in FIG. 12, for example. Figure 21

[0188] The size of each of the comparison target regions Re1 is set to be one time or more and several times (for example, three times) or less of the size of the reference image as reference shape information in each of the longitudinal direction and the lateral direction, for example. In addition, the plurality of comparison target regions Re1 are set so as to partially overlap each other in adjacent comparison target regions Re1. Figure 21 In the example of FIG. 12, six comparison target regions Re1 are set in the edge image Ir3. Figure 21 ​​In the above, the outer edge of the first comparison object region Re1, i.e., the first comparison object region Re11, is depicted with a thick dashed line. The outer edge of the second comparison object region Re1, i.e., the second comparison object region Re12, is depicted with a thin dashed line. The outer edge of the third comparison object region Re1, i.e., the third comparison object region Re13, is depicted with a thick dashed line. The outer edge of the fourth comparison object region Re1, i.e., the fourth comparison object region Re14, is depicted with a thin dashed line. The outer edge of the fifth comparison object region Re1, i.e., the fifth comparison object region Re15, is depicted with a thick double dashed line. The outer edge of the sixth comparison object region Re1, i.e., the sixth comparison object region Re1, is depicted with a thin double dashed line. Here, the first calculation unit 6113 detects the comparison object region Re1 among the plurality of comparison object regions Re1, the one comparison object region Re1 with the highest contour consistency with the reference image as reference shape information, and the offset γ1 in the rotation direction with the highest contour consistency between the one comparison object region Re1 and the reference image as reference shape information.

[0189] In process 1b described above, the first calculation unit 6113 rotates the edge image, which serves as actual shape information, by correcting the offset γ1 detected in process 1a. This generates a rotated edge image in which the two-dimensional shape of the contour in the edge image, which serves as actual shape information, and the two-dimensional shape of the contour in the reference image, which serves as reference shape information, are aligned in orientation. Here, the first calculation unit 6113 rotates the edge image of the comparison object region Re1 detected in process 1a, by correcting the offset γ1 detected in process 1a. This generates a rotated edge image in which the two-dimensional shape of the contour in the comparison object region Re1, which constitutes actual shape information, and the two-dimensional shape of the contour in the reference image, which serves as reference shape information, are aligned in orientation. Figure 22 and Figure 23 For example, in Chinese Figure 22 The edge image rotation offset γ of the fourth comparison object region Re1, i.e., the fourth comparison object region Re14, is generated. Figure 23 The state of the rotated edge image Ir4 is shown. Figure 23 In the image, the outer edge of the fourth comparison object region Re14 before rotation is schematically represented by a thin double-dotted line.

[0190] In the above-described processing 1c, the first calculation section 6113 performs template matching using a reference image that is the reference shape information, for example, with respect to the rotated edge image related to the actual shape information generated in the above-described processing 1b. Here, the first calculation section 6113, for example, scans the reference image within the rotated edge image, and detects the position of a region in which the degree of coincidence (similarity) of each partial region within the rotated edge image and the reference image is the highest. Thus, in the edge image that is the actual shape information, the position of a region in which the degree of coincidence with the two-dimensional shape of the outline of the 3D model 3dm of the target part in the reference image that is the reference shape information is the highest (also referred to as a matching candidate position) can be detected. The degree of coincidence (similarity) here is not particularly limited, but for example, the degree of coincidence of the reference image with respect to the partial region of the rotated edge image when the partial region of the rotated edge image and the reference image are overlapped can be applied as the degree of coincidence. The degree of coincidence (similarity) can be expressed, for example, using a well-known score (also referred to as a matching score) that represents the degree of coincidence (similarity), such as a sum of squared differences of pixel values, a sum of absolute values of differences of pixel values, normalized cross-correlation, or zero-mean normalized cross-correlation. Here, when the above-described template matching is performed, a value that represents the degree of coincidence (similarity) when the degree of coincidence (similarity) of the partial region within the rotated edge image and the reference image is the highest is calculated, as a value that represents the degree of coincidence between the edge image that is the actual shape information and the reference image that is the reference shape information with respect to the first virtual camera position P11. In other words, with respect to each first virtual camera position P11, the degree of coincidence when the degree of coincidence of the reference image with respect to the partial region within the rotated edge image is the highest is set as the degree of coincidence between the edge image that is the actual shape information and the reference image that is the reference shape information. Here, the value that represents the degree of coincidence can be, for example, a matching score that represents the degree of coincidence. For example, the greater the degree of coincidence between the edge image that is the actual shape information and the reference image that is the reference shape information, the greater the matching score that represents the degree of coincidence can be, and the smaller the matching score that represents the degree of coincidence can be. In other words, there are cases in which the greater the matching score that represents the degree of coincidence, the greater the degree of coincidence between the edge image that is the actual shape information and the reference image that is the reference shape information can be evaluated to be, and the smaller the matching score that represents the degree of coincidence, the greater the degree of coincidence between the edge image that is the actual shape information and the reference image that is the reference shape information can be evaluated to be. Hereinafter, unless otherwise noted, the greater the degree of coincidence between the edge image that is the actual shape information and the reference image that is the reference shape information, the greater the matching score that represents the degree of coincidence. Figure 24 In the example, the outer edge of the matching candidate position Pm0 in the edge image Ir4 is depicted by a thin double-dot chain line.

[0191] <<1st detection unit 6114>>

[0192] The 1st detection unit 6114 detects, based on the calculation result of the 1st calculation unit 6113, the 1st virtual camera position P11 in which the degree of coincidence between the information on the two-dimensional shape of the object captured in the actual image (actual shape information) and the reference shape information is the highest among the plurality of 1st virtual camera positions P11, that is, the high-degree-of-coincidence virtual camera position. Here, for example, the 1st virtual camera position P11 in which the matching score calculated by the 1st calculation unit 6113 between the information on the two-dimensional shape of the object captured in the actual image (actual shape information) and the reference shape information is the highest among the plurality of 1st virtual camera positions P11 can be detected as the high-degree-of-coincidence virtual camera position.

[0193] Here, as described above, for example, it is assumed that the reference shape information generated for each of the M1xT1 1st virtual camera positions P11 is acquired by the 1st shape information acquisition unit 6112. In this case, the 1st detection unit 6114 detects, based on the calculation result of the 1st calculation unit 6113, the virtual camera position in which the degree of coincidence between the actual shape information and the reference shape information is the highest among the M1xT1 1st virtual camera positions P11, that is, the high-degree-of-coincidence virtual camera position. Here, for example, the 1st virtual camera position P11 in which the matching score calculated by the 1st calculation unit 6113 between the information on the two-dimensional shape of the object captured in the actual image (actual shape information) and the reference shape information is the highest among the M1xT1 1st virtual camera positions P11 can be detected as the high-degree-of-coincidence virtual camera position.

[0194] In addition, the 1st detection unit 6114, for example, when detecting the high-degree-of-coincidence virtual camera position, detects, as the object part candidate position, the matching candidate position detected by the 1st calculation unit 6113 for the high-degree-of-coincidence virtual camera position.

[0195] <3-2-2-2. 2nd search processing unit>

[0196] The second search processing unit 612 includes, for example, a reference object region setting unit 6121, a segmentation surface generation unit 6122, a second shape information generation unit 6123, a second calculation unit 6124, and a second detection unit 6125, serving as a plurality of functional processing units. These reference object region setting units 6121, segmentation surface generation units 6122, second shape information generation units 6123, second calculation units 6124, and second detection units 6125 are implemented, for example, in the control unit 45 (more specifically, the arithmetic unit 45a), by the CPU reading and executing the program Pg1 stored in the storage unit 45b. The second search processing unit 612 can, for example, perform two search processes using the segmentation surface generation unit 6122, the second shape information generation unit 6123, the second calculation unit 6124, and the second detection unit 6125.

[0197] <<Comparison Object Area Setting Section 6121>>

[0198] The comparison object region setting unit 6121 sets a region (also called the comparison object region) in the edge image obtained by the contour extraction unit 592, which serves as actual shape information, for matching (comparison) processing by the second calculation unit 6124 based on the candidate positions of the object parts detected by the first detection unit 6114. This reduces the computational load of the second calculation unit 6124 and improves its processing efficiency.

[0199] Figure 25 This diagram illustrates a specific example of setting a reference object region Re2 for the edge image Ir3, which serves as actual shape information, obtained by the contour extraction unit 592. For example... Figure 25 As shown, for the edge image Ir3, the region containing the candidate position Pm1 of the object part detected by the first detection unit 6114 and whose size is larger than the candidate position Pm1 of the object part is set as the reference object region Re2. Figure 25 In the diagram, the outer edge of the candidate position Pm1 is depicted with a thin double-dotted line, while the outer edge of the reference area Re2 is depicted with a thick dashed line. The size of the reference area Re2 is, for example, based on the size of the candidate position Pm1, and is set to be greater than 1 and less than several times (e.g., 2 times) in both the longitudinal and transverse directions.

[0200] <<Segmentation Surface Generation Unit 6122>>

[0201] The segmentation surface generation unit 6122 generates a plurality of segmented virtual surfaces (also called virtual segmentation surfaces) by segmenting a plurality of virtual surfaces St1, including virtual surfaces with virtual camera positions detected by the first detection unit 6114 that are virtually set. The plurality of virtual surfaces St1 constitute a surface assembly As1, and are arranged along a virtual sphere that surrounds the 3D model 3dm of the object part with a reference point Po as the center.

[0202] Figure 26 is a diagram schematically showing a specific example of dividing the high-consistency virtual face Stlm into a plurality of virtual divided faces St2. Figure 26 In the case where Figure 17 is shown, a case where each of the plurality of virtual faces Stl constituting the face aggregate Asl is a triangular face is shown. Figure 26 In the case where Figure 26 is shown, the divided face generating section 6122 divides the high-consistency virtual face Stlm into three virtual divided faces St2 as the plurality of virtual divided faces St2, for example, by connecting three line segments of the three vertices of the high-consistency virtual face Stlm and the high-consistency virtual camera position Pllm, respectively. Thereby, the division of the virtual face can be easily performed.

[0203] Here, as described above, for example, a case where the reference shape information generated for each of the MlxTl first virtual camera positions Pll is acquired by the first shape information acquiring section 6112 is assumed. In this case, the divided face generating section 6122 divides each of T2 (T2 is a natural number of 2 or more) of the virtual faces Stl included in the Ml virtual faces Stl of each of the Tl face aggregates Asl and having distances from the reference point Po of the 3D model 3dm of the target part different from each other, in the same rule. Thereby, the divided face generating section 6122 generates M2xT2 virtual divided faces St2 by generating M2 (M2 is a natural number of 2 or more) of the virtual divided faces St2 for each of the T2 virtual faces Stl. Here, each of the T2 virtual faces Stl is a plurality of virtual faces Stl that are on the high-consistency virtual camera position Pllm side from the reference point Po of the 3D model 3dm of the target part and intersected by a straight line passing through the reference point Po and the high-consistency virtual camera position Pllm, and have distances from the reference point Po different from each other. The T2 is the same as the Tl, for example. The T2 can be smaller than the Tl, for example.

[0204] Figure 27 is an image diagram schematically showing an example of the T2 virtual faces Stl. Figure 27 In the case where Figure 27 In the case where Figure 27For convenience, three of the T2 virtual faces St1 are depicted, and the illustrations of the other virtual faces St1 are omitted. As shown in FIG. 10, each of the T2 virtual faces St1 intersects with a portion of the straight line Ln11 on the higher- consistency virtual camera position Pllm side than the reference point Po. Figure 27

[0205] Here, the T2 virtual faces St1 have shapes that have a similar relationship of expanding and contracting in a radial direction from the reference point Po of the 3D model 3dm of the object part. When viewed from the reference point Po of the 3D model 3dm of the object part, each of the T2 virtual faces St1 is divided by a line having a shape that has a similar relationship of expanding and contracting in a radial direction from the reference point Po, thereby generating M2 x T2 virtual division faces St2. Thus, in the T2 virtual faces St1, for each of the M2 virtual division faces St2, there is a state in which there are T2 virtual division faces St2 having shapes that have a similar relationship of expanding and contracting in a radial direction from the reference point Po of the 3D model 3dm of the object part.

[0206] The same rule related to the division of the T2 virtual faces St1 can be a rule in which each of the T2 virtual faces St1 is divided by a line having a shape that has a similar relationship of expanding and contracting in a radial direction from the reference point Po of the 3D model 3dm of the object part, when viewed from the reference point Po of the 3D model 3dm of the object part. In other words, the same rule related to the division of the T2 virtual faces St1 can be a rule in which each of the T2 virtual faces St1 is divided by a line having a shape that has a similar relationship of expanding and contracting in a radial direction from the reference point Po, when viewed from the reference point Po of the 3D model 3dm of the object part. In further other words, the same rule related to the division of the T2 virtual faces St1 can be a rule in which, between the T2 virtual faces St1, for each of the M2 virtual division faces, a manner in which the same shape of T2 division virtual faces is generated, when viewed from the reference point Po of the 3D model 3dm of the object part. The same rule related to the division of the T2 virtual faces St1 can be, for example, a rule in which a division object face is divided into a plurality of faces by connecting a center point of the division object face, which is also referred to as a division object face, and a plurality of line segments of all vertices of the division object face, respectively. Thus, the division of the division object face can be easily performed. As shown in FIG. 10, if each of the plurality of virtual faces St1 that constitute the face aggregate As1 is a triangular face, the M2 can be three. In this case, each of the T2 virtual faces St1 intersects with a portion of the high-consistency virtual face Stlm shown in FIG. 11 on the higher-consistency virtual camera position Pllm side than the reference point Po. Figure 17 Figure 26 As shown in FIG. 11, if each of the plurality of virtual faces St1 that constitute the face aggregate As1 is a triangular face, the M2 can be three. In this case, each of the T2 virtual faces St1 intersects with a portion of the high-consistency virtual face Stlm shown in FIG. 11 on the higher-consistency virtual camera position Pllm side than the reference point Po.​​

[0207] <<Second Shape Information Generation Unit 6123>>

[0208] The second shape information generation unit 6123, based on the three-dimensional design information related to the object part stored in the storage unit 45b, assumes that a 3D model 3dm of the object part is photographed from each of the plurality of virtual camera positions (second virtual camera positions) P12 of the plurality of segmentation surfaces St2 generated by the segmentation surface generation unit 6122. For each of the plurality of second virtual camera positions P12, reference shape information is generated. The plurality of second virtual camera positions P12 are multiple virtual camera positions set by virtually setting one virtual camera position for each of the plurality of virtual segmentation surfaces St2 generated by the segmentation surface generation unit 6122. Here, the reference shape information is also information generated based on the three-dimensional design information related to the object part, and is information related to the two-dimensional shape of the 3D model 3dm of the object part in a virtual image obtainable from the virtual camera position (more specifically, the second virtual camera position P12). The virtual image can be generated, for example, by projecting the 3D model 3dm onto a virtual plane using rendering or other processing.

[0209] In each virtual segmentation plane St2, for example, the position P12 of the second virtual camera is virtually set at a predetermined position on the virtual segmentation plane St2. The predetermined position is, for example, the position of the center of the virtual segmentation plane St2. The center of the virtual segmentation plane St2 is, for example, the centroid of the virtual segmentation plane St2. For example, Figure 26 As shown, the position P12 of the second virtual camera is set on each virtual segmentation surface St2. If the plurality of virtual segmentation surfaces St2 are triangular faces, then the center of the virtual segmentation surface St2 can be, for example, the centroid of the triangle or the incenter.

[0210] In the second shape information generation unit 6123, for example for Figure 26 Each of the plurality of second virtual camera positions P12 shown generates reference shape information related to the two-dimensional shape of the 3D model 3dm of the object part in the virtual image obtainable from the second virtual camera position P12. At this time, the shooting direction of the virtual camera shooting the 3D model 3dm of the object part from the second virtual camera position P12 is set to the direction from the second virtual camera position P12 toward the reference point Po of the 3D model 3dm of the object part. The roll angle γ of the pose of the virtual camera shooting the 3D model 3dm of the object part from the second virtual camera position P12 is defined, for example, as zero (0) degrees.

[0211] Here, as described above, for example, assume that the reference shape information generated for each of the M1×T1 first virtual camera positions P11 is obtained by the first shape information acquisition unit 6112. In this case, the second shape information generation unit 6123, for example, based on the three-dimensional design information related to the object part stored in the storage unit 45b, assumes that the 3D model 3dm of the object part is captured from each of the M2×T2 second virtual camera positions P12, and generates reference shape information for each of the M2×T2 second virtual camera positions P12. The M2×T2 second virtual camera positions P12 are the M2×T2 virtual camera positions set by virtually setting one virtual camera position for each of the aforementioned M2×T2 virtual dividing surfaces St2. Here, the reference shape information is also information generated based on the three-dimensional design information related to the object part, and is information related to the two-dimensional shape of the 3D model 3dm of the object part in the virtual image that can be obtained from the capture from the second virtual camera position P12. Virtual images can be generated, for example, by projecting a 3D model (3dm) onto a virtual plane using rendering or other processing methods.

[0212] Here, the positions P12 of the second virtual cameras, M2×T2, are determined by, for example, among the T2 virtual surfaces St1, as follows: Figure 26 The second virtual camera position P12 is set at each of the plurality of virtual segmentation surfaces St2 as shown.

[0213] <<Second Calculation Department 6124>>

[0214] The second calculation unit 6124 calculates a value for each of the plurality of second virtual camera positions P12, representing the consistency between information related to the two-dimensional shape of the object captured in the actual image (actual shape information) and reference shape information.

[0215] As described above, in this first embodiment, a second virtual camera position P12 is set for each of the plurality of virtual surfaces (virtual segmentation surfaces) St2 generated by dividing a plurality of virtual surfaces St1 into virtual surfaces (high-consistency virtual surfaces) St1m, each containing a virtual surface (high-consistency virtual surface) St1m detected by the first detection unit 6114, and for each second virtual camera position P12, a value representing the consistency between the actual shape information and the reference shape information is calculated. Therefore, the high-consistency virtual surface St1m of the first search process and the plurality of virtual segmentation surfaces St2 of the second search process are not unrelated surfaces, and for the plurality of virtual segmentation surfaces St2, an increase in at least one of the number and area can be reduced. As a result, the computational load for identifying the pose of the object part captured in the actual image can be reduced. Consequently, the pose identification of the object part can be performed efficiently in the substrate processing apparatus 1.

[0216] As described above, for example, assume that the reference shape information generated for each of the M1×T1 first virtual camera positions P11 is obtained by the first shape information acquisition unit 6112. In this case, the second calculation unit 6124 calculates a value representing the consistency between the information related to the two-dimensional shape of the object captured in the actual image (actual shape information) and the reference shape information for each of the M2×T2 second virtual camera positions P12. Thus, when the distance between the camera CM and the object part changes due to the movement of the object part or errors in the position of the object part mounted in the substrate processing apparatus 1, the pose recognition of the object part can be performed efficiently and effectively.

[0217] Here, a specific example will be given to explain the calculation process of the value of the consistency between the actual shape information and the reference shape information related to the position P12 of a second virtual camera in the second calculation unit 6124.

[0218] A specific example of the calculation process of the second calculation unit 6124, which represents the consistency between the actual shape information and the reference shape information related to a second virtual camera position P12, can be the same as a specific example of the calculation process of the first calculation unit 6113, which represents the consistency between the actual shape information and the reference shape information related to a first virtual camera position P11.

[0219] However, the second calculation unit 6124 calculates, for example, a reference image representing reference shape information obtained by the second shape information generation unit 6123, and a comparison object region Re2 representing an edge image set by the comparison object region setting unit 6121 as actual shape information. Figure 25 The numerical value of the consistency of the two-dimensional shape of the contours between the two sides.

[0220] When the second calculation unit 6124 calculates the value representing the consistency between the edge image, which is actual shape information, and the reference image, which is reference shape information, for each second virtual camera position P12, it performs the following steps in the following order: [Process 2a] processing to detect the offset of the orientation of the two-dimensional shape of the contour corresponding to the offset of the roll angle γ between the edge image and the reference image; [Process 2b] processing to rotate the edge image corresponding to the offset of the roll angle γ; and [Process 2c] processing to detect the position of the region with the highest consistency with the reference image, with the rotated edge image as the object.

[0221] In process 2a described above, similar to process 1a, the second calculation unit 6124, for example, uses the RIPOC method to detect the offset of the orientation of the two-dimensional shape of the contour between the edge image, which serves as actual shape information, and the reference image, which serves as reference shape information. This offset of the orientation of the two-dimensional shape of the contour is the offset of the rotation direction of the contour on the image. Here, the second calculation unit 6124, for example, uses the RIPOC method to detect the reference object region Re2 (which is set by the reference object region setting unit 6121 as the edge image, serving as actual shape information). Figure 25 The offset γ2 in the rotation direction that maximizes the consistency of the contour between the reference image (which serves as reference shape information) and the edge image (which serves as actual shape information). This offset γ2 is equivalent to the roll angle γ that maximizes the consistency between the reference image (which serves as reference shape information) and the edge image (which serves as actual shape information).

[0222] In process 2b described above, similar to process 1b, the second calculation unit 6124 rotates the edge image, which serves as actual shape information, by correcting the offset γ2 detected in process 2a. This generates a rotated edge image in which the two-dimensional shape of the contour in the edge image, which serves as actual shape information, and the two-dimensional shape of the contour in the reference image, which serves as reference shape information, are aligned in orientation. Here, the second calculation unit 6124 rotates the edge image of the reference object region Re2, for example, by correcting the offset γ2 detected in process 2a. This generates a rotated edge image in which the two-dimensional shape of the contour in the reference object region Re2, which constitutes actual shape information, and the two-dimensional shape of the contour in the reference image, which serves as reference shape information, are aligned in orientation.

[0223] In process 2c described above, similar to process 1c, the second calculation unit 6124, for example, uses the rotated edge image related to the actual shape information generated in process 2b as the object and performs template matching using a reference image as reference shape information. Here, the second calculation unit 6124, for example, scans the reference image within the rotated edge image and detects the position of the region within the rotated edge image where the consistency (similarity) with the reference image is greatest. Thus, in the edge image which serves as actual shape information, the position (matching candidate position) of the region with the greatest consistency with the two-dimensional shape of the 3dm outline of the object part in the reference image which serves as reference shape information can be detected. Although the consistency (similarity) referred to here is not particularly limited, it refers to the degree of consistency between the reference image and a portion of the rotated edge image when the rotated edge image overlaps with the reference image. This similarity can be represented by well-known scores such as the sum of squares of the differences in pixel values, the sum of the absolute values ​​of the differences in pixel values, normalized cross-correlation, or zero-mean normalized cross-correlation, as described above. Furthermore, when performing the template matching described above, a value representing the maximum similarity between a portion of the rotated edge image and the reference image is calculated and used as the similarity value between the edge image (actual shape information) and the reference image (reference shape information) related to the second virtual camera position P12. In other words, for each second virtual camera position P12, the similarity value at which the reference image has the maximum similarity with respect to a portion of the rotated edge image is set as the similarity between the edge image (actual shape information) and the reference image (reference shape information). Here, the value representing the similarity can also be, for example, a matching score representing the similarity.

[0224] <<Second Inspection Department 6125>>

[0225] The second detection unit 6125 detects the virtual camera position with the highest consistency between the information related to the two-dimensional shape of the object (actual shape information) captured in the actual image of the object part and the reference shape information among the high-consistency virtual camera position P11m detected by the first detection unit 6114 and the aforementioned M2×T2 second virtual camera positions P12. Therefore, the virtual camera position with greater consistency between the actual shape information and the reference shape information can be detected efficiently.

[0226] Here, for example, the matching score calculated by the first calculation unit 6113 for the high-consistency virtual camera position P11m and the matching score calculated by the second calculation unit 6124 for each of the M2×T2 second virtual camera positions P12 are compared. Moreover, for example, the virtual camera position with the highest matching score among the high-consistency virtual camera position P11m and the M2×T2 second virtual camera positions P12 is selected based on the information related to the two-dimensional shape of the object captured in the actual image of the captured object part (actual shape information) and the reference shape information.

[0227] Here, the virtual camera position detected by the second detection unit 6125 is set as the first virtual camera position detected by the second detection unit 6125 in the object part-related search process. This first virtual camera position detected by the second detection unit 6125 is, for example, the first reference virtual camera position described later. In this specification, the Xth (X is a natural number) refers to the Xth position when performing one object part-related search process.

[0228] <3-2-2-3. Repeated processing per unit>

[0229] For example, the search processing unit 61 performs the first unit processing on the target part by means of the second search processing unit 612, and then performs the nth unit processing (n is a natural number of 2 or more) more times.

[0230] A unit process refers to a specific process that is repeated two or more times when they are the same or similar to each other. In this first embodiment, the first unit process and the nth unit process are similar processes, and the nth unit processes are substantially the same to each other. Here, the first unit process means the first unit process in two or more unit processes. The natural number n greater than 2 in the nth unit process, i.e., the variable n, represents the nth unit process. In other words, the nth unit process means the nth unit process in two or more unit processes. For example, if the variable n is 2, then the nth unit process means the second unit process, i.e., the second unit process. From another point of view, when performing the nth unit process once or more, the natural number n greater than 2, i.e., the variable n, is set to the initial value of 2, and each time the nth unit process is performed, the variable n is incremented by 1, and the next nth unit process is performed. Moreover, the search processing unit 61 performing the first unit process and then performing the nth unit process once or more means, for example, that the search processing unit 61 performs the first unit process, the second unit process, and so on in sequence. For example, when the search processing unit 61 performs the first unit processing and then performs the nth unit processing once, the search processing unit 61 performs the first unit processing and the second unit processing in sequence. When the search processing unit 61 performs the first unit processing and then performs the nth unit processing twice, the search processing unit 61 performs the first unit processing, the second unit processing, and the third unit processing in sequence.

[0231] The first unit processing, for example, when performing two search processes related to the object part in the search processing unit 61, starts in response to the first detection of the virtual camera position by the second detection unit 6125.

[0232] Here, the search processing unit 61, through the second search processing unit 612, sequentially performs the following first-unit processing: the first A process, the first B process, the first C process, and the first D process. Furthermore, the search processing unit 61, through the second search processing unit 612, sequentially performs the following nA process, the nB process, the nC process, and the nD process in the nth unit processing. In other words, the search processing unit 61, through the second search processing unit 612, sequentially performs the following nA process, the nB process, the nC process, and the nD process in each of the more than one nth unit processing.

[0233] <<Process 1A>>

[0234] Process 1A involves the generation of a first M3×T3 virtual segmentation surfaces (M3 and T3 are natural numbers of 2 or more) by the segmentation surface generation unit 6122. In this process, the segmentation surface generation unit 6122 divides each of the plurality of virtual segmentation surfaces (also referred to as the first T3 virtual segmentation surfaces) generated by dividing the aforementioned T2 virtual surfaces St1 according to the same rule. Thus, the segmentation surface generation unit 6122 generates M3 virtual segmentation surfaces (also referred to as the first M3 virtual segmentation surfaces) for each of the first T3 virtual segmentation surfaces, thereby generating the first M3×T3 virtual segmentation surfaces. Here, the first T3 virtual segmentation surfaces include virtual segmentation surfaces containing the first reference virtual camera position (also referred to as the first reference virtual segmentation surface) detected by the second detection unit 6125, and all of them have different distances from the reference point Po of the 3dm 3dm dimensional of the object part's 3D model. Furthermore, the first T3 virtual dividing surfaces are a plurality of virtual dividing surfaces that intersect the reference point Po (3dm) of the 3D model of the object part with the first reference virtual camera position on the side near the reference point Po and the first reference virtual camera position. T3 is, for example, the same as T2. T3 may also be less than T2.

[0235] Here, the same rule related to the division of the first T3 virtual segmentation surfaces can be observed from the reference point Po of the 3dm 3D model of the object part, and the rule for dividing each of the first T3 virtual segmentation surfaces in the same form. In other words, when the same rule related to the division of the first T3 virtual segmentation surfaces can be observed from the reference point Po of the 3dm 3D model of the object part, the rule for dividing each of the first T3 virtual segmentation surfaces is a line with a similar shape that expands and shrinks towards each other in a radial direction centered on the reference point Po. Furthermore, in other words, the same rule related to the division of the first T3 virtual segmentation surfaces can be observed from the reference point Po of the 3dm 3D model of the object part, and the rule for dividing the first T3 virtual segmentation surfaces is a rule for dividing the first T3 virtual segmentation surfaces in a way that generates T3 segmentation virtual surfaces of the same shape for each of the M3 virtual segmentation surfaces between the first T3 virtual segmentation surfaces. Therefore, in the first T3 virtual dividing surfaces St2, there exists a state where each of the M3 virtual dividing surfaces has a shape that expands and shrinks radially from a reference point Po of the 3dm 3D model of the object part. The same rule related to the division of the first T3 virtual dividing surfaces can be, for example, a rule that divides the object surface into multiple surfaces by connecting the center point of the object surface to all vertices of the object surface with multiple line segments. This makes the division of the object surface easy. M3 can be, for example, the same as M2. If each of the first T3 virtual dividing surfaces St2 is a triangular surface, then M3 can be 3.

[0236] Figure 28 This is an image schematically representing an example of the first T3 virtual segmentation plane St2. Figure 28 In the diagram, the direction (also known as the second direction) Dr12 from the reference point Po of the 3dm 3D model of the object part toward the first reference virtual camera position Ps1 is represented by an arrow drawn with a thin line. The straight line Ln12 passing through the reference point Po and the first reference virtual camera position Ps1 is drawn with a thin double-dotted line. Figure 28 In the middle, the line passing through the reference point Po and the outer edge of each of the first T3 virtual dividing surfaces St2 is drawn with a thin dashed line. Figure 28 For simplicity, only three virtual dividing surfaces St2 out of the first T3 virtual dividing surfaces St2 are depicted; illustrations of other virtual dividing surfaces St2 are omitted. (See diagram for example.) Figure 28As shown, each of the first T3 virtual segmentation surfaces St2 intersects the portion of the reference point Po in the straight line Ln12 on the side near the first reference virtual camera position Ps1. Here, the first T3 virtual segmentation surfaces St2 have a shape that expands and contracts towards each other in a radial direction, centered on the reference point Po of the 3dm 3D model of the object part.

[0237] In this first A process, the generation method of the first M3×T3 virtual segmentation surfaces differs depending on the first virtual camera position (first reference virtual camera position) Ps1 detected by the second detection unit 6125. More specifically, the generation method of the first M3×T3 virtual segmentation surfaces differs depending on whether the first reference virtual camera position Ps1 is any one of the aforementioned M2×T2 second virtual camera positions P12 (also referred to as the first case of the first) or whether the first reference virtual camera position Ps1 is the high-consistency virtual camera position P11m detected by the first detection unit 6114 (also referred to as the second case of the first).

[0238] <<<Scenario 1 of the First Case>>>

[0239] When the second detection unit 6125 detects any one of the M2×T2 second virtual camera positions P12 as the first reference virtual camera position Ps1, the consistency between the two-dimensional shape-related information (actual shape information) of the object captured in the actual image of the captured object part and the reference shape information is greater than that of the high consistency virtual camera position P11m.

[0240] In other words, compared to the consistency between the actual shape information and the reference shape information related to the high-consistency virtual camera position P11m calculated by the first calculation unit 6113 in a single search process, the consistency between the actual shape information and the reference shape information related to any of the M2×T2 second virtual camera positions P12 calculated by the second calculation unit 6124 in the first unit process of the two search processes is greater. More specifically, for example, compared to the matching score representing the consistency between the actual shape information and the reference shape information related to the high-consistency virtual camera position P11m calculated by the first calculation unit 6113 in a single search process, the matching score representing the consistency between the actual shape information and the reference shape information related to any of the M2×T2 second virtual camera positions P12 calculated by the second calculation unit 6124 in the first unit process of the two search processes is greater.

[0241] Therefore, the segmentation surface generation unit 6122 takes the second virtual camera position P12 with the highest consistency between the two-dimensional shape-related information (actual shape information) of the object captured in the actual image of the captured object part and the reference shape information among the aforementioned M2×T2 second virtual camera positions P12, and uses it as the segmentation reference for the next virtual surface, i.e., the first reference virtual camera position Ps1, to generate the first M3×T3 virtual segmentation surfaces. Thus, for example, in two search processes, a virtual camera position with a higher consistency between the reference shape information and the actual shape information can be searched.

[0242] Here, the segmentation surface generation unit 6122 segments each of the aforementioned M2×T2 virtual segmentation surfaces St2, including the virtual segmentation surface containing the first reference virtual camera position Ps1 (the first reference virtual segmentation surface) and whose distances from the reference point Po (3dm) of the 3D model of the object part are different, using the same rule. Thus, the segmentation surface generation unit 6122 generates M3 virtual segmentation surfaces (i.e., the first M3 virtual segmentation surfaces) from each of the first T3 virtual segmentation surfaces St2, thereby generating the first M3×T3 virtual segmentation surfaces. The first T3 virtual segmentation surfaces St2 are a plurality of virtual segmentation surfaces St2 that intersect the reference point Po (3dm) of the 3D model of the object part near the first reference virtual camera position Ps1 and the straight line Ln12 passing through the reference point Po and the first reference virtual camera position Ps1.

[0243] Here, the same rule related to the division of the first T3 virtual segmentation surfaces St2 is as described above. It involves dividing the segmentation object surface into multiple surfaces by connecting the center point of the segmentation object (i.e., the surface to all vertices of the surface) with multiple line segments. For example, if each of the first T3 virtual segmentation surfaces St2 is a triangle, then each of the first T3 virtual segmentation surfaces St2 can be divided into M3 virtual segmentation surfaces (i.e., 3 virtual segmentation surfaces) by connecting 3 vertices to the 3-line segment from the 2nd virtual camera position P12. Thus, each of the first T3 virtual segmentation surfaces St2 can be divided into 3 triangular virtual segmentation surfaces. As a result, the first M3×T3 virtual segmentation surface is generated as an M3×T3 virtual segmentation surface.

[0244] Here, for example, assuming that detection Figure 26 The top left virtual camera position P12 out of the three second virtual camera positions P12 is used as the first reference virtual camera position Ps1. In this case, as... Figure 26 As shown, among the three virtual segmentation surfaces St2 generated by segmenting virtual surface St1, the virtual segmentation surface St2 containing the first reference virtual camera position Ps1 becomes the first reference virtual segmentation surface Ss1.

[0245] Figure 29 This is a diagram illustrating the first specific example of generating the first M3×T3 virtual segmentation surfaces St3 by the segmentation surface generation unit 6122. Figure 29 In this section, the division of the first reference virtual dividing surface Ss1 among the first T3 virtual dividing surfaces St2 is represented. The first T3 virtual dividing surface St2 is the T3 virtual dividing surface St2 among the M2×T2 virtual dividing surfaces St2 generated by the dividing surface generation unit 6122. The first T3 virtual dividing surface St2 has a shape that expands and shrinks towards each other in a radial direction, centered on the reference point Po of the 3dm reference point of the 3D model of the object part. Figure 29 In, such as Figure 17 As shown, this is a specific example of the case where each of the plurality of virtual faces St1 constituting the face assembly As1 is a triangular face. Figure 29 In the diagram, the position of the first reference virtual camera, Ps1, is indicated by a white circle. (For example...) Figure 29 As shown, the segmentation surface generation unit 6122, for example, divides the first reference virtual segmentation surface Ss1 into three virtual segmentation surfaces St3, which are the first M3 virtual segmentation surfaces, by connecting the three vertices of the first reference virtual segmentation surface Ss1 with the three line segments of the first reference virtual camera position Ps1.

[0246] Here, when viewed from the reference point Po of the 3dm 3D model of the object part, the first M3×T3 virtual dividing surfaces St3 are generated by dividing each of the first T3 virtual dividing surfaces St2 with lines having a similar shape that expands and shrinks towards each other in a radial direction centered on the reference point Po. Thus, among the first T3 virtual dividing surfaces St2, there exists a state where each of the M3 virtual dividing surfaces St3 has a shape that expands and shrinks towards each other in a radial direction centered on the reference point Po of the 3dm 3D model of the object part. The M3 can be, for example, the same as the M2.

[0247] like Figure 17 As shown, if each of the plurality of virtual faces St1 constituting the face assembly As1 is a triangular face, then M3 can be the same as M2, or there can be 3 of them. In this case, each of the first T3 virtual dividing faces St2 is the same as... Figure 29 Similarly, the first reference virtual segmentation surface Ss1 shown can be divided into three virtual segmentation surfaces St3 by connecting the three vertices to the position P12 of the second virtual camera. Thus, each of the T3 virtual segmentation surfaces St2 can be divided into three triangular virtual segmentation surfaces St3. As a result, an M3×T3 virtual segmentation surface St3 is generated as the first of the M3×T3 virtual segmentation surfaces.

[0248] <<<Scenario 2 of the first one>>>

[0249] When the second detection unit 6125 detects that the high-consistency virtual camera position P11m is the first reference virtual camera position Ps1, compared to any of the above M2×T2 second virtual camera positions P12, the high-consistency virtual camera position P11m has a greater consistency with the reference shape information of the two-dimensional shape-related information of the object captured in the actual image of the captured object part.

[0250] In other words, compared to the consistency between the actual shape information and the reference shape information related to the high-consistency virtual camera position P11m calculated by the first calculation unit 6113 in the first search process, the consistency between all the actual shape information and the reference shape information related to the M2×T2 second virtual camera positions P12 calculated by the second calculation unit 6124 in the second search process is smaller. More specifically, for example, compared to the matching score representing the consistency between the actual shape information and the reference shape information related to the high-consistency virtual camera position P11m calculated by the first calculation unit 6113 in the first search process, the matching score representing the consistency between all the actual shape information and the reference shape information related to the M2×T2 second virtual camera positions P12 calculated by the second calculation unit 6124 in the second search process is smaller.

[0251] Therefore, the segmentation surface generation unit 6122 uses the high-consistency virtual camera position P11m as the segmentation reference for the next virtual surface, i.e., the first reference virtual camera position Ps1, and generates the first M3×T3 virtual segmentation surfaces St3. Thus, for example, in two search processes, a virtual camera position with greater consistency between the reference shape information and the actual shape information can be searched.

[0252] Here, the segmentation surface generation unit 6122 divides each of the first T3 virtual segmentation surfaces, which contain a virtual segmentation surface (the first reference virtual segmentation surface) that is the same as the first reference virtual camera position Ps1 with a high consistency virtual camera position P11m and whose distances from the reference point Po of the 3dm dimensional model of the object part are different, using the same rule. Thus, the segmentation surface generation unit 6122 generates M3 virtual segmentation surfaces (i.e., the first M3 virtual segmentation surfaces) for each of the first T3 virtual segmentation surfaces, thereby generating the first M3×T3 virtual segmentation surfaces. The first T3 virtual segmentation surfaces are a plurality of virtual segmentation surfaces that intersect the reference point Po of the 3dm dimensional model of the object part near the first reference virtual camera position Ps1 and the straight line Ln12 passing through the reference point Po and the first reference virtual camera position Ps1.

[0253] Here, the first reference virtual segmentation surface is set according to a predetermined rule. For example, if each of the plurality of virtual surfaces St1 constituting the surface assembly As1 is a triangular surface, then one of the four triangular virtual segmentation surfaces generated by dividing the plurality of virtual surfaces St1 by lines connecting the midpoints of each side into highly consistent virtual surfaces St1m can be set as the first reference virtual segmentation surface Ss1.

[0254] Figure 30 and Figure 31 These are diagrams illustrating a second specific example of generating the first M3×T3 virtual segmentation surfaces St3 by the segmentation surface generation unit 6122. Figure 30 and Figure 31 In, such as Figure 17 As shown, this is a specific example of the case where each of the plurality of virtual faces St1 constituting the face assembly As1 is a triangular face. Figure 30 and Figure 31 In the diagram, the highly consistent virtual camera position P11m, which serves as the first reference virtual camera position Ps1, is indicated by a circular marker on a white background.

[0255] Here, among the M4 virtual segmentation surfaces (M4 being a natural number greater than 2) generated by segmenting the high-consistency virtual surface St1m, the virtual segmentation surface containing the high-consistency virtual camera position P11m, which serves as the first reference virtual camera position Ps1, becomes the first reference virtual segmentation surface Ss1. Since each of the M4 virtual segmentation surfaces is a surface generated by segmenting the virtual surface St1, it is designated as the redefined virtual segmentation surface St2. For example... Figure 30 As shown, by dividing the high-consistency virtual surface St1m into three line segments connecting the midpoints of each side of the high-consistency virtual surface St1m, four virtual segmentation surfaces St2, consisting of M4 triangles, are generated. Furthermore, among these four triangular virtual segmentation surfaces St2, the virtual segmentation surface St2 containing the high-consistency virtual camera position P11m, which serves as the first reference virtual camera position Ps1, becomes the first reference virtual segmentation surface Ss1. Here, the midpoint of each side can, for example, be a point slightly offset from the midpoint of each side.

[0256] Here, by dividing each of the T3 virtual surfaces St1 out of the aforementioned T2 virtual surfaces St1 according to the same rules as the high-consistency virtual surface St1m, each of the T3 virtual surfaces St1 is divided into M4 virtual segmentation surfaces St2. From another perspective, when viewed from the reference point Po of the 3dm of the object part's 3D model, each of the T3 virtual surfaces St1 is divided by lines with shapes that have a similar relationship of expanding and shrinking in a radial direction centered on the reference point Po, resulting in M4×T3 virtual segmentation surfaces St2. Moreover, the first of the T3 virtual segmentation surfaces St2, which includes the first reference virtual segmentation surface Ss1, is a part of the M4×T3 virtual segmentation surfaces St2, and has a shape that has a similar relationship of expanding and shrinking in a radial direction centered on the reference point Po of the 3dm of the object part's 3D model.

[0257] For example, each of the T3 virtual surfaces St1 in the aforementioned T2 virtual surfaces St1 is... Figure 30 Similarly, the highly consistent virtual surface St1m shown can be divided by lines connecting the midpoints of each side. Thus, each of the T3 virtual surfaces St1 can be divided into four triangular virtual dividing surfaces St2. From another perspective, when viewed from the reference point Po of the 3D model 3dm of the object part, 4×T3 virtual dividing surfaces St2 are generated by dividing each of the T3 virtual surfaces St1 with lines having similar shapes that expand and contract in a radial direction centered on the reference point Po. Furthermore, the first of the T3 virtual dividing surfaces St2, including the first reference virtual dividing surface Ss1, is a part of the 4×T3 virtual dividing surfaces St2, having a shape that expands and contracts radially in a radial direction centered on the reference point Po of the 3D model 3dm of the object part.

[0258] Here, the same rule related to the division of the first T3 virtual segmentation surfaces, as described above, can be a rule that divides the segmentation object surface into multiple surfaces by connecting the center point of the segmentation object (i.e., the segmentation object surface) to all vertices of the segmentation object surface with multiple line segments respectively. For example, if each of the first T3 virtual segmentation surfaces St2 is a triangle, then each of the first T3 virtual segmentation surfaces St2 can be divided into M3 virtual segmentation surfaces, i.e., 3 virtual segmentation surfaces, by connecting 3 vertices to the 3 line segments of the 1st virtual camera position P11 respectively. Thus, each of the first T3 virtual segmentation surfaces St2 can be divided into 3 triangular virtual segmentation surfaces. As a result, the first M3×T3 virtual segmentation surface is generated as an M3×T3 virtual segmentation surface.

[0259] Figure 31In this paper, the segmentation of the first reference virtual segmentation surface Ss1 in the first T3 virtual segmentation surface St2 is represented. The first T3 virtual segmentation surface St2 has a shape with a similar relationship of mutual expansion and contraction in a radial direction, centered on the reference point Po of the 3dm reference point of the 3D model of the object part. Figure 31 As shown, the segmentation surface generation unit 6122, for example, divides the first reference virtual segmentation surface Ss1 into three virtual segmentation surfaces St3, which are the first M3 virtual segmentation surfaces, by connecting the three vertices of the first reference virtual segmentation surface Ss1 and the three line segments that are the high-consistency virtual camera position P11m, which is the first reference virtual camera position Ps1.

[0260] Here, when viewed from the reference point Po of the 3dm 3D model of the object part, the first M3×T3 virtual dividing surfaces St3 are generated by dividing each of the first T3 virtual dividing surfaces St2 with lines having similar shapes that expand and shrink towards each other in a radial direction centered on the reference point Po. Thus, among the first T3 virtual dividing surfaces St2, there exists a state where each of the M3 virtual dividing surfaces St3 has a shape that expands and shrinks towards each other in a radial direction centered on the reference point Po of the 3dm 3D model of the object part.

[0261] like Figure 17 As shown, if each of the plurality of virtual faces St1 constituting the face assembly As1 is a triangular face, then M3 can be 3. In this case, each of the first T3 virtual dividing faces St2 and... Figure 31 Similarly, the first reference virtual segmentation surface Ss1 shown can be divided into three virtual segmentation surfaces St3 by connecting the three vertices to the position P11 of the first virtual camera. Thus, each of the T3 virtual segmentation surfaces St2 can be divided into three triangular virtual segmentation surfaces St3. As a result, an M3×T3 virtual segmentation surface St3 is generated as the first of the M3×T3 virtual segmentation surfaces.

[0262] <<Process 1B>>

[0263] The first B process is as follows: The second shape information generation unit 6123, based on the three-dimensional design information related to the object part stored in the storage unit 45b, assumes that the 3D model 3dm of the object part is photographed from each of the first M3×T3 third virtual camera positions, and generates reference shape information for each of the first M3×T3 third virtual camera positions. The first M3×T3 third virtual camera positions are the M3×T3 virtual camera positions set by virtually setting one virtual camera position for each of the first M3×T3 virtual segmentation surfaces St3 generated in the first A process. Here, the reference shape information is also information generated based on the three-dimensional design information related to the object part, and is information related to the two-dimensional shape of the 3D model 3dm of the object part in the virtual image obtained by photographing the 3D model 3dm of the object part from the virtual camera position. The virtual image can be generated, for example, by projecting the 3D model 3dm onto a virtual plane using rendering or other processes.

[0264] In each of the first M3×T3 virtual segmentation planes St3, for example, the position P13 of the third virtual camera is virtually set at a predetermined position on the virtual segmentation plane St3. The predetermined position is, for example, applied to the position of the center of the virtual segmentation plane St3. The center of the virtual segmentation plane St3 is, for example, applied to the position of the centroid of the virtual segmentation plane St3. For example, Figure 29 or Figure 31 As shown, the position of the third virtual camera is set at P13 on each virtual segmentation surface St3. If each of the first M3×T3 virtual segmentation surfaces St3 is a triangle, then the center of the virtual segmentation surface St3 can be, for example, the centroid of the triangle or the incenter. Figure 29 or Figure 31 In the diagram, the position of the third virtual camera, P13, is indicated by a black circle.

[0265] In the second shape information generation unit 6123, for example for Figure 29 or Figure 31 For each of the plurality of third virtual camera positions P13 shown, reference shape information related to the two-dimensional shape of the 3D model 3dm of the object part in a virtual image obtainable from the third virtual camera position P13 is generated. At this time, the shooting direction of the virtual camera shooting the 3D model 3dm of the object part from the third virtual camera position P13 is set to the direction from the third virtual camera position P13 toward the reference point Po of the 3D model 3dm of the object part. The roll angle γ of the pose of the virtual camera shooting the 3D model 3dm of the object part from the third virtual camera position P13 is defined, for example, as zero (0) degrees. Here, for each third virtual camera position P13, for example, as shown... Figure 10 The reference image Iv1 is schematically represented as a reference shape information.

[0266] <<Process 1C>>

[0267] The first C process is the second calculation unit 6124, which calculates a value representing the consistency between the information related to the two-dimensional shape of the object captured in the actual image of the captured object part (actual shape information) and the reference shape information for each of the first M3×T3 third virtual camera positions P13.

[0268] In the first C process, the second calculation unit 6124 calculates, for example, a reference image generated by the second shape information generation unit 6123 in the first B process as reference shape information, and a comparison object region Re2 set by the comparison object region setting unit 6121 as an edge image as actual shape information. Figure 25 The numerical value of the consistency between the two-dimensional shapes of the outlines.

[0269] In the first C process, when the second calculation unit 6124 calculates the value representing the consistency between the edge image, which is actual shape information, and the reference image, which is reference shape information, for each third virtual camera position P13, it performs the following steps in the following order: [Process 2a1] processing to detect the offset of the orientation of the two-dimensional shape of the contour corresponding to the offset of the roll angle γ between the edge image and the reference image; [Process 2b1] processing to rotate the edge image corresponding to the offset of the roll angle γ; and [Process 2c1] processing to detect the position of the region with the highest consistency with the reference image, with the rotated edge image as the object.

[0270] In the above-described process 2a1, similar to process 2a, the second calculation unit 6124, for example, uses the RIPOC method to detect the offset of the orientation of the two-dimensional shape of the contour between the edge image, which serves as actual shape information, and the reference image, which serves as reference shape information. This offset of the orientation of the two-dimensional shape of the contour is the offset of the rotation direction of the contour on the image. Here, the second calculation unit 6124, for example, calculates the reference object region Re2 (which is set by the reference object region setting unit 6121 as the edge image, serving as actual shape information), for example, using the RIPOC method. Figure 25 The offset γ21 in the rotation direction that maximizes the consistency of the contour between the reference image (which serves as reference shape information) and the edge image (which serves as actual shape information). This offset γ21 is equivalent to the roll angle γ that maximizes the consistency between the reference image (which serves as reference shape information) and the edge image (which serves as actual shape information).

[0271] In process 2b1 described above, similar to process 2b described above, the second calculation unit 6124 rotates the edge image, which serves as actual shape information, by correcting the offset γ21 detected in process 2a1. This generates a rotated edge image in which the two-dimensional shape of the contour in the edge image, which serves as actual shape information, and the two-dimensional shape of the contour in the reference image, which serves as reference shape information, are aligned in orientation. Here, for example, the second calculation unit 6124 rotates the edge image of the reference target region Re2 by correcting the offset γ21 detected in process 2a1. This generates a rotated edge image in which the two-dimensional shape of the contour in the reference target region Re2, which constitutes actual shape information, and the two-dimensional shape of the contour in the reference image, which serves as reference shape information, are aligned in orientation.

[0272] In process 2c1 described above, similar to process 2c, the second calculation unit 6124, for example, uses the rotated edge image related to the actual shape information generated in process 2b1 as the object and performs template matching using a reference image as reference shape information. Here, the second calculation unit 6124, for example, scans the reference image within the rotated edge image and detects the position of the region within the rotated edge image where the consistency (similarity) with the reference image is greatest. Thus, in the edge image, which serves as actual shape information, the position (matching candidate position) of the region with the greatest consistency (similarity) with the 3dm outline of the object part in the reference image can be detected. Furthermore, when performing the template matching, a value representing the consistency (similarity) when the consistency (similarity) with the reference image is greatest is calculated and used as a value related to the third virtual camera position P13, representing the consistency between the edge image, which serves as actual shape information, and the reference image, which serves as reference shape information. In other words, for each third virtual camera position P13, the consistency degree when the consistency of the reference image with respect to a portion of the rotated edge image is maximized is defined as the consistency degree between the edge image (which serves as actual shape information) and the reference image (which serves as reference shape information). Here, the numerical value representing the consistency degree can also be, for example, the matching score representing the consistency degree.

[0273] <<First-D Processing>>

[0274] The first D processing is as follows: The second detection unit 6125 detects the virtual camera position with the highest consistency between the information related to the two-dimensional shape of the object captured in the actual image of the captured object part (actual shape information) and the reference shape information among the first reference virtual camera position Ps1 and the first M3×T3 third virtual camera positions P13, namely the second reference virtual camera position.

[0275] Here, for example, the matching score calculated for the first reference virtual camera position Ps1 is compared with the matching score calculated by the second calculation unit 6124 for each of the first M3×T3 third virtual camera positions P13 in the first C process. Moreover, for example, the virtual camera position with the highest matching score among the first reference virtual camera position Ps1 and the first M3×T3 third virtual camera positions P13, calculated between the information related to the two-dimensional shape of the object captured in the actual image of the captured object part (actual shape information) and the reference shape information, is used as the second reference virtual camera position.

[0276] <<Process nA>>

[0277] The nth process is the process by which the segmentation surface generation unit 6122 generates the nth M3×T3 virtual segmentation surface. In this process, the segmentation surface generation unit 6122 divides each of the T3 virtual segmentation surfaces (also called the nth T3 virtual segmentation surface) generated by dividing the (n-1)th T3 virtual segmentation surfaces according to the same rule. Thus, the segmentation surface generation unit 6122 generates the nth M3×T3 virtual segmentation surface by generating M3 virtual segmentation surfaces (also called the nth M3 virtual segmentation surface) for each of the nth T3 virtual segmentation surfaces. Here, the nth T3 virtual segmentation surface includes a virtual segmentation surface (also called the nth reference virtual segmentation surface) containing the virtual camera position detected by the second detection unit 6125, i.e., the nth reference virtual camera position, and the distances from the reference point Po of the 3dm of the 3D model of the object part are all different. Furthermore, the nth T3 virtual segmentation surface is a plurality of virtual segmentation surfaces that intersect the reference point Po on the 3dm reference point Po of the 3D model of the object part with the straight line passing through the reference point Po and the nth reference virtual camera position.

[0278] Here, the same rule related to the segmentation of the nth T3 virtual segmentation surface can be observed from the reference point Po of the 3dm 3D model of the object part, and the rule for segmenting each of the nth T3 virtual segmentation surfaces in the same form. In other words, when the same rule related to the segmentation of the nth T3 virtual segmentation surface can be observed from the reference point Po of the 3dm 3D model of the object part, the rule for segmenting each of the nth T3 virtual segmentation surfaces with lines having similar shapes that expand and shrink towards each other in a radial direction centered on the reference point Po is a rule for segmenting the nth T3 virtual segmentation surface in a way that generates T3 segmentation virtual surfaces of the same shape for each of the M3 virtual segmentation surfaces between the nth T3 virtual segmentation surfaces. The same rule related to the segmentation of the nth T3 virtual segmentation surface can be, for example, a rule that divides the segmentation object surface into multiple surfaces by connecting the center point of the segmentation object (i.e., the segmentation object surface) with all the vertices of the segmentation object surface. Therefore, the segmentation of the segmentation object surface can be easily performed.

[0279] Figures 36 to 46 This is an image schematically representing a specific example of the nth T3 virtual segmentation surface St3, that is, an example of the second T3 virtual segmentation surface St3. Figure 36 In the diagram, the direction (also known as the third direction) Dr13 from the reference point Po of the 3dm 3D model of the object part toward the second reference virtual camera position Ps2 is represented by an arrow drawn with a thin line. The straight line Ln13 passing through the reference point Po and the second reference virtual camera position Ps2 is drawn with a thin double-dotted line. Figure 37 In the middle, the lines passing through the reference point Po and through the outer edges of each of the aforementioned second T3 virtual dividing surfaces St3 are depicted with thin dashed lines. Figure 36 For simplicity, only three virtual dividing surfaces St3 of the second T3 are depicted; illustrations of other virtual dividing surfaces St3 are omitted. Figure 38 As shown, each of the second T3 virtual segmentation surfaces St3 intersects the portion of the reference point Po in the straight line Ln13 on the side near the second reference virtual camera position Ps2. Here, the second T3 virtual segmentation surfaces St3 have a shape that expands and contracts relative to each other in a radial direction, centered on the reference point Po of the 3dm 3dm model of the searched object part.

[0280] In this nth A process, the generation method of the nth M3×T3 virtual segmentation surface differs depending on the nth virtual camera position (nth reference virtual camera position) detected by the second detection unit 6125. More specifically, the generation method of the nth M3×T3 virtual segmentation surface differs when the nth reference virtual camera position is any of the (n-1)th M3×T3 third virtual camera positions (also known as the first case of the nth case) and when the nth reference virtual camera position is the (n-1)th reference virtual camera position (also known as the second case of the nth case).

[0281] <<<The first case of the nth one>>>

[0282] When the second detection unit 6125 detects any one of the above-mentioned (n-1)th M3×T3 third virtual camera positions as the nth reference virtual camera position, the consistency between the two-dimensional shape-related information (actual shape information) of the object captured in the actual image of the object part and the reference shape information of any of the above-mentioned (n-1)th M3×T3 third virtual camera positions is greater than that of the reference shape information.

[0283] For example, compared to the matching score representing the consistency between the actual shape information and the reference shape information related to the (n-1)th reference virtual camera position, the matching score representing the consistency between the actual shape information and the reference shape information related to either the (n-1)th M3×T3 third virtual camera position is greater.

[0284] Therefore, the segmentation surface generation unit 6122 uses the third virtual camera position with the highest consistency between the two-dimensional shape-related information (actual shape information) of the object captured in the actual image of the captured object part and the reference shape information among the (n-1)th M3×T3 third virtual camera positions as the segmentation reference for the next virtual surface, i.e., the nth reference virtual camera position, and generates the nth M3×T3 virtual segmentation surface. Thus, for example, in two search processes, a virtual camera position with a higher consistency between the reference shape information and the actual shape information can be searched.

[0285] Here, the segmentation surface generation unit 6122 segments each of the aforementioned (n-1)th M3×T3 virtual segmentation surfaces, including the virtual segmentation surface containing the nth reference virtual camera position (i.e., the nth reference virtual segmentation surface) and the nth T3 virtual segmentation surfaces whose distances from the reference point Po (3dm) of the object part's 3D model are all different, using the same rule. Thus, the segmentation surface generation unit 6122 generates the nth M3×T3 virtual segmentation surface by generating M3 virtual segmentation surfaces (i.e., the nth M3 virtual segmentation surfaces) for each of the nth T3 virtual segmentation surfaces. The nth T3 virtual segmentation surface is a plurality of virtual segmentation surfaces that intersect the reference point Po (3dm) of the object part's 3D model near the nth reference virtual camera position and the straight line passing through the reference point Po and the nth reference virtual camera position.

[0286] Here, the same rule related to the segmentation of the nth T3 virtual segmentation surface is as described above. For example, it can be a rule that divides the segmentation object surface into multiple surfaces by connecting the center point of the segmentation object (i.e., the segmentation object surface) to all vertices of the segmentation object surface with multiple line segments respectively. For example, if each of the nth T3 virtual segmentation surfaces is a triangle, then each of the nth T3 virtual segmentation surfaces can be divided into the nth M3 virtual segmentation surfaces, i.e., 3 virtual segmentation surfaces, by connecting 3 vertices to the position of the (n-1)th 3rd virtual camera with 3 line segments respectively. Thus, each of the nth T3 virtual segmentation surfaces can be divided into 3 triangular virtual segmentation surfaces. As a result, the nth M3×T3 virtual segmentation surface is generated as an M3×T3 virtual segmentation surface.

[0287] Here, for example, assuming that detection Figure 36 The lowermost of the three third virtual camera positions P13 is used as the second reference virtual camera position Ps2. In this case, as... Figure 39 As shown, the virtual segmentation surface St3 containing the second reference virtual camera position Ps2 among the three virtual segmentation surfaces St3 generated by segmenting virtual segmentation surface St2 becomes the second reference virtual segmentation surface Ss2.

[0288] Figure 38 This is a diagram illustrating the first specific example of the second M3×T3 virtual segmentation surface St3a generated by the segmentation surface generation unit 6122 as an example of the nth M3×T3 virtual segmentation surface. Figures 40 to 42In this section, the division of the second reference virtual dividing surface Ss2 in the second T3 virtual dividing surface St3 is represented. The second T3 virtual dividing surface St3 is the T3 virtual dividing surface St3 in the first M3×T3 virtual dividing surface St3 generated by the dividing surface generation unit 6122. The second T3 virtual dividing surface St3 has a shape that expands and shrinks towards each other in a radial direction, centered on the reference point Po of the 3dm reference point of the 3D model of the object part. Figure 38 In, such as Figure 43 As shown, this is a specific example of the case where each of the plurality of virtual faces St1 constituting the face assembly As1 is a triangular face. Figure 44 In the diagram, the position of the second reference virtual camera, Ps2, is indicated by a circular marker on a white background. (For example...) Figure 45 As shown, the segmentation surface generation unit 6122, for example, divides the second reference virtual segmentation surface Ss2 into three virtual segmentation surfaces St3a, which are the second M3 virtual segmentation surfaces, by connecting the three vertices of the second reference virtual segmentation surface Ss2 and the three line segments of the second reference virtual camera position Ps2.

[0289] Here, when viewed from the reference point Po of the 3dm 3D model of the object part, the second M3×T3 virtual dividing surfaces St3a are generated by dividing each of the second T3 virtual dividing surfaces St3 with lines having a similar shape that expands and shrinks towards each other in a radial direction centered on the reference point Po. Thus, among the second T3 virtual dividing surfaces St3, there exists a state where each of the M3 virtual dividing surfaces St3a has a shape that expands and shrinks towards each other in a radial direction centered on the reference point Po of the 3dm 3D model of the object part.

[0290] like Figure 46 As shown, if each of the complex number of virtual faces St1 constituting the face assembly As1 is a triangular face, then each of the second T3 virtual dividing faces St3 and... Figure 36 Similarly, the second reference virtual segmentation surface Ss2 shown can be divided into three virtual segmentation surfaces St3a by connecting the three vertices to the position P13 of the third virtual camera. Thus, each of the second T3 virtual segmentation surfaces St3 can be divided into three triangular virtual segmentation surfaces St3a. As a result, the second 3×T3 virtual segmentation surfaces St3a are generated as M3×T3 virtual segmentation surfaces.

[0291] <<<The second case of the nth one>>>

[0292] The second detection unit 6125 detects the (n-1)th reference virtual camera position. When the nth reference virtual camera position is used, compared to any of the (n-1)th M3×T3 third virtual camera positions, the (n-1)th reference virtual camera position has a greater consistency between the two-dimensional shape-related information (actual shape information) of the object captured in the actual image of the object part and the reference shape information.

[0293] In other words, compared to the consistency between the actual shape information related to the (n-1)th reference virtual camera position calculated by the first calculation unit 6113 or the second calculation unit 6124 and the reference shape information, the consistency between all relevant actual shape information of the (n-1)th M3×T3 third virtual camera position calculated by the second calculation unit 6124 in the (n-1)th unit processing is smaller than the consistency between the actual shape information related to the (n-1)th reference virtual camera position calculated by the first calculation unit 6113 or the second calculation unit 6124 and the reference shape information. More specifically, for example, compared to the matching score indicating the consistency between the actual shape information related to the (n-1)th reference virtual camera position calculated by the first calculation unit 6113 or the second calculation unit 6124 and the reference shape information, the matching score indicating the consistency between all relevant actual shape information of the (n-1)th M3×T3 third virtual camera position calculated by the second calculation unit 6124 in the (n-1)th unit processing is smaller.

[0294] Therefore, the segmentation surface generation unit 6122 uses the (n-1)th reference virtual camera position as the segmentation reference for the next virtual segmentation surface, i.e., the nth reference virtual camera position, and generates the nth M3×T3 virtual segmentation surface. Thus, for example, in two search processes, a virtual camera position with greater consistency between the reference shape information and the actual shape information can be searched.

[0295] Here, the segmentation surface generation unit 6122 divides each of the nth T3 virtual segmentation surfaces (the nth reference virtual segmentation surface) containing a virtual segmentation surface (the nth reference virtual segmentation surface) that is at the same position as the (n-1)th reference virtual camera and whose distances from the reference point Po (3dm) of the 3D model of the object part are different, using the same rule. Thus, the segmentation surface generation unit 6122 generates M3 virtual segmentation surfaces (i.e., the nth M3 virtual segmentation surfaces) for each of the nth T3 virtual segmentation surfaces, thereby generating the nth M3×T3 virtual segmentation surfaces. The nth T3 virtual segmentation surface is a plurality of virtual segmentation surfaces that intersect the reference point Po (3dm) of the 3D model of the object part near the nth reference virtual camera position and a straight line passing through the reference point Po and the nth reference virtual camera position.

[0296] Here, the nth reference virtual segmentation surface is set according to a predetermined rule.

[0297] For example, when the variable n is 2, if each of the M2×T2 virtual segmentation surfaces St2 is a triangle, then the virtual segmentation surface containing the second reference virtual camera position Ps2 among the four triangular virtual segmentation surfaces generated by dividing the first reference virtual segmentation surface Ss1 of the M2×T2 virtual segmentation surfaces St2 by lines connecting the midpoints of each side can be set as the second reference virtual segmentation surface. Here, the midpoint of each side can, for example, be a point slightly offset from the midpoint of each side.

[0298] Furthermore, for example, when the variable n is 3 or more, if each of the (n-2)th M3×T3 virtual segmentation surfaces is a triangle, then among the four triangular virtual segmentation surfaces generated by dividing the (n-1)th reference virtual segmentation surface in the (n-2)th M3×T3 virtual segmentation surface by three line segments connecting the midpoints of each side, the virtual segmentation surface containing the position of the nth reference virtual camera can be set as the nth reference virtual segmentation surface. Here, the midpoint of each side can, for example, be a point slightly offset from the midpoint of each side.

[0299] Figure 44 and Figure 43 These are diagrams illustrating the second specific example of generating the second M3×T3 virtual segmentation surface St3a, which is an example of generating the nth M3×T3 virtual segmentation surface, through the segmentation surface generation unit 6122. Figure 43 and Figure 37 In the above example, we have a specific case where each of the M2×T2 virtual dividing surfaces St2 is a triangular face. Figure 38 and Figure 39 In the diagram, the second reference virtual camera position Ps2 is indicated by a circular marker on a white background.

[0300] Here, among the M4 virtual segmentation surfaces generated by dividing the first reference virtual segmentation surface Ss1, the virtual segmentation surface containing the position of the second reference virtual camera Ps2 becomes the second reference virtual segmentation surface Ss2. Since each of the M4 virtual segmentation surfaces is a surface generated by dividing the virtual segmentation surface St2, it is used as the redefined virtual segmentation surface St3. For example... Figure 40 As shown, by dividing the first reference virtual segmentation surface Ss1 into three line segments connecting the midpoints of each side of the first reference virtual segmentation surface Ss1, four virtual segmentation surfaces St3, consisting of M4 triangles, are generated. Furthermore, among these four triangular virtual segmentation surfaces St3, the virtual segmentation surface St3 containing the position Ps2 of the second reference virtual camera becomes the second reference virtual segmentation surface Ss2.

[0301] Here, by dividing each of the aforementioned T3 virtual dividing surfaces St2 according to the same rules as the first reference virtual dividing surface Ss1, each of the T3 virtual dividing surfaces St2 is divided into M4 virtual dividing surfaces St3. From another perspective, when viewed from the reference point Po of the 3dm 3D model of the object part, M4 × T3 virtual dividing surfaces St3 are generated by dividing each of the T3 virtual dividing surfaces St2 with a shape that expands and contracts in a radial direction centered on the reference point Po. Moreover, the second T3 virtual dividing surface St3, which includes the second reference virtual dividing surface Ss2, is a part of the M4 × T3 virtual dividing surfaces St3, and has a shape that expands and contracts in a radial direction centered on the reference point Po of the 3dm 3D model of the object part.

[0302] For example, each of the above T3 virtual dividing surfaces St2 and Figure 41 Similarly, the first reference virtual dividing surface Ss1 shown can be divided by three line segments connecting the midpoints of each side. Thus, each of the T3 virtual dividing surfaces St2 can be divided into four triangular virtual dividing surfaces St3. From another perspective, when viewed from the reference point Po of the 3dm 3D model of the object part, 4×T3 virtual dividing surfaces St3 are generated by dividing each of the T3 virtual dividing surfaces St2 with lines having similar shapes that expand and contract in a radial direction centered on the reference point Po. Furthermore, the second T3 virtual dividing surface St3, including the second reference virtual dividing surface Ss2, is part of the 4×T3 virtual dividing surfaces St3, having a shape that expands and contracts radially in a radial direction centered on the reference point Po of the 3dm 3D model of the object part.

[0303] Here, the same rule related to the division of the nth T3 virtual dividing facets, as described above, can be a rule that divides the dividing object facet into multiple faces by connecting the center point of the dividing object facet to all vertices of the dividing object facet with multiple line segments respectively. For example, when the variable n is 2, if each of the second T3 virtual dividing facest St3 is a triangle, then each of the second T3 virtual dividing facest St3 can be divided into M3 virtual dividing faces, i.e., 3 virtual dividing faces, by connecting 3 vertices to the center position (e.g., the centroid position) of a predetermined location. Thus, each of the second T3 virtual dividing facest St3 can be divided into 3 triangular virtual dividing faces. As a result, the second M3×T3 virtual dividing facest is generated as M3×T3 virtual dividing facest.

[0304] Figure 42In this example, the segmentation of the second reference virtual segmentation surface Ss2 in the second T3 virtual segmentation surface St3 is represented. The second T3 virtual segmentation surface St3 has a shape with a similar relationship of mutual expansion and contraction in a radial direction, centered on the reference point Po of the 3dm 3D model of the object part. Figure 41 As shown, the segmentation surface generation unit 6122, for example, divides the second reference virtual segmentation surface Ss2 into three virtual segmentation surfaces St3a, which are the second M3 virtual segmentation surfaces, by connecting the three vertices of the second reference virtual segmentation surface Ss2 and the three line segments of the second reference virtual camera position Ps2.

[0305] Here, when viewed from the reference point Po of the 3dm 3D model of the object part, the second M3×T3 virtual dividing surfaces St3a are generated by dividing each of the second T3 virtual dividing surfaces St3 with lines having a similar shape that expands and shrinks towards each other in a radial direction centered on the reference point Po. Thus, among the second T3 virtual dividing surfaces St3, there exists a state where each of the M3 virtual dividing surfaces St3a has a shape that expands and shrinks towards each other in a radial direction centered on the reference point Po of the 3dm 3D model of the object part.

[0306] like Figure 42 As shown, if each of the complex number of virtual faces St1 constituting the face assembly As1 is a triangular face, then each of the second T3 virtual dividing faces St3 and... Figure 13 Similarly, the second reference virtual segmentation surface Ss2 shown can be divided into three virtual segmentation surfaces St3a by connecting the three vertices to the position P12 of the second virtual camera. Thus, each of the second T3 virtual segmentation surfaces St3 can be divided into three triangular virtual segmentation surfaces St3a. As a result, the second M3×T3 virtual segmentation surfaces St3a are generated as M3×T3 virtual segmentation surfaces.

[0307] Furthermore, for example, when the variable n is 3 or more, if each of the nth T3 virtual dividing surfaces is a triangle, then each of the nth T3 virtual dividing surfaces can be divided into the nth M3 virtual dividing surfaces, i.e., 3 virtual dividing surfaces, by connecting 3 vertices to 3 line segments at a predetermined center position (e.g., the centroid). Thus, each of the nth T3 virtual dividing surfaces can be divided into 3 triangular virtual dividing surfaces. As a result, the nth M3×T3 virtual dividing surface is generated as an M3×T3 virtual dividing surface.

[0308] <<Process nB>>

[0309] The nth process is as follows: The second shape information generation unit 6123, based on the three-dimensional design information related to the object part stored in the storage unit 45b, assumes that the 3D model 3dm of the object part is photographed from each of the nth third virtual camera positions, and generates reference shape information for each of the nth M3×T3 third virtual camera positions. The nth M3×T3 third virtual camera positions are M3×T3 virtual camera positions set by virtually setting one virtual camera position for each of the nth M3×T3 virtual segmentation surfaces. Here, the reference shape information is also information generated based on the three-dimensional design information related to the object part, and is information related to the two-dimensional shape of the 3D model 3dm of the object part in the virtual image obtained by photographing the 3D model 3dm of the object part from the virtual camera position. The virtual image can be generated, for example, by projecting the 3D model 3dm onto a virtual plane using rendering or other processes.

[0310] In each of the nth M3×T3 virtual segmentation surfaces, for example, the position of the third virtual camera (the nth third virtual camera position) is virtually set at a predetermined position on the virtual segmentation surface. The predetermined position is, for example, applied to the center of the virtual segmentation surface. The center of the virtual segmentation surface is, for example, applied to the centroid of the virtual segmentation surface. For example, when the variable n is 2, such as... Figure 21 or Figure 45 As shown, the position of the third virtual camera is set at P13a on each virtual segmentation surface St3a. If each of the nth M3×T3 virtual segmentation surfaces is a triangle, then the center of the virtual segmentation surface can be, for example, the centroid of the triangle or the incenter. Figure 44 and Figure 37 In the diagram, the position of the third virtual camera, P13a, is indicated by a black circle.

[0311] In the second shape information generation unit 6123, for example, for each of the nth M3×T3 third virtual camera positions, reference shape information related to the two-dimensional shape of the 3D model 3dm of the object part in a virtual image obtainable from the third virtual camera position is generated, which is the 3D model 3dm of the object part. At this time, the shooting direction of the virtual camera that takes pictures of the 3D model 3dm of the object part from each third virtual camera position is set to the direction from the third virtual camera position toward the reference point Po of the 3D model 3dm of the object part. The roll angle γ of the pose of the virtual camera that takes pictures of the 3D model 3dm of the object part from the third virtual camera position is set to zero (0) degrees, for example. Here, for each third virtual camera position, for example, reference shape information is also generated, which is the 2D shape of the 3D model 3dm of the object part. Figure 43 The reference image Iv1 is schematically represented as a reference shape information.

[0312] <<Process nC>>

[0313] The nth C processing is the second calculation unit 6124, which calculates a value representing the consistency between the information related to the two-dimensional shape of the object captured in the actual image of the captured object part (actual shape information) and the reference shape information for each of the aforementioned nth M3×T3 third virtual camera positions.

[0314] In the nCth process, the second calculation unit 6124 calculates, for example, a reference image representing reference shape information obtained by the second shape information generation unit 6123 in the nBth process, and a comparison object region Re2 representing an edge image set by the comparison object region setting unit 6121 as actual shape information. Figure 46 The numerical value of the consistency between the two-dimensional shapes of the outlines.

[0315] In the nC process, when the second calculation unit 6124 calculates a value representing the consistency between the edge image, which is actual shape information, and the reference image, which is reference shape information, for each third virtual camera position, it performs the following steps in the following order: [Process 2an] processing to detect the offset of the orientation of the two-dimensional shape of the contour corresponding to the offset of the roll angle γ between the edge image and the reference image; [Process 2bn] processing to rotate the edge image corresponding to the offset of the roll angle γ; and [Process 2cn] processing to detect the position of the region with the highest consistency with the reference image, with the rotated edge image as the object.

[0316] In the above-described process 2an, similar to process 2a1, the second calculation unit 6124, for example, uses the RIPOC method to detect the offset of the orientation of the two-dimensional shape of the contour between the edge image, which serves as actual shape information, and the reference image, which serves as reference shape information. This offset of the orientation of the two-dimensional shape of the contour is the offset of the rotation direction of the contour on the image. Here, the second calculation unit 6124, for example, calculates the reference object region Re2 (which is set by the reference object region setting unit 6121 as the edge image, serving as actual shape information), using the RIPOC method. Figure 46 The offset γ2n in the rotation direction that maximizes the consistency of the contour between the reference image (which serves as reference shape information) and the edge image (which serves as actual shape information). This offset γ2n is equivalent to the roll angle γ that maximizes the consistency between the reference image (which serves as reference shape information) and the edge image (which serves as actual shape information).

[0317] In the above-described process 2bn, similar to process 2b1, the second calculation unit 6124 rotates the edge image, which serves as actual shape information, by correcting the offset γ2n detected in process 2an. This generates a rotated edge image in which the two-dimensional shape of the contour in the edge image, which serves as actual shape information, and the two-dimensional shape of the contour in the reference image, which serves as reference shape information, are aligned in orientation. Here, the second calculation unit 6124 rotates the edge image of the reference object region Re2, for example, by correcting the offset γ2n detected in process 2an. This generates a rotated edge image in which the two-dimensional shape of the contour in the reference object region Re2, which constitutes actual shape information, and the two-dimensional shape of the contour in the reference image, which serves as reference shape information, are aligned in orientation.

[0318] In the above-described process 2cn, similar to process 2c1, the second calculation unit 6124, for example, uses the rotated edge image related to the actual shape information generated in process 2bn as the object and performs template matching using a reference image as reference shape information. Here, the second calculation unit 6124, for example, scans the reference image within the rotated edge image and detects the position of the region within the rotated edge image where the consistency (similarity) with the reference image is greatest. Thus, in the edge image, which serves as actual shape information, the position (matching candidate position) of the region with the greatest consistency (similarity) with the 3dm outline of the object part in the reference image can be detected. Furthermore, when performing the template matching, a value representing the consistency (similarity) when the consistency (similarity) between a region within the rotated edge image and the reference image is greatest is calculated and used as a value related to the position of the third virtual camera, representing the consistency between the edge image, which serves as actual shape information, and the reference image, which serves as reference shape information. In other words, for each third virtual camera position, the consistency degree at which the reference image has the highest consistency with a portion of the rotated edge image is defined as the consistency degree between the edge image (which serves as actual shape information) and the reference image (which serves as reference shape information). Here, the numerical value representing the consistency degree can also be, for example, a matching score representing the consistency degree.

[0319] <<Process nD>>

[0320] The nth processing is as follows: The second detection unit 6125 detects the virtual camera position with the highest consistency between the information related to the two-dimensional shape of the object captured in the actual image of the captured object part (actual shape information) and the reference shape information among the above-mentioned nth reference virtual camera position and the above-mentioned nth M3×T3 third virtual camera positions.

[0321] Here, the matching score calculated for the nth reference virtual camera position is compared with the matching score calculated by the second calculation unit 6124 for each of the nth M3×T3 third virtual camera positions in the nth C process. Moreover, for example, the virtual camera position with the highest matching score among the nth reference virtual camera position and the nth M3×T3 third virtual camera positions is selected based on the information related to the two-dimensional shape of the object captured in the actual image of the captured object part (actual shape information) and the reference shape information.

[0322] <<The Effect of Repeated Processing per Unit>>

[0323] As described above, during repeated unit processing, multiple virtual segmentation surfaces, each containing a virtual camera position with the highest consistency between the actual shape information and the reference shape information, and with varying distances from the reference point Po (3dm from the 3D model of the object part), are segmented according to the same rule to generate multiple virtual segmentation surfaces for setting the next virtual camera position. Therefore, the virtual segmentation surfaces before and after segmentation are not unrelated surfaces, and the increase in at least one of the number and area of ​​the segmented virtual segmentation surfaces can be reduced. As a result, the computational load for identifying the pose of the object part captured in the actual image can be reduced. Thus, in the substrate processing apparatus 1, pose identification of the object part can be performed efficiently.

[0324] <<End of the nth unit processing after 1 or more times>>

[0325] For example, in response to the second detection unit 6125 continuously detecting one reference virtual camera position from the first reference virtual camera position Ps1 to the nth reference virtual camera position for a predetermined number of times (also referred to as the first predetermined number of times), the nth unit processing (which is then used as the virtual camera position with the highest consistency between the actual shape information and the reference shape information) ends the execution of more than one unit processing. The first predetermined number of times can be set to any number of times, such as two or more. In this case, even if the unit processing is repeated to a certain extent in the two search processes, if no virtual camera position with a higher consistency between the reference shape information and the actual shape information is detected, the repeated execution of the unit processing ends. As a result, the pose recognition of the object part can be performed efficiently by reducing the amount of computation.

[0326] Furthermore, for example, the search processing unit 61 can also terminate the execution of more than one nth unit processing step in response to performing the nth unit processing step more than once (also referred to as the second predetermined number of times). In other words, the search processing unit 61 can also terminate the execution of more than one nth unit processing step in response to performing the nth unit processing step more than once in response to performing the nth unit processing step more than once in response to performing the nth unit processing step more than once in response to the pre-set second predetermined number of times. The second predetermined number of times can be set to any number of times more than once. As a result, the computational workload is reduced, and the posture recognition of the object part can be performed efficiently.

[0327] Here, the search processing unit 61, for example, can obtain the latitude α, longitude β, and distance D of the virtual camera position (also called the highest consistency virtual camera position) that has the highest consistency between the actual shape information detected by the second detection unit 6125 and the reference shape information in the last nD process of the nth unit of processing (also called the highest consistency virtual camera position), and the roll angle γ corresponding to the offset γ2n calculated by the second calculation unit 6124, as the result of the search processing. Thus, the pose of the object part captured in the actual image can be identified. Here, the search processing unit 61, for example, can identify the latitude α, longitude β, and distance D of the highest consistency virtual camera position, and the roll angle γ corresponding to the offset γ2n calculated by the second calculation unit 6124 for the highest consistency virtual camera position, as the result of the search processing, as information related to the pose of the object part (real-world information). In this case, for example, the information related to the posture of the object part (normal information) used by the anomaly detection unit 63, which is based on the three-dimensional design information of the object part when its state is normal, can also be information related to latitude α, longitude β, distance D, and roll angle γ.

[0328] In other words, the search processing unit 61 can, for example, identify the pose-related information (real-world information) of the object part for the anomaly detection unit 63 based on the position of the virtual camera that has the highest consistency between the actual shape information detected by the second detection unit 6125 and the reference shape information in the last nD process of the nth unit of more than one execution. Therefore, the identification of the pose-related real-world information of the object part can be performed efficiently, thus enabling efficient anomaly detection of the object part.

[0329] In addition, the search processing unit 61 may, for example, obtain, for the virtual camera position with the highest consistency, the matching candidate position (also called the final matching position) that is detected by the second calculation unit 6124 and has the highest consistency (similarity) with the reference image that is the reference shape information within the edge image which is the actual shape information, as part of the result of the search processing.

[0330] <4. Specific Examples of Substrate Processing Apparatus>

[0331] Next, refer to Figure 37 A specific example of the processing of the substrate processing apparatus 1 will be described. Figure 43 This is a flowchart illustrating a specific example of the general processing flow of the substrate processing apparatus 1. Figure 46 It means Figure 37 The flowchart shows a specific example of the image processing flow for steps S3 and S10. Figure 36 It means Figure 47 The flowchart shows a specific example of the search processing flow for steps S4 and S11. ​ It means ​ The flowchart is a specific example of the processing flow of step Sb1's first search. ​ It means ​ The flowchart below shows a specific example of the processing flow for the second search of step Sb2. ​ This is a diagram illustrating an example of an object part moving to its original position. ​ and ​ This diagram is used to illustrate the anomaly detection of fixture 9. ​ This diagram is used to illustrate the abnormality detection of nozzle 33 and protective component 23.

[0332] The operator pre-operates the instruction unit 47, instructing the execution of the procedures within the procedure information 53. The motion control unit 51 controls the operation of each unit according to the instructed procedures, performing processing on each substrate W. At this time, [the process continues]. ​ The processing of steps S1 to S18.

[0333] <<Step S1>>

[0334] In step S1, the object part is moved to the origin position. Here, the motion control unit 51 controls the operation of the fixture drive mechanism 17, the protective part moving mechanism 25, and the nozzle moving mechanism 35.

[0335] The motion control unit 51 activates the clamp drive mechanism 17 via a clamp motion command, causing the clamp 9 to move to the origin position. Here, with the substrate W not mounted on the lower surface support 11, the clamp 9 rotates around the rotation center PL2 according to the clamp motion command, and the peripheral support 13 moves to the origin position on the side of the rotation center PL1 of the rotary chuck 3. At this time, the output signal of the origin sensor Z1 is activated. The motion control unit 51 identifies that the clamp 9 has moved to the origin position based on the output signal of the origin sensor Z1. ​In the diagram, the state of the clamp 9 at the origin is represented by a solid line, and the position of the outer edge of the substrate W when it is placed on the lower surface support 11 is represented by a thin double-dotted line. Here, the peripheral support 13 of each clamp 9 is moved to a position where the outer edge of the substrate W is slightly closer to the circle on the rotation center PL1 side than when the substrate W is placed on the lower surface support 11.

[0336] The motion control unit 51 activates the protective member moving mechanism 25 via a protective member motion command, causing the protective member 23 to move to the origin position. Here, the protective member 23 moves to the descending origin position. At this time, the output of the origin sensor Z2 is activated. The motion control unit 51 identifies that the protective member 23 has moved to the origin position based on the output signal of the origin sensor Z2. ​ In the diagram, the state of the protective component 23 moving to the original position is represented by a solid line, and the state of the protective component 23 being in the processing position is represented by a thin double-dotted line.

[0337] The motion control unit 51 activates the nozzle moving mechanism 35 via a nozzle motion command, causing the nozzle 33 to move to the origin position. Here, the nozzle 33 rotates around the rotation center PL3, and its front end 33c moves to the origin position, offset to the side from the protective member 23. At this time, the output of the origin sensor Z3 is activated. The motion control unit 51 identifies that the nozzle 33 has moved to the origin position based on the output signal of the origin sensor Z3. ​ In the diagram, the state where the front end 33c of the nozzle 33 is at the origin position is represented by a solid line, and the state where the front end 33c of the nozzle 33 is at the ejection position is represented by a thin double-dotted line.

[0338] <<Step S2>>

[0339] In step S2, a photograph is taken using the camera CM. Here, the motion control unit 51 is triggered by the movement of each of the nozzle 33, clamp 9, and protective member 23, which are multiple object parts, to their origin positions, and then takes a photograph using the camera CM. Specifically, the camera CM takes a photograph of the nozzle 33, clamp 9, and protective member 23, which are multiple object parts.

[0340] <<Step S3>>

[0341] In step S3, the image processing unit 59 performs image processing on the actual image acquired by the camera CM in step S2. In this step S3, the following steps are performed sequentially: ​ The processing of steps Sa1 to Sa4.

[0342] In step Sa1, the image processing unit 59 acquires the actual image obtained by the camera CM in step S2. This step Sa1 is equivalent to the step in this invention where the processing unit 45a acquires the actual image of the object part captured by the camera CM (also known as the actual image acquisition step).

[0343] In step Sa2, the processing target area extraction unit 591 of the image processing unit 59 sets the processing target area for the actual image. Here, the processing target area is set for each object part.

[0344] In step Sa3, the processing target region extraction unit 591 of the image processing unit 59 extracts the part related to the processing target region from the actual image as an image (actual image of the processing target).

[0345] In step Sa4, the contour extraction unit 592 of the image processing unit 59 extracts the contours of all parts captured in the actual image of the processing object. This obtains an edge image containing information related to the two-dimensional shape of the object captured in the actual image (actual shape information). More specifically, it obtains an edge image containing information related to the two-dimensional shape of the object including the nozzle 33. It also obtains an edge image containing information related to the two-dimensional shape of the object including the clamp 9. Finally, it obtains an edge image containing information related to the two-dimensional shape of the object including the protective member 23.

[0346] <<Step S4>>

[0347] In step S4, the search processing unit 61 performs search processing on each target part. More specifically, it performs search processing on each of the nozzle 33, the clamp 9, and the protective member 23.

[0348] Here, the search processing unit 61 performs a search process based on a plurality of reference shape information and actual shape information to find the virtual camera position among a plurality of virtual camera positions where the consistency between the reference shape information and the actual shape information is the greatest. Here, the plurality of reference shape information are information generated based on the three-dimensional design information related to the object part stored in the storage unit 45b, and are information related to the two-dimensional shape of the 3D model 3dm of the object part among a plurality of virtual images obtained from the multiple virtual camera positions of the 3D model 3dm of the object part. One reference shape information is information generated based on the three-dimensional design information related to one object part, and is information related to the two-dimensional shape of the 3D model 3dm of the object part in a virtual image obtained from the single virtual camera position. This step S4 is equivalent to the search step performed by the calculation unit 45a in this invention.

[0349] In step S4, each object part is processed sequentially.​ The processing in step Sb1 and the processing in step Sb2. In step Sb1, the first search processing unit 611 performs one search process. More specifically, in step Sb1, the processing in step Sb2 is... ​ The processing in steps Sb11 to Sb14. In step Sb2, the second search processing unit 612 performs two search processes. More specifically, in step Sb2, the processing is performed... ​ Processing steps Sb21 to Sb25 ​ The processing of steps Sb31 to Sb34, and ​ The processing of steps Sb41 to Sb48. Here, ​ The processing steps Sb31 to Sb34 correspond to the first unit processing described above. ​ The processing steps Sb42 to Sb45 correspond to the nth unit processing described above.

[0350] <<<Step Sb1>>>

[0351] <<<<Step Sb11>>>>>

[0352] In step Sb11, the density determination unit 6111 determines regions with low density (low-density regions) of the part's contour from the edge image, which serves as actual shape information, obtained in step Sa4. The determination result of the density determination unit 6111 is used in the processing of the first calculation unit 6113 and the second calculation unit 6124. For example, in the processing of the first calculation unit 6113 and the second calculation unit 6124, the low-density regions in the edge image, which serve as actual shape information, are excluded from the calculations used to determine the consistency value, thereby reducing the computational load required for the search process.

[0353] <<<<Step Sb12>>>>

[0354] In step Sb12, the first shape information acquisition unit 6112, based on the three-dimensional design information related to the object part stored in the storage unit 45b, assumes that the 3D model 3dm of the object part is photographed from each of the plurality of virtual camera positions (first virtual camera positions) P11, and acquires reference shape information generated for each of the plurality of first virtual camera positions P11. The plurality of first virtual camera positions P11 are multiple virtual camera positions set by virtually setting one virtual camera position for each of the plurality of virtual surfaces St1 (surface assembly) As1 when a plurality of virtual surfaces St1 are virtually set to include a virtual sphere surrounding the 3D model 3dm of the object part centered at a reference point Po of the 3D model 3dm of the object part. The reference shape information is information generated based on the three-dimensional design information related to the object part. It is information related to the two-dimensional shape of the 3D model 3dm of the object part in a virtual image obtained from a virtual camera position P11, where the 3D model 3dm is captured. The virtual image can be generated, for example, by projecting the 3D model 3dm onto a virtual plane using rendering or other processing. Specifically, the reference shape information is a reference image. This step Sb12 corresponds to the first shape information acquisition step in this invention.

[0355] More specifically, in step Sb12, the first shape information acquisition unit 6112, based on the three-dimensional design information related to the object part stored in the storage unit 45b, assumes that the 3D model 3dm of the object part is photographed from each of the M1×T1 first virtual camera positions P11, and obtains reference shape information generated for each of the M1×T1 first virtual camera positions P11. The M1×T1 first virtual camera positions P11 are multiple virtual camera positions set by virtually setting one virtual camera position for each of the M1 virtual faces St1 of each of the T1 face sets As1 when there are T1 face sets As1 that are virtually set at different distances from the reference point Po of the 3D model 3dm of the object part. The reference shape information is generated based on the three-dimensional design information related to the object part, and is the two-dimensional shape information related to the 3D model 3dm of the object part in the virtual image that can be obtained by taking a picture of the 3D model 3dm of the object part from the position of the first virtual camera P11.

[0356] Here, in each virtual surface St1, the position P11 of the first virtual camera is virtually set at a predetermined position on the virtual surface St1. The position of the center of the virtual surface St1 can be applied to the predetermined position. The centroid of the virtual surface can be applied to the center of the virtual surface. The surface assembly As1 can be a polyhedron composed of multiple triangular virtual surfaces. Here, as... ​As shown, the positions and poses of a plurality of virtual faces St1 are defined by setting the xyz coordinates of a right-handed system with the reference point Po of the 3D model 3dm of the object part, which is virtually generated based on the 3D design information related to the object part, as the origin. Furthermore, the positions of the plurality of first virtual camera positions P11 are defined by the angle (latitude) α in the rotation direction centered on the x-axis, the angle (longitude) β in the rotation direction centered on the z-axis, and the distance D from the origin.

[0357] In step Sb12, the first shape information acquisition unit 6112 may, for example, set a plurality of first virtual camera positions P11 based on the three-dimensional design information related to the object part stored in the storage unit 45b, and generate reference shape information for each of the plurality of first virtual camera positions P11, thereby obtaining the reference shape information generated for each of the plurality of first virtual camera positions P11.

[0358] <<<<Step Sb13>>>>

[0359] In step Sb13, the first calculation unit 6113 calculates, for each of the plurality of first virtual camera positions P11, a value representing the consistency between an edge image (actual shape information) obtained in step Sa4, which is related to the two-dimensional shape of an object captured in the actual image, and a reference image obtained in step Sb12, which is related to the reference shape information. This step Sb13 is equivalent to the first calculation step in this invention.

[0360] More specifically, in step Sb13, the first calculation unit 6113 calculates, for each of the above M1×T1 first virtual camera positions, a value representing the consistency between the edge image (actual shape information) obtained in step Sa4, which is related to the two-dimensional shape of the object captured in the actual image, and the reference image obtained in step Sb12, which is the reference shape information.

[0361] Here, when the first calculation unit 6113 calculates a value representing the consistency between the edge image, which is actual shape information, and the reference image, which is reference shape information, for each first virtual camera position P11, it performs the above-described processes 1a, 1b, and 1c in the following order. Specifically, in step Sb13, the first calculation unit 6113 performs the processes from steps Sb131 to Sb135. The process of step Sb132 is equivalent to the above-described process 1a, the process of step Sb133 is equivalent to the above-described process 1b, and the process of step Sb134 is equivalent to the above-described process 1c.

[0362] In step Sb131, the first calculation unit 6113 designates one first virtual camera position P11 and the reference image related to that first virtual camera position P11 as reference shape information among the reference images obtained in step Sb12 for each of the plurality of first virtual camera positions P11, as the processing objects of steps Sb132 to Sb134.

[0363] In step Sb132, the first calculation unit 6113, for example, uses the RIPOC method to detect the offset of the orientation of the two-dimensional shape of the contour between the edge image, which serves as actual shape information, and the reference image, which serves as reference shape information. Here, for example... ​ As shown, the first calculation unit 6113 divides the edge image Ir3, which serves as actual shape information, input from the image processing unit 59, into a plurality of comparison object regions Re1. For each comparison object region Re1, the RIPOC method is used to calculate the offset γ1 in the rotation direction where the contour consistency with the reference image is likely to be the greatest. This offset γ1 corresponds to the roll angle γ, where the consistency between the reference image (serving as reference shape information) and the edge image (serving as actual shape information) is likely to be greater. Here, the comparison object region Re1 among the plurality of comparison object regions Re1 that is likely to have the greatest contour consistency with the reference image, and the offset γ1 in the rotation direction where the contour consistency between this one comparison object region Re1 and the reference image is likely to be the greatest, are detected.

[0364] In step Sb133, the first calculation unit 6113 rotates the edge image, which serves as actual shape information, by correcting the offset γ1 detected in step Sb132. This generates a rotated edge image in which the orientation of the two-dimensional shape of the contour in the edge image, which serves as actual shape information, is consistent with the orientation of the two-dimensional shape of the contour in the reference image, which serves as reference shape information. Here, the first calculation unit 6113 rotates the edge image of the comparison object region Re1 detected in step Sb132, by correcting the offset γ1 detected in step Sb132. This generates a rotated edge image in which the orientation of the two-dimensional shape of the contour in the comparison object region Re1, which constitutes actual shape information, is consistent with the orientation of the two-dimensional shape of the contour in the reference image, which serves as reference shape information.

[0365] In step Sb134, the first calculation unit 6113, for example, uses the rotated edge image related to the actual shape information generated in step Sb133 as the object and performs template matching using a reference image as reference shape information. Here, the first calculation unit 6113 scans the reference image within the rotated edge image and detects the position of the region within the rotated edge image where the consistency (similarity) with the reference image is greatest. Thus, in the edge image, which is the actual shape information, the position (matching candidate position) of the region with the greatest consistency with the 3dm outline of the object part in the reference image is detected. Here, when performing the above template matching, a value representing the consistency (similarity) when the consistency (similarity) between a region within the rotated edge image and the reference image is greatest is calculated and used as a value related to the first virtual camera position P11, representing the consistency between the edge image, which is the actual shape information, and the reference image, which is the reference shape information. Here, the value representing the consistency can, for example, be the well-known matching score representing the consistency.

[0366] In step Sb135, the first calculation unit 6113 determines whether, among the reference shape information related to each of the plurality of first virtual camera positions P11 obtained in step Sb12, there is any first virtual camera position P11 and its related reference shape information that has not yet been specified as the processing object of steps Sb132 to Sb134. Here, if there is first virtual camera position P11 and its related reference shape information that has not yet been specified as the processing object of steps Sb132 to Sb134, the process returns to step Sb131, and the first calculation unit 6113 specifies the next first virtual camera position P11 and its related reference shape information from the plurality of reference shape information related to each of the plurality of first virtual camera positions P11 obtained in step Sb12 as the processing object of steps Sb132 to Sb134. On the other hand, if there is no first virtual camera position P11 and related reference shape information that is not yet specified as the processing object of steps Sb132 to Sb134, then the process proceeds from step Sb135 to step S14.

[0367] That is, the first calculation unit 6113 repeats the processing of steps Sb131 to Sb135 until, among the reference shape information related to each of the plurality of first virtual camera positions P11 obtained in step Sb12, the first virtual camera position P11 and the reference shape information related to that first virtual camera position P11, which are not yet specified as processing objects of steps Sb132 to Sb134, disappear. Thus, for each first virtual camera position P11, the first calculation unit 6113 calculates a value representing the consistency between the edge image, which is actual shape information, and the reference image, which is reference shape information.

[0368] <<<<Step Sb14>>>>

[0369] In step Sb14, the first detection unit 6114, based on the calculation result of step Sb13, detects the first virtual camera position P11 with the highest consistency between the edge image (actual shape information) and the reference image (reference shape information) among the plurality of first virtual camera positions P11, i.e., the high consistency virtual camera position P11m. Here, for example, the first virtual camera position P11 with the highest matching score calculated in step Sb13 between the edge image (actual shape information) related to the two-dimensional shape of the object captured in the actual image and the reference image (reference shape information) is identified as the high consistency virtual camera position P11m. This step Sb14 corresponds to the first detection step in this invention.

[0370] More specifically, in step Sb14, the first detection unit 6114, based on the calculation result of step Sb13, detects the first virtual camera position P11 with the highest consistency between the edge image (actual shape information) and the reference image (reference shape information) among the M1×T1 first virtual camera positions P11, i.e., the high consistency virtual camera position P11m. Here, for example, the first virtual camera position P11 with the highest matching score calculated in step Sb13 between the edge image (actual shape information) related to the two-dimensional shape of the object captured in the actual image and the reference image (reference shape information) is detected.

[0371] In step Sb14, when the first detection unit 6114 detects the high-consistency virtual camera position P11m, it also detects the matching candidate position detected for the high-consistency virtual camera position P11m in step Sb13, and uses it as the candidate position of the target part.

[0372] <<<Step Sb2>>>

[0373] <<<<Step Sb21>>>>

[0374] In step Sb21, the comparison object region setting unit 6121 of the second search processing unit 612 sets the region (comparison object region) in the edge image obtained in step Sa4, which serves as actual shape information, for the matching (comparison) processing in step Sb24, based on the candidate position of the object part detected in step Sb14. Here, for the edge image, the region containing the candidate position of the object part and whose size is larger than the candidate position of the object part is set as the comparison object region.

[0375] <<<<Step Sb22>>>>

[0376] In step Sb22, the segmentation surface generation unit 6122 generates a plurality of virtual surfaces (virtual segmentation surfaces) St2 by segmenting a plurality of virtual surfaces St1 that constitute the surface assembly As1 and surround the 3D model 3dm of the object part with a reference point Po centered on the 3D model 3dm of the object part, and virtual surfaces St1m that are virtually set with the high consistency virtual camera position P11m detected in step Sb14. Here, the plurality of virtual surfaces St1 are the plurality of virtual surfaces St1 used for the processing in step Sb12. If each of the plurality of virtual surfaces St1 is a triangular face, the high consistency virtual surface St1m is segmented into three virtual segmentation surfaces St2, which are the plurality of segmented virtual surfaces (virtual segmentation surfaces) St2, by connecting the three vertices of the high consistency virtual surface St1m to the three line segments of the high consistency virtual camera position P11m. This step Sb22 is equivalent to the segmentation surface generation step in this invention.

[0377] More specifically, in step Sb22, the segmentation surface generation unit 6122 segments each of the T1 virtual surfaces St1 of each of the T1 surface assemblies As1, including the high-consistency virtual surface St1m, and each of the T2 virtual surfaces St1 with different distances from the reference point Po of the 3dm ... Here, the M1 virtual faces St1 of each of the T1 face sets As1 are the M1 virtual faces St1 of each of the T1 face sets As1 used for the processing in step Sb12.

[0378] Here, T2 virtual surfaces St1 have shapes that expand and contract relative to each other in a radial direction, centered on a reference point Po of the 3dm 3D model of the object part. When viewed from the reference point Po of the 3dm 3D model of the object part, M2×T2 virtual subdivision surfaces St2 are generated by dividing each of the T2 virtual surfaces St1 with lines that have shapes that expand and contract relative to each other in a radial direction centered on the reference point Po. Thus, among the T2 virtual surfaces St1, there exists a state where, for each of the M2 virtual subdivision surfaces St2, there are T2 virtual subdivision surfaces St2 with shapes that expand and contract relative to each other in a radial direction, centered on the reference point Po of the 3dm 3D model of the object part. The T2 can be the same as or smaller than the T1. The same rule related to the segmentation of the aforementioned T2 virtual faces St1 can be, for example, a rule that divides the object to be segmented (i.e., the segmented object face) into multiple faces by connecting the center point of the object to be segmented with all the vertices of the object to be segmented. If each of the multiple virtual faces St1 is a triangular face, then each of the T2 virtual faces St1, like the high-consistency virtual face St1m, can be divided into 3 virtual segmentation faces St2 by connecting 3 vertices with 3 line segments connecting the 3 vertices to the position P11 of the first virtual camera. Thus, each of the T2 virtual faces St1 is divided into 3 triangular virtual segmentation faces St2.

[0379] <<<<Step Sb23>>>>

[0380] In step Sb23, the second shape information generation unit 6123, based on the three-dimensional design information related to the object part stored in the storage unit 45b, assumes that the 3D model 3dm of the object part is photographed from each of the plurality of virtual camera positions (second virtual camera positions) P12 set by virtually setting one virtual camera position for each of the plurality of virtual dividing surfaces St2. For each of the plurality of second virtual camera positions P12, reference shape information is generated. Here, the reference shape information is also information generated based on the three-dimensional design information related to the object part, and is information related to the two-dimensional shape of the 3D model 3dm of the object part in a virtual image obtained from the photograph of the 3D model 3dm of the object part from the second virtual camera position P12. The virtual image can be generated, for example, by projecting the 3D model 3dm onto a virtual plane using rendering or other processing. A specific example of the reference shape information is a reference image. This step Sb23 corresponds to the second shape information generation step in this invention.

[0381] More specifically, in step Sb23, the second shape information generation unit 6123, based on the three-dimensional design information related to the object part stored in the storage unit 45b, assumes that the 3D model 3dm of the object part is captured from each of the M2×T2 second virtual camera positions P12, and generates reference shape information for each of the M2×T2 second virtual camera positions P12. The M2×T2 second virtual camera positions P12 are the M2×T2 virtual camera positions set by virtually setting one virtual camera position for each of the M2×T2 virtual segmentation surfaces St2 generated in step Sb22. Here, the reference shape information is also information generated based on the three-dimensional design information related to the object part, and is information related to the two-dimensional shape of the 3D model 3dm of the object part in the virtual image obtained by capturing the 3D model 3dm of the object part from the second virtual camera position P12.

[0382] Here, in each virtual segmentation surface St2, the position P12 of the second virtual camera is virtually set at a predetermined position on the virtual segmentation surface St2. The position of the center of the virtual segmentation surface St2 can be applied to the predetermined position. The centroid of the virtual segmentation surface St2 can be applied to the center of the virtual segmentation surface St2. The plurality of virtual segmentation surfaces St2 can be triangular faces.

[0383] <<<<Step Sb24>>>>

[0384] In step Sb24, the second calculation unit 6124 calculates, for each of the plurality of second virtual camera positions P12, a value representing the consistency between the edge image (actual shape information), which is related to the two-dimensional shape of the object captured in the actual image obtained in step Sa4, and the reference image, which is the reference shape information obtained in step Sb23. This step Sb24 corresponds to the second calculation step in this invention.

[0385] As described above, in each of the plurality of virtual surfaces (virtual segmented surfaces) St2 generated by dividing a plurality of virtual surfaces St1 into virtual surfaces St1, a second virtual camera position P12 is set, and for each second virtual camera position P12, a value representing the consistency between the actual shape information and the reference shape information is calculated. Therefore, the high-consistency virtual surface St1m and the plurality of virtual segmented surfaces St2 are not unrelated surfaces, and for the plurality of virtual segmented surfaces St2, an increase in at least one of the number and area can be reduced. As a result, the computational load used to identify the pose of the object part captured in the actual image can be reduced. As a result, the pose identification of the object part can be performed efficiently in the substrate processing apparatus 1.

[0386] More specifically, in step Sb24, the second calculation unit 6124 calculates, for each of the M2×T2 second virtual camera positions P12 virtually set in step Sb23, a value representing the consistency between the edge image (actual shape information) which is related to the two-dimensional shape of the object captured in the actual image and the reference image generated in step Sb23 as reference shape information.

[0387] Here, when the second calculation unit 6124 calculates a value representing the consistency between the edge image (actual shape information) and the reference image (reference shape information) for each second virtual camera position P12, it performs the above-described processes 2a, 2b, and 2c in the following order. Specifically, in step Sb24, the second calculation unit 6124 performs the processes Sb241 to Sb245. The process of step Sb242 is equivalent to the above-described process 2a, the process of step Sb243 is equivalent to the above-described process 2b, and the process of step Sb244 is equivalent to the above-described process 2c.

[0388] In step Sb241, the second calculation unit 6124 specifies one second virtual camera position P12 and the reference image related to the reference shape information of the plurality of second virtual camera positions P12 generated in step Sb23 as the processing objects of steps Sb242 to Sb244.

[0389] In step Sb242, for example, the second calculation unit 6124 uses the RIPOC method to detect the offset of the orientation of the two-dimensional shape of the contour between the edge image, which serves as actual shape information, and the reference image, which serves as reference shape information. Here, for example, the second calculation unit 6124 detects the offset γ2 in the rotation direction in which the consistency of the contour between the reference object region set in step Sb21 and the reference image in the edge image, which serves as actual shape information, is maximized. This offset γ2 is equivalent to the roll angle γ, which maximizes the consistency between the reference image, which serves as reference shape information, and the edge image, which serves as actual shape information.

[0390] In step Sb243, for example, the second calculation unit 6124 rotates the edge image, which serves as actual shape information, by correcting the offset γ2 detected in step Sb242. This generates a rotated edge image in which the two-dimensional shape of the contour in the edge image, which serves as actual shape information, and the two-dimensional shape of the contour in the reference image, which serves as reference shape information, are aligned in orientation. Here, for example, the second calculation unit 6124 rotates the edge image of the reference target area set in step Sb21 by correcting the offset γ2 detected in step Sb242. This generates a rotated edge image in which the two-dimensional shape of the contour in the reference target area, which constitutes actual shape information, and the two-dimensional shape of the contour in the reference image, which serves as reference shape information, are aligned in orientation.

[0391] In step Sb244, for example, the second calculation unit 6124 uses the rotated edge image related to the actual shape information generated in step Sb243 as the object and performs template matching using a reference image as reference shape information. Here, for example, the second calculation unit 6124 scans the reference image within the rotated edge image and detects the position of the region within the rotated edge image where the consistency (similarity) with the reference image is greatest. Thus, in the edge image which is the actual shape information, the position (matching candidate position) of the region with the greatest consistency with the 3dm outline of the object part in the reference image is detected. Here, when performing the above template matching, a value representing the consistency (similarity) when the consistency (similarity) between a region within the rotated edge image and the reference image is greatest is calculated, and this value is used as a value related to the second virtual camera position P12 to represent the consistency between the edge image which is the actual shape information and the reference image which is the reference shape information. Here, the value representing the consistency can also be, for example, the well-known matching score representing consistency.

[0392] In step Sb245, the second calculation unit 6124 determines whether, among the reference shape information related to each of the plurality of second virtual camera positions P12 obtained in step Sb23, there is any second virtual camera position P12 and its related reference shape information that has not yet been specified as the processing object of steps Sb242 to Sb244. Here, if there is second virtual camera position P12 and its related reference shape information that has not yet been specified as the processing object of steps Sb242 to Sb244, the process returns to step Sb241, and the second calculation unit 6124 specifies the next second virtual camera position P12 and its related reference shape information from the plurality of reference shape information related to each of the plurality of second virtual camera positions P12 generated in step Sb23 as the processing object of steps Sb242 to Sb244. On the other hand, if there is no second virtual camera position P12 and related reference shape information that has not yet been specified as the processing object of steps Sb242 to Sb244, then the process proceeds from step Sb245 to step Sb25.

[0393] That is, the second calculation unit 6124 repeats the processing of steps Sb241 to Sb245 until, among the reference shape information related to each of the plurality of second virtual camera positions P12 obtained in step Sb23, the second virtual camera position P12 and the reference shape information related to that second virtual camera position P12, which are not yet specified as processing objects of steps Sb242 to Sb244, disappear. Thus, for each second virtual camera position P12, the second calculation unit 6124 calculates a value representing the consistency between the edge image, which is actual shape information, and the reference image, which is reference shape information.

[0394] <<<<Step Sb25>>>>

[0395] In step Sb25, the second detection unit 6125 detects the virtual camera position P11m with the high consistency detected in step Sb14, and the edge image of the M2×T2 second virtual camera positions P12 obtained in step Sa4 as information related to the two-dimensional shape of the object captured in the actual image (actual shape information), and the virtual camera position with the highest consistency with the reference image generated in steps Sb12 and Sb23 as reference shape information. The virtual camera position detected here becomes the first virtual camera position detected by the second detection unit 6125, that is, the first reference virtual camera position Ps1.

[0396] Here, for example, the matching score calculated in step Sb13 for the high-consistency virtual camera position P11m is compared with the matching score calculated in step Sb24 for each of the M2×T2 second virtual camera positions P12. Furthermore, for example, the virtual camera position with the highest matching score among the high-consistency virtual camera positions P11m and the M2×T2 second virtual camera positions P12 is compared with the edge image obtained in step Sa4, which is related to the two-dimensional shape of the object captured in the actual image (actual shape information), and the reference image, which is the reference shape information.

[0397] <<<<Step Sb31>>>>

[0398] In step Sb31, the search processing unit 61 performs the first A process described above. Here, the segmentation surface generation unit 6122 generates the first M3×T3 virtual segmentation surfaces St3.

[0399] <<<<Step Sb32>>>>

[0400] In step Sb32, the search processing unit 61 performs the first B process described above. Here, the second shape information generation unit 6123 generates reference images as reference shape information for each of the first M3×T3 third virtual camera positions P13 set by virtually setting one virtual camera position for each of the first M3×T3 virtual segmentation surfaces St3 generated in step Sb31. Here, the reference shape information is also information generated based on the three-dimensional design information related to the object part, and is information related to the two-dimensional shape of the 3D model 3dm of the object part in a virtual image obtained by taking a picture of the 3D model 3dm of the object part from the third virtual camera position P13.

[0401] <<<<Step Sb33>>>>

[0402] In step Sb33, the search processing unit 61 performs the first C process described above. Here, the second calculation unit 6124 calculates a value representing the consistency between the edge image (actual shape information) obtained in step Sa4, which is related to the two-dimensional shape of the object captured in the actual image, and the reference image generated in step Sb32, which is related to the reference shape information, for each of the first M3×T3 third virtual camera positions P13 virtually set in step Sb32.

[0403] Here, when the second calculation unit 6124 calculates the value representing the consistency between the edge image, which is actual shape information, and the reference image, which is reference shape information, for each of the third virtual camera positions P13, it performs the above-described processes 2a1, 2b1, and 2c1 in the following order. Specifically, in step Sb33, the second calculation unit 6124 performs the processes Sb331 to Sb335. The process of step Sb332 is equivalent to the above-described process 2a1, the process of step Sb333 is equivalent to the above-described process 2b1, and the process of step Sb334 is equivalent to the above-described process 2c1.

[0404] In step Sb331, the second calculation unit 6124 specifies, in step Sb32, one of the reference images related to the reference shape information of each of the first M3×T3 third virtual camera positions P13, one third virtual camera position P13 and the reference image related to the reference shape information of that one third virtual camera position P13, as the processing objects of steps Sb332 to Sb334.

[0405] In step Sb332, for example, the second calculation unit 6124 uses the RIPOC method to detect the offset of the orientation of the two-dimensional shape of the contour between the edge image, which serves as actual shape information, and the reference image, which serves as reference shape information. Here, the second calculation unit 6124 detects the offset γ21 in the rotation direction in which the consistency of the contour between the reference object region set in step Sb21 and the reference image in the edge image, which serves as actual shape information, is maximized. This offset γ21 is equivalent to the roll angle γ, which maximizes the consistency between the reference image, which serves as reference shape information, and the edge image, which serves as actual shape information.

[0406] In step Sb333, for example, the second calculation unit 6124 rotates the edge image, which serves as actual shape information, by correcting the offset γ21 detected in step Sb332. This generates a rotated edge image in which the two-dimensional shape of the contour in the edge image, which serves as actual shape information, and the two-dimensional shape of the contour in the reference image, which serves as reference shape information, are aligned in orientation. Here, for example, the second calculation unit 6124 rotates the edge image of the reference target area set in step Sb21, which serves as actual shape information, by correcting the offset γ21 detected in step Sb332. This generates a rotated edge image in which the two-dimensional shape of the contour in the reference target area, which constitutes actual shape information, and the two-dimensional shape of the contour in the reference image, which serves as reference shape information, are aligned in orientation.

[0407] In step Sb334, for example, the second calculation unit 6124 uses the rotated edge image related to the actual shape information generated in step Sb333 as the object and performs template matching using a reference image as reference shape information. Here, for example, the second calculation unit 6124 scans the reference image within the rotated edge image and detects the position of the region within the rotated edge image where the consistency (similarity) with the reference image is the greatest. Thus, in the edge image, which is the actual shape information, the position (matching candidate position) of the region with the greatest consistency with the 3dm outline of the object part in the reference image is detected. Here, when performing the above template matching, a value representing the consistency (similarity) when the consistency (similarity) between a region within the rotated edge image and the reference image is the greatest is calculated, and this value is used as a value related to the third virtual camera position P13 to represent the consistency between the edge image, which is the actual shape information, and the reference image, which is the reference shape information. Here, the value representing the consistency can also be, for example, the well-known matching score representing consistency.

[0408] In step Sb335, the second calculation unit 6124 determines whether the reference shape information related to each of the first M3×T3 third virtual camera positions P13 generated in step Sb32 contains reference shape information related to a third virtual camera position P13 that has not yet been specified as a processing object in steps Sb332 to Sb334. If there is reference shape information related to a third virtual camera position P13 that has not yet been specified as a processing object in steps Sb332 to Sb334, the process returns to step Sb331, where the second calculation unit 6124 specifies the next third virtual camera position P13 and its related reference shape information from the reference shape information related to each of the first M3×T3 third virtual camera positions P13 generated in step Sb32 as the processing object in steps Sb332 to Sb334. On the other hand, if there is no reference shape information related to the third virtual camera position P13 that has not yet been specified as the processing object of steps Sb332 to Sb334, then the process proceeds from step Sb335 to step Sb34.

[0409] That is, the second calculation unit 6124 repeats the processing of steps Sb331 to Sb335 until the reference shape information related to each of the first M3×T3 third virtual camera positions P13 generated in step Sb32, where the third virtual camera position P13 and the reference shape information related to it, which are not yet specified as processing objects of steps Sb332 to Sb334, disappear. Thus, the second calculation unit 6124 calculates a value representing the consistency between the edge image (actual shape information) and the reference image (reference shape information) for each of the first M3×T3 third virtual camera positions P13.

[0410] <<<<Step Sb34>>>>

[0411] In step Sb34, the search processing unit 61 performs the first D processing described above. Here, the second detection unit 6125 detects the virtual camera position detected in step Sb25, namely the first reference virtual camera position Ps1, and the virtual camera position Ps2, which has the highest consistency with the reference image generated in step Sb12 or step Sb23 and step Sb32 as reference shape information related to the two-dimensional shape of the object captured in the actual image.

[0412] Here, for example, the matching score calculated for the first reference virtual camera position Ps1 in step Sb13 or step Sb24, and the matching score calculated for each of the first M3×T3 third virtual camera positions P13 in step Sb33 are compared. Furthermore, for example, the virtual camera position with the highest matching score among the first reference virtual camera position Ps1 and the first M3×T3 third virtual camera positions P13, calculated between the edge image (actual shape information), which is related to the two-dimensional shape of the object captured in the actual image, obtained in step Sa4, and the reference image, which is the reference shape information, is detected.

[0413] <<<<Step Sb41>>>>

[0414] In step Sb41, the search processing unit 61 sets the variable n to 2 as the initial value.

[0415] <<<<Step Sb42>>>>

[0416] In step Sb42, the search processing unit 61 performs the aforementioned nth processing. Here, the segmentation surface generation unit 6122 generates the nth M3×T3 virtual segmentation surfaces.

[0417] <<<<Step Sb43>>>>

[0418] In step Sb43, the search processing unit 61 performs the aforementioned nBth processing. Here, the second shape information generation unit 6123 generates a reference image as reference shape information for each of the nth M3×T3 third virtual camera positions set by setting one virtual camera position for each of the nth M3×T3 virtual segmentation surfaces generated in step Sb42. Here, the reference shape information is also information generated based on the three-dimensional design information related to the object part, and is information related to the two-dimensional shape of the 3D model 3dm of the object part in a virtual image obtained by taking a picture of the 3D model 3dm of the object part from the third virtual camera position.

[0419] <<<<Step Sb44>>>>

[0420] In step Sb44, the search processing unit 61 performs the aforementioned nth C-th processing. Here, the second calculation unit 6124 calculates, for each of the nth M3×T3 third virtual camera positions virtually set in step Sb43, a value representing the consistency between the edge image (actual shape information) obtained in step Sa4 as information related to the two-dimensional shape of the object captured in the actual image and the reference image generated in step Sb43 as reference shape information.

[0421] Here, when the second calculation unit 6124 calculates the value representing the consistency between the edge image, which is actual shape information, and the reference image, which is reference shape information, for each of the third virtual camera positions, it performs the above-described processes 2an, 2bn, and 2cn in the following order. Specifically, in step Sb44, the second calculation unit 6124 performs the processes Sb441 to Sb445. The process of step Sb442 is equivalent to the above-described process 2an, the process of step Sb443 is equivalent to the above-described process 2bn, and the process of step Sb444 is equivalent to the above-described process 2cn.

[0422] In step Sb441, the second calculation unit 6124 specifies, in step Sb43, a reference image related to the reference shape information of each of the nth M3×T3 third virtual camera positions generated, a third virtual camera position, and a reference image related to the reference shape information of that third virtual camera position, as the processing objects of steps Sb442 to Sb444.

[0423] In step Sb442, for example, the second calculation unit 6124 uses the RIPOC method to detect the offset of the orientation of the two-dimensional shape of the contour between the edge image, which serves as actual shape information, and the reference image, which serves as reference shape information. Here, the second calculation unit 6124 detects the offset γ2n in the rotation direction in which the consistency of the contour between the reference object region set in step Sb21 and the reference image in the edge image, which serves as actual shape information, is maximized. This offset γ2n is equivalent to the roll angle γ, which maximizes the consistency between the reference image, which serves as reference shape information, and the edge image, which serves as actual shape information.

[0424] In step Sb443, for example, the second calculation unit 6124 rotates the edge image, which serves as actual shape information, by correcting the offset γ2n detected in step Sb442. This generates a rotated edge image in which the two-dimensional shape of the contour in the edge image, which serves as actual shape information, and the two-dimensional shape of the contour in the reference image, which serves as reference shape information, are aligned in orientation. Here, for example, the second calculation unit 6124 rotates the edge image of the reference target area set in step Sb21, which serves as actual shape information, by correcting the offset γ2n detected in step Sb442. This generates a rotated edge image in which the two-dimensional shape of the contour in the reference target area, which constitutes actual shape information, and the two-dimensional shape of the contour in the reference image, which serves as reference shape information, are aligned in orientation.

[0425] In step Sb444, for example, the second calculation unit 6124 uses the rotated edge image related to the actual shape information generated in step Sb443 as the object and performs template matching using a reference image as reference shape information. Here, for example, the second calculation unit 6124 scans the reference image within the rotated edge image and detects the position of the region within the rotated edge image where the consistency (similarity) with the reference image is greatest. Thus, in the edge image which is the actual shape information, the position (matching candidate position) of the region with the greatest consistency with the 3dm outline of the object part in the reference image is detected. Here, when performing the above template matching, a value representing the consistency (similarity) when the consistency (similarity) between a region within the rotated edge image and the reference image is greatest is calculated and used as a value related to the position of the third virtual camera, representing the consistency between the edge image which is the actual shape information and the reference image which is the reference shape information. Here, the value representing the consistency can, for example, be the well-known matching score representing the consistency.

[0426] In step Sb445, the second calculation unit 6124 determines whether the reference shape information related to each of the nth M3×T3 third virtual camera positions generated in step Sb43 contains reference shape information related to the third virtual camera position that has not yet been specified as a processing object in steps Sb442 to Sb444. If there is reference shape information related to the third virtual camera position that has not yet been specified as a processing object in steps Sb442 to Sb444, the process returns to step Sb441, where the second calculation unit 6124 specifies the next third virtual camera position and its related reference shape information from the reference shape information related to each of the nth M3×T3 third virtual camera positions generated in step Sb43 as the processing object in steps Sb442 to Sb444. On the other hand, if there is no third virtual camera position and related reference shape information that has not yet been specified as the processing object of steps Sb442 to Sb444, then the process proceeds from step Sb445 to step Sb45.

[0427] That is, the second calculation unit 6124 repeats the processing of steps Sb441 to Sb445 until the reference shape information related to each of the nth M3×T3 third virtual camera positions generated in step Sb43, where the third virtual camera position and the reference shape information related to that third virtual camera position, which is not yet specified as the processing object of steps Sb442 to Sb444, disappears. Thus, for each of the nth M3×T3 third virtual camera positions, the second calculation unit 6124 calculates a value representing the consistency between the edge image, which is actual shape information, and the reference image, which is reference shape information.

[0428] <<<<Step Sb45>>>>

[0429] In step Sb45, the search processing unit 61 performs the aforementioned nth processing. Here, the second detection unit 6125 detects the virtual camera position detected in step Sb34 or the (n-1)th step Sb45, i.e., the nth reference virtual camera position, and the virtual camera position with the highest consistency between the edge image (actual shape information) related to the two-dimensional shape of the object captured in the actual image and the reference image (reference shape information) among the nth M3×T3 third virtual camera positions virtually set in step Sb43.

[0430] Here, for example, the matching score calculated for the nth reference virtual camera position in steps Sb13, Sb24, Sb33, or the (n-1)th step Sb44, and the matching score calculated for each of the nth M3×T3 third virtual camera positions calculated by step Sb44 are compared. Furthermore, for example, the virtual camera position with the highest matching score among the nth reference virtual camera position and the nth M3×T3 third virtual camera positions is detected, calculated between the edge image (actual shape information), which is related to the two-dimensional shape of the object captured in the actual image, obtained in step Sa4, and the reference image, which is the reference shape information.

[0431] <<<<Steps Sb46, Sb47>>>>

[0432] In step Sb46, the search processing unit 61 determines whether the condition for ending the second search process (also called the termination condition) is met. If the termination condition is not met, in step Sb47, the search processing unit 61 increments the variable n by 1 and returns to step Sb42. On the other hand, if the termination condition is met, the process moves from step Sb46 to step Sb48. That is, the processes from step Sb42 to step Sb46 are repeated until the termination condition is met.

[0433] Here, the termination condition could be, for example, that the second detection unit 6125 continuously detects one of the reference virtual camera positions from the first reference virtual camera position Ps1 to the nth reference virtual camera position for a predetermined first number of times, and uses this position as the virtual camera position with the highest consistency between the actual shape information and the reference shape information. Alternatively, the termination condition could also be that the second search process performs the nth unit of processing for a predetermined second number of times. In other words, the termination condition could also be that the second search process performs the nth unit of processing (n-1 times) to reach the second predetermined number of times.

[0434] <<<<Step Sb48>>>>

[0435] In step Sb48, the search processing unit 61 identifies the result of the search processing. Specifically, the search processing unit 61 detects the virtual camera position with the highest consistency between the actual shape information detected by the second detection unit 6125 and the reference shape information in the final step Sb45 of the second search processing, and takes this as the highest consistency virtual camera position. In other words, the virtual camera position with the highest consistency between the actual shape information detected by the second detection unit 6125 and the reference shape information in the final step Sb45 of the second search processing is the highest consistency virtual camera position. Furthermore, the search processing unit 61 obtains the latitude α, longitude β, and distance D of the highest consistency virtual camera position, and the roll angle γ corresponding to the offset (offset γ1, offset γ2, offset γ21, or offset γ2n) calculated by the first calculation unit 6113 or the second calculation unit 6124 for the highest consistency virtual camera position, and takes this as the result of the search processing. Thus, the pose of the object part captured in the actual image is identified. Here, the search processing unit 61 can, for example, identify the latitude α, longitude β, and distance D of the virtual camera position with the highest consistency as a result of the search processing, and the roll angle γ corresponding to the offset (offset γ1, offset γ2, offset γ21, or offset γ2n) calculated by the first calculation unit 6113 or the second calculation unit 6124 for the virtual camera position with the highest consistency as information related to the posture of the object part.

[0436] Additionally, here, the search processing unit 61 may also obtain, for the virtual camera position with the highest consistency, the matching candidate position (final matching position) detected in steps Sb13, Sb24, Sb33 or Sb44, which has the highest consistency (similarity) with the reference image as reference shape information within the edge image as actual shape information, and use it as part of the result of the search processing.

[0437] Here, the search processing unit 61 obtains real-world information related to the pose of the object part identified using actual images based on the results of the search processing.

[0438] <<Step S5>>

[0439] In step S5, the control unit 45 sets normal information for each object part based on the search processing results obtained for each object part in step S4.

[0440] For example, for nozzle 33, the control unit 45 sets normal information for the confirmation timing when each nozzle 33 is located outside the origin position, based on the search processing results obtained for each nozzle 33 in step S4. More specifically, the control unit 45 establishes a correlation between the three-dimensional information of the posture of each nozzle 33 obtained based on the search processing results obtained when each nozzle 33 is located at the origin position and the origin position of each nozzle 33. Furthermore, the control unit 45 sets normal information for the ejection position of each nozzle 33 based on the number of pulse signals output from the position detection unit 43 when the nozzle moving mechanism 35 moves each nozzle 33 from the origin position to the ejection position and the design information of each nozzle 33.

[0441] For example, for fixture 9, the control unit 45 sets normal information for the confirmation time when each fixture 9 is located outside the origin position, based on the search processing results obtained for each fixture 9 in step S4 and the design information of each fixture 9.

[0442] For example, for the protective component 23, the control unit 45 sets normal information on the timing of confirmation when the protective component 23 is located outside the origin position, based on the results of the search processing of the protective component 23 obtained in step S4 and the design information of the protective component 23.

[0443] <<Step S6>>

[0444] In step S6, the substrate W is moved into the substrate processing apparatus 1. The moving of the substrate W into the substrate processing apparatus 1 is performed, for example, by a transfer robot (not shown in the figure). At this time, each clamp 9 is positioned in the open position.

[0445] <<Step S7>>

[0446] In step S7, the substrate W is processed according to the procedure. Specifically, first, the substrate W is placed on a plurality of clamps 9. The motion control unit 51, for example, activates the clamp drive mechanism 17 via a clamp motion command, causing each clamp 9 to move from the open position to the closed position. This state is, for example, as follows: ​ As shown. That is, each clamp 9, with the substrate W on it, rotates around the rotation center PL2, and the peripheral support 13 moves to the rotation center PL1 side of the rotary chuck 3. Thus, the peripheral support 13 of each clamp 9 abuts against the outer diameter of the substrate W, and the substrate W is clamped by the plurality of clamps 9. At this time, when viewed from above, the peripheral support 13 is located at a relatively... ​ The peripheral support portion 13 is located slightly to the outer periphery. This outer periphery may be the outer periphery of the rotary chuck 3.

[0447] <<Step S8>>

[0448] In step S8, the motion control unit 51 confirms whether the target part is in a confirmation period. Specifically, the motion control unit 51 refers to the target part and the confirmation period in the parameter information 55. Here, if the target part is not in a confirmation period, the process proceeds from step S8 to step S14; if the target part is in a confirmation period, the process proceeds from step S8 to step S9. In step S8, for example, if the fixture 9, which is the target part, is in a confirmation period where the fixture 9 is in the closed position, the process proceeds from step S8 to step S9.

[0449] <<Step S9>>

[0450] In step S9, the motion control unit 51 causes the camera CM to take a picture. Specifically, for example, the motion control unit 51 causes the camera CM to move at the timing when each clamp 9 moves to the closed position according to the clamp action command. At this time, the camera CM acquires an actual image containing each clamp 9 by taking a picture.

[0451] <<Step S10>>

[0452] In step S10, the image processing unit 59 performs image processing on the actual image acquired by the camera CM in step S9, through the same process as in step S3 described above. In this step S10, the process is performed sequentially, similar to step S3 described above. ​ The processing of steps Sa1 to Sa4.

[0453] Specifically, in step Sa1, the image processing unit 59 acquires the actual image obtained by the camera CM in step S9. This step Sa1 corresponds to the actual image acquisition step in this invention. In step Sa2, the processing target area extraction unit 591 of the image processing unit 59 sets a processing target area for the actual image. Here, for example, a processing target area is set for each fixture 9, which is a component. In step Sa3, the processing target area extraction unit 591 of the image processing unit 59 extracts the portion related to the processing target area from the actual image as an image (processing target actual image). In step Sa4, the contour extraction unit 592 of the image processing unit 59 performs contour extraction processing on all components captured in the processing target actual image. As a result, an edge image is obtained, which is information related to the two-dimensional shape of the object captured in the actual image (actual shape information). More specifically, an edge image is obtained, which is information related to the two-dimensional shape of the object including the fixture 9 (actual shape information). Here, for example, an edge image is obtained for each fixture 9, which is a component.

[0454] <<Step S11>>

[0455] In step S11, the search processing unit 61 performs a search process on each fixture 9, which is an object part, through the same process as in step S4 described above. Here, the edge images related to the actual shape information of each fixture 9, which is an object part, obtained in step S10, are used. This step S11 is the same as step S4 described above, and is equivalent to the search step performed by the calculation unit 45a in this invention. In addition, here, based on the result of the search processing, the search processing unit 61 obtains real-world information related to the posture of each fixture 9, which is an object part.

[0456] <<Step S12>>

[0457] In step S12, the anomaly detection unit 63 detects anomalies in each fixture 9, which are each target part, based on the real-time information obtained in step S11. Here, the anomaly detection unit 63 compares the real-time information with normal information for each fixture 9, which are each target part. The anomaly detection unit 63 detects an anomaly when the comparison result between the real-time information and the normal information for each fixture 9, which are each target part, is inconsistent.

[0458] <<Step S13>>

[0459] In step S13, the control unit 45 determines whether an abnormality was detected in each fixture 9, which is the target part, in step S12. If no abnormality was detected in each fixture 9, which is the target part, in step S12, the process proceeds from step S13 to step S14. Conversely, if an abnormality was detected in the fixture 9, which is the target part, in step S12, the process proceeds from step S13 to step S17.

[0460] <<Step S14>>

[0461] In step S14, the control unit 45 determines whether the processing of the substrate W is complete. If the processing of the substrate W is complete, the process proceeds from step S14 to step S15. Conversely, if the processing of the substrate W is not complete, the process returns from step S14 to step S7. For example, if the substrate W is merely placed on a plurality of clamps 9 and held by the plurality of clamps 9, the processing of the substrate W is not complete, and the process returns from step S14 to step S7.

[0462] <<Step S7>>

[0463] In step S7, the substrate W is processed according to the procedure. Specifically, for example, the motion control unit 51 activates the protective member moving mechanism 25 by issuing a protective member motion command, so that... ​ The protective component 23, located at the origin, is shown as follows: ​ The device moves to the processing location as shown. At that time point, ​The nozzle 33, located above the substrate W, is at the origin position.

[0464] <<Step S8>>

[0465] In step S8, as described above, the motion control unit 51 confirms whether the target part is in a confirmation period. Specifically, the motion control unit 51 refers to the target part and the confirmation period in the parameter information 55. Here, if the target part is not in a confirmation period, the process proceeds from step S8 to step S14; if the target part is in a confirmation period, the process proceeds from step S8 to step S9. In step S8, for example, if the protective member 23, which is the target part, is in a confirmation period where the protective member 23 is in the processing position, the process proceeds from step S8 to step S9.

[0466] <<Step S9>>

[0467] In the second step S9, the motion control unit 51 causes the camera CM to take a picture. Specifically, for example, the motion control unit 51 causes the camera CM to move when the protective member 23 is raised by the protective member moving mechanism 25 according to the protective member movement command, thus completing the movement of the protective member 23 to the processing position. At this time, the camera CM acquires an actual image containing the protective member 23 by taking a picture.

[0468] <<Step S10>>

[0469] In the second step S10, the image processing unit 59 performs image processing on the actual image captured by the camera CM in the second step S9, through the same processing as in the first step S10 described above. This second step S10, like the first step S10, proceeds sequentially. ​ The processing of steps Sa1 to Sa4.

[0470] Specifically, in step Sa1, the image processing unit 59 acquires the actual image obtained from the camera CM during the second capture in step S9. This step Sa1 corresponds to the actual image acquisition step in this invention. In step Sa2, the processing target area extraction unit 591 of the image processing unit 59 sets a processing target area for the actual image. Here, the processing target area is set for the protective member 23, which is the target part. In step Sa3, the processing target area extraction unit 591 of the image processing unit 59 extracts the portion related to the processing target area from the actual image as an image (processing target actual image). In step Sa4, the contour extraction unit 592 of the image processing unit 59 performs contour extraction processing on all parts captured in the processing target actual image. As a result, an edge image is obtained, which is information related to the two-dimensional shape of the object captured in the actual image (actual shape information). More specifically, an edge image is obtained, which is information related to the two-dimensional shape of the object containing the protective member 23 (actual shape information).

[0471] <<Step S11>>

[0472] In the second step S11, the search processing unit 61 performs a search process on the protective member 23, which is the target part, through the same process as in step S4 described above. Here, the edge image, which is the actual shape information related to the protective member 23, which is the target part, obtained in the second step S10, is used. This step S11 is the same as step S4 described above, and is equivalent to the search step performed by the calculation unit 45a in this invention. In addition, here, based on the result of the search processing, the search processing unit 61 obtains the actual information related to the posture of the prote...

Claims

1. A substrate processing apparatus which performs processing of a substrate, wherein Possessing: a storage section that stores three-dimensional design information related to an object part; a photographing section that obtains an actual image that captures the object part by photographing; and a search processing section that searches for a virtual camera position at which a degree of coincidence between reference shape information and actual shape information is the greatest, among a plurality of virtual camera positions, based on the reference shape information and the actual shape information, the reference shape information being information related to a two-dimensional shape of a three-dimensional model in each of a plurality of virtual images that can be obtained by photographing the three-dimensional model from the plurality of virtual camera positions, which are respectively generated based on the three-dimensional design information, the actual shape information being information related to a two-dimensional shape of an object in the actual image; the search processing section including: a first shape information acquisition section that acquires the reference shape information generated for each of a plurality of first virtual camera positions, based on the three-dimensional design information, assuming photographing of the three-dimensional model from each of the plurality of first virtual camera positions, the plurality of first virtual camera positions being a plurality of virtual camera positions that are virtually set by virtually setting one virtual camera position for each of a plurality of virtual faces that virtually surround the three-dimensional model along a virtual sphere that surrounds the three-dimensional model with a reference point of the three-dimensional model as a center; a first calculation section that calculates, for each of the plurality of first virtual camera positions, a value that indicates a degree of coincidence between the actual shape information and the reference shape information; a first detection section that detects, based on a result of the calculation by the first calculation section, a first virtual camera position at which the degree of coincidence between the actual shape information and the reference shape information is the greatest, among the plurality of first virtual camera positions, as a high-coincidence virtual camera position; a divided face generation section that generates a plurality of virtual divided faces by dividing a virtual face in which the high-coincidence virtual camera position is virtually set, among the plurality of virtual faces; a second shape information generation section that generates the reference shape information for each of a plurality of second virtual camera positions, based on the three-dimensional design information, assuming photographing of the three-dimensional model from each of the plurality of second virtual camera positions, the plurality of second virtual camera positions being a plurality of virtual camera positions that are virtually set by virtually setting one virtual camera position for each of the plurality of virtual divided faces; and a second calculation section that calculates, for each of the plurality of second virtual camera positions, a value that indicates a degree of coincidence between the actual shape information and the reference shape information.

2. The substrate processing apparatus according to claim 1, wherein The first shape information acquisition section acquires the reference shape information generated for each of the M1xT1 first virtual camera positions, based on the three-dimensional design information, assuming that the three-dimensional model is imaged from each of the M1xT1 first virtual camera positions, the M1xT1 first virtual camera positions being M1xT1 virtual camera positions set by setting one virtual camera position for each of the M1 virtual faces of each of the T1 face assemblies, the T1 face assemblies being virtually set so as to have distances from the reference point different from each other, T1 being a natural number of 2 or more, and M1 being a natural number of 2 or more; The first calculation section calculates, for each of the M1xT1 first virtual camera positions, a value indicating the degree of coincidence between the actual shape information and the reference shape information; The first detection section detects, based on the calculation result of the first calculation section, a virtual camera position, among the M1xT1 first virtual camera positions, at which the degree of coincidence between the actual shape information and the reference shape information is the highest, as the high-coincidence virtual camera position; The divided face generation section divides, by the same rule, each of T2 virtual faces included in the high-coincidence virtual face and on the side of the high-coincidence virtual camera position from the reference point, intersecting a straight line passing through the reference point and the high-coincidence virtual camera position, and having distances from the reference point different from each other, among the M1 virtual faces of each of the T1 face assemblies, and generates M2xT2 virtual divided faces by generating M2 virtual divided faces for each of the T2 virtual faces, T2 being a natural number of 2 or more, and M2 being a natural number of 2 or more; The second shape information generation section generates, for each of the M2xT2 second virtual camera positions, the reference shape information, based on the three-dimensional design information, assuming that the three-dimensional model is imaged from each of the M2xT2 second virtual camera positions, the M2xT2 second virtual camera positions being M2xT2 virtual camera positions set by setting one virtual camera position for each of the M2xT2 virtual divided faces; The second calculation section calculates, for each of the M2xT2 second virtual camera positions, a value indicating the degree of coincidence between the actual shape information and the reference shape information.

3. The substrate processing apparatus according to claim 2, wherein The search processing section includes: a second detection section that detects the high-coincidence virtual camera position and a virtual camera position, among the M2xT2 second virtual camera positions, at which the degree of coincidence between the actual shape information and the reference shape information is the highest.

4. The substrate processing apparatus according to claim 3, wherein The search processing section executes the n-th unit processing one or more times after executing the first unit processing on the target part, n being a natural number of 2 or more; The search processing section sequentially performs a first A processing, a first B processing, a first C processing, and a first D processing in the first unit processing; The first A processing is a processing of: The above-mentioned division surface generating section divides each of the T3 virtual division surfaces, i.e., each of the T3 virtual division surfaces including a virtual division surface including the virtual camera position detected by the above-mentioned second detecting section for the first time, i.e., the first reference virtual camera position, and intersecting a straight line passing through the reference point and the first reference virtual camera position on the side of the first reference virtual camera position farther from the reference point, and having a distance from the reference point different from each other, by the same rule, generates M3 virtual division surfaces, i.e., the M3 virtual division surfaces for each of the T3 virtual division surfaces for the first time, and generates M3 x T3 virtual division surfaces, i.e., the M3 x T3 virtual division surfaces for the first time, T3 being a natural number of 2 or more, and M3 being a natural number of 2 or more; The above-mentioned first B processing is the following processing: The above-mentioned second shape information generating section assumes that the three-dimensional model is photographed from each of the M3 x T3 third virtual camera positions for the first time based on the above-mentioned three-dimensional design information, and generates the above-mentioned reference shape information for each of the M3 x T3 third virtual camera positions for the first time, the M3 x T3 third virtual camera positions for the first time being the M3 x T3 virtual camera positions set by virtually setting one virtual camera position for each of the M3 x T3 virtual division surfaces for the first time; The above-mentioned first C processing is the following processing: The above-mentioned second calculating section calculates a value indicating the degree of coincidence between the above-mentioned actual shape information and the above-mentioned reference shape information for each of the M3 x T3 third virtual camera positions for the first time; The above-mentioned first D processing is the following processing: The above-mentioned second detecting section detects the virtual camera position, i.e., the second reference virtual camera position, having the greatest degree of coincidence between the above-mentioned actual shape information and the above-mentioned reference shape information among the first reference virtual camera position and the M3 x T3 third virtual camera positions for the first time; The above-mentioned search processing section sequentially performs the nA processing, the nB processing, the nC processing, and the nD processing in each of the one or more nth unit processes; The above-mentioned nA processing is the following processing: The above-mentioned division surface generating section divides each of the T3 virtual division surfaces for the nth time, i.e., each of the T3 virtual division surfaces for the nth time including a virtual division surface including the virtual camera position detected by the above-mentioned second detecting section for the nth time, i.e., the nth reference virtual camera position, and intersecting a straight line passing through the reference point and the nth reference virtual camera position on the side of the nth reference virtual camera position farther from the reference point, and having a distance from the reference point different from each other, by the same rule, generates M3 virtual division surfaces, i.e., the M3 virtual division surfaces for the nth time for each of the T3 virtual division surfaces for the nth time, and generates M3 x T3 virtual division surfaces, i.e., the M3 x T3 virtual division surfaces for the nth time; The above-mentioned nB processing is the following processing: The first shape information generating section generates, based on the three-dimensional design information, the reference shape information for each of the M3 x T3 third virtual camera positions, the M3 x T3 third virtual camera positions being the M3 x T3 virtual camera positions set by virtually setting one virtual camera position for each of the M3 x T3 virtual division surfaces; The nC processing is the following processing: The second calculating section calculates, for each of the M3 x T3 third virtual camera positions, a value indicating the degree of coincidence between the actual shape information and the reference shape information; The nD processing is the following processing: The second detecting section detects, among the n-th reference virtual camera position and the M3 x T3 third virtual camera positions, a virtual camera position at which the degree of coincidence between the actual shape information and the reference shape information is the greatest.

5. The substrate processing apparatus of claim 4, wherein, The search processing section ends the execution of the n-unit processing of one time or more in response to the detection, by the second detecting section, of one reference virtual camera position from among the first reference virtual camera position to the n-th reference virtual camera position as the virtual camera position at which the degree of coincidence between the actual shape information and the reference shape information is the greatest, for a first predetermined number of times set in advance.

6. The substrate processing apparatus of claim 4, wherein, The search processing section ends the execution of the n-unit processing of one time or more in response to the execution of the n-unit processing among the n-unit processing of one time or more for a second predetermined number of times set in advance.

7. The substrate processing apparatus of any of claims 4 to 6, wherein, Further comprising: an abnormality detecting section that detects an abnormality of the object part by comparing real information related to the posture of the object part identified based on the virtual camera position at which the degree of coincidence between the actual shape information and the reference shape information is the greatest, which is the n+1-th detected by the second detecting section in the last nD processing among the n-unit processing of one time or more, with normal information related to the posture of the object part based on the three-dimensional design information when the state of the object part is normal.

8. The substrate processing apparatus according to any one of claims 2 to 6, wherein The same rule includes the following rule: The division object surface is divided into a plurality of surfaces by connecting a plurality of line segments each connecting a center point of the division object surface to all vertices of the division object surface, respectively.

9. The substrate processing apparatus according to any one of claims 1 to 6, wherein Each of the plurality of virtual surfaces is a triangular surface, The surface aggregate is a polyhedron composed of a plurality of triangular surfaces.

10. The substrate processing apparatus according to claim 9, wherein The division surface generating section divides the high-coincidence virtual surface into three virtual division surfaces as the plurality of virtual division surfaces by connecting three line segments each connecting three vertices of the high-coincidence virtual surface to the three line segments of the high-coincidence virtual camera position, respectively.

11. An information processing method of a substrate processing apparatus, which is an information processing method of a substrate processing apparatus that performs processing of a substrate, wherein has: An actual image acquisition step acquires, by the arithmetic unit, an actual image of the object part captured by the photographing unit; A search step searches, by the arithmetic unit, for a virtual camera position at which the degree of coincidence between the reference shape information and the actual shape information is the highest, among the plurality of virtual camera positions, based on the reference shape information and the actual shape information; the reference shape information is information about the two-dimensional shape of the three-dimensional model included in each of the plurality of virtual images that can be acquired by photographing the three-dimensional model of the object part from the plurality of virtual camera positions, which are virtually set based on the three-dimensional design information stored in the storage unit; and the actual shape information is information about the two-dimensional shape of the object in the actual image; The search step includes: A first shape information acquisition step acquires the reference shape information generated for each of the plurality of first virtual camera positions, based on the three-dimensional design information, assuming that the three-dimensional model is photographed from each of the plurality of first virtual camera positions; the plurality of first virtual camera positions are a plurality of virtual camera positions set by virtually setting one virtual camera position for each of the plurality of virtual faces included in a virtual face aggregate that virtually surrounds the three-dimensional model along a virtual sphere centered on a reference point of the three-dimensional model; A first calculation step calculates, for each of the plurality of first virtual camera positions, a value indicating the degree of coincidence between the actual shape information and the reference shape information; A first detection step detects, based on the calculation result of the first calculation step, a first virtual camera position at which the degree of coincidence between the actual shape information and the reference shape information is the highest, among the plurality of first virtual camera positions, as a high-coincidence virtual camera position; A split face generation step generates a plurality of virtual split faces by splitting a virtual face of the plurality of virtual faces in which the high-coincidence virtual camera position is virtually set, as a high-coincidence virtual face; A second shape information generation step generates the reference shape information for each of the plurality of second virtual camera positions, based on the three-dimensional design information, assuming that the three-dimensional model is photographed from each of the plurality of second virtual camera positions; the plurality of second virtual camera positions are a plurality of virtual camera positions set by virtually setting one virtual camera position for each of the plurality of virtual split faces; and A second calculation step calculates, for each of the plurality of second virtual camera positions, a value indicating the degree of coincidence between the actual shape information and the reference shape information.

Images